Abstract
Objective:
To develop and validate a patient-specific multivariable prediction model that utilizes variables readily available in the electronic medical record to predict 12-month mobility at the time of initial post-amputation prosthetic prescription. The prediction model is designed for patients who have undergone their initial transtibial (TT) or transfemoral (TF) amputation due to complications of diabetes and/or peripheral artery disease.
Design:
Multi-methodology cohort study that identified patients retrospectively through a large Veteran’s Affairs (VA) dataset then prospectively collected their patient-reported mobility.
Setting:
The VA Corporate Data Warehouse (CDW), the National Prosthetics Patient Database, participant mailings and phone calls.
Participants:
357 Veterans who underwent an incident dysvascular TT or TF amputation and received a qualifying lower limb prosthesis (LLP) between March 1, 2018, and November 30, 2020.
Interventions:
NA
Main Outcome Measure:
The Amputee Single Item Mobility Measure (AMPSIMM) was divided into a 4-category outcome to predict wheelchair mobility (0-2), and household (3), basic community (4), or advanced community ambulation (5-6).
Results:
Multinomial logistic lasso regression, a machine learning methodology designed to select variables that most contribute to prediction while controlling for overfitting, led to a final model including 23 predictors of the 4-category AMPSIMM outcome that effectively discriminates household ambulation from basic community ambulation and from advanced community ambulation – levels of key clinical importance when estimating future prosthetic demands. The overall model performance was modest as it did not discriminate wheelchair from household mobility as effectively.
Conclusions:
The AMPREDICT PROsthetics model can assist providers in estimating individual patients’ future mobility at the time of prosthetic prescription, thereby aiding in the formulation of appropriate mobility goals, as well as facilitating the prescription of a prosthetic device that is most appropriate for anticipated functional goals.
Keywords: Dysvascular, diabetes, peripheral artery disease, lower extremity amputation, lower limb prosthesis, mobility prediction model
Introduction
The ability to ambulate after lower limb amputation (LLA) is an important part of daily function and a key contributor to quality of life.1 Greater prosthesis use among persons with LLA is associated with enhanced clinical outcomes, including higher levels of function and independence via improved self-care and mobility,2 improved quality of life,3–8 and employment success.9
The current paradigm for prescription of a lower limb prosthesis (LLP) is largely driven by a U.S. classification, which is commonly described as the Medicare Functional Classification Levels “MFCL,” or “K-levels”. This mobility classification system is the current “gold standard” for establishing the medical necessity of specific complexities of prosthetic feet, knees, and ankles.10 However, K-levels rely solely on clinicians to estimate future mobility in prosthetic candidates based upon their clinical experience; therefore, the validity of their use is uncertain, leaving a significant gap in prosthetic prescription decision making.
The primary purpose of this study was to develop and validate a patient-specific multivariable prediction model (AMPREDICT PROsthetics) that predicts mobility outcome at 12-months post prescription, using a composite of easily accessible risk factors. The prediction model was designed to be applied to patients who have undergone their first transtibial (TT) or transfemoral (TF) amputation from complications due to diabetes and/or peripheral artery disease (PAD). The model is designed to empirically support the complex decisions around prosthetic prescriptions and rehabilitation planning; by specifying future mobility probability in an evidence-informed prediction model, providers can prescribe prosthetic limb componentry that more accurately matches an individual patient’s future prosthetic mobility, and rehabilitation teams will be better able to set evidence-based goals and inform patient expectations.
Methods
Study design
This was a multi-methodology cohort study that identified persons with LLA retrospectively through a large VA dataset, then prospectively collected their self-reported mobility.
Study sample
The Veterans Affairs Corporate Data Warehouse (CDW) is a repository comprising data from multiple VA clinical and administrative systems. The CDW was used to identify patients who were age 40 years and older, undergoing their first amputation at the TT (CPT 27880, 27881) or TF (CPT 27590, 27591) level. Only amputations performed as a result of complications due to diabetes or PAD confirmed by principal ICD-9-CM codes were included. Patients were excluded if the amputation was due to trauma, or if they required bilateral amputation or a contralateral amputation after incident amputation prior to prosthetic prescription. Patients were also excluded if they had a diagnosis of paraplegia, quadriplegia, spinal cord injury, dementia, or a body mass index (BMI) <15 or >52 kg/m2. These exclusions ensured that the amputation was a result of vascular disease and that patients had the potential for ambulatory mobility. In the case of BMI, values outside this range are likely to be implausible owing to data entry errors.11 For patients meeting criteria, we looked back 5 years to ensure that there was no prior diagnostic or procedure code for an amputation at the transmetatarsal level or higher, or its treatment,12–14 which enabled us to exclude patients undergoing a reamputation whose prosthetic prescription may be related to a prior amputation. Participants were eligible if not excluded based on criteria listed above and received a qualifying prosthetic prescription within 12 months of their incident amputation between March 1, 2018, and November 30, 2020. A qualifying prosthetic prescription, obtained via the National Prosthetics Patient Database, included Health Care Common Procedure Coding System (HCPCS) codes listed in a publication under review (ranging from L51xx-L53xx, L55xx-L56xx, and L58xx-L59xx).15 These codes were selected to ensure this represented a definitive prosthesis for rehabilitation and home use rather than an immediate post-op prosthesis, replacement, or repair. Participants who received a qualifying prosthetic prescription but died within the following 12 months were excluded (because we would be unable to obtain their mobility outcome), as were those who had a single code for an initial postoperative dressing or a repair to, or supplies for, a previous prosthesis.
Patient recruitment
A study coordinator employed a modification of the Total Design Method to contact patients.16 This process was performed in monthly waves. Patients were contacted in chronological order of their prosthetic prescription first by phone interview, and, if they could not be reached, by mail written questionnaire, to obtain their mobility outcome at 12-months (10-4 months) post-prosthetic prescription. All procedures were approved by the local institutional review board.
Predictors
The prediction model development evaluated factors available within the electronic heath record that fell into several key domains based on the clinical experience of an expert panel and the published literature. All predictors were identified through the CDW using corresponding ICD-9 and −10 codes and CPT codes where appropriate. All 33 candidate predictors preceded the prosthetic prescription, which was considered time zero for the prediction model (Table 1). We did not include sex or race in the prediction model as these may result in inappropriate conclusions about their contribution to outcome, leading to ill-founded treatment provision or prosthesis prescription.
Table 1.
Description of all candidate predictors evaluated for the AMPREDICT PROsthetics prediction model.
| Predictors* | Description (when applicable) |
|---|---|
| Amputation and prosthetic prescription | |
| Amputation level | At time of prosthetic prescription |
| Ipsilateral reamputation | Between incident amputation and prosthetic prescription |
| Time to prosthetic prescription | From incident amputation |
| Demographics † | |
| Age | Years |
| Body mass index (BMI) | Kg/m2 |
| Marital status | Married versus not |
| Urban versus rural/highly rural | Current living environment |
| Comorbidities † | |
| Diabetes | |
| Peripheral artery disease (PAD) | Atherosclerosis of native arteries or bypass graft of the lower extremities with claudication, rest pain, ulceration or gangrene |
| Asthma | |
| Chronic obstructive pulmonary disease (COPD) | |
| Coronary artery disease (CAD) | |
| Congestive heart failure (CHP) | |
| Kidney failure | |
| Stroke | |
| Chronic liver disease | Categorized as alcoholic, non-alcoholic, and “any” which combined the two |
| Peripheral neuropathy | |
| Myocardial infarction | In the past 6 months |
| Dialysis | Currently |
| Mental health and health behaviors ‡ | |
| Mild cognitive impairment | |
| Post-traumatic stress disorder (PTSD) | |
| Schizophrenia/psychosis | Combined into a single predictor |
| Bipolar disorder | |
| Anxiety | |
| Depression | Categorized as no depression, depressive without major disorder, and major depressive disorder (MDD) |
| Tobacco use disorder | |
| Cocaine/opioid use disorder | |
| Alcohol misuse | Most recent Alcohol Use Disorders Identification Test (AUDIT-C) score) prior to prosthetic prescription categorized as mild, moderate, and severe separately by male and female29 |
| Prior revascularization procedure § | |
| Any prior revascularization | Included ipsilateral, contralateral, open and endovascular without a time component |
| Ipsilateral open | Categorized as none, recent (≤3 months) or distant (>3 months) prior to prosthetic prescription |
| Ipsilateral endovascular | Same as above |
| Contralateral open | Same as above |
| Contralateral endovascular | Same as above |
A set of fractional polynomial terms was considered for all continuous predictors (time to prosthetic prescription, age, and BMI) to allow for nonlinear associations with (logit) risk.30
Race and sex information were collected, but were not included in the prediction model development because the number of eligible females was too small, and race may simply be a surrogate for other predictors.
History of ever/never being diagnosed in the past except for myocardial infarction and dialysis
Based on clinical experience that these procedures may influence healing, prosthetic fitting, and ultimately prosthetic mobility
Outcome
The 7-item Amputee Single Item Mobility Measure (AMPSIMM) was developed to classify a broad range of mobility in individuals following a dysvascular amputation. It includes wheeled mobility and a finer distinction of home versus community ambulation. It has demonstrated strong criterion and construct validity, excellent responsiveness and floor/ceiling characteristics.17 For purposes of mobility prediction, the AMPSIMM was divided into a 4-category outcome to predict wheelchair mobility (0-2), and household (3), basic community (4), or advanced community ambulation (5-6).
Model development and validation
For the purposes of data familiarization and to identify potential data errors, the candidate predictors were initially summarized individually as well as bivariately with the 4-category AMPSIMM outcome.
Multinomial logistic regression, where predictors can potentially play a different role at each level of mobility, was used to fit prediction models for the AMPSIMM outcome. We considered interactions of age, BMI, chronic obstructive pulmonary disease (COPD), marital status, depression, dialysis, and congestive heart failure (CHF) with amputation level, as the effect of these conditions on mobility outcome may differ by amputation level. Variable selection was performed using lasso, a machine learning methodology that identifies models which optimize future predictive performance. The lasso leads to more parsimonious models, tuned to reduce prediction error by avoiding overfitting.18 Ten-fold cross validation was used to identify lasso tuning parameters according to two standard alternative criteria (“min” and “1SE”).19 Both “grouped” and “ungrouped” lasso models were considered, where the “grouped” method restricts selection to the same set of predictors for each of the 4 outcome categories and the ungrouped method removes this restriction. Variable selection using bi-directional stepwise selection was also undertaken.
Data were missing for tobacco use (n=22; 6.2%); multiple imputations for the missing values were performed using sequential regressions against all other predictors and the outcome. Variable selection was performed simultaneously with imputation by stacking the imputed data and weighting each observation by its proportion of non-missing variables.20
To assess the fitted prediction models, discrimination plots were assessed visually and several quantitative measures of discrimination, analogous to c-indices, were estimated.21 The polytomous discrimination index (PDI), which simultaneously compares sets rather than pairs of outcomes, was also considered.21 The PDI for random performance with a 4-category outcome would be 0.25, whereas for c-indices it is 0.5.
The selected final model was validated internally with bootstrap sampling to obtain estimates for the optimism (discrepancy due to overfitting) of the estimated discrimination measures. As a sensitivity analysis, direct internal validation was also performed using 10-fold cross validation. All statistical analyses were carried out using R statistical software with lasso regression performed using the glmnet package.18,22
Results
Recruitment of participants
Among the consecutive 6,620 TT and TF persons with an amputation identified due to PAD or diabetes, 3,758 (57%) were excluded for not representing patients with an incident TT or TF amputation, possessing comorbidities that are unikely to make them candidates for a prosthesis, or died in the first year prior to a prosthesis prescription. Among the remaining 2,862, 1,690 (59%) received a qualifying prosthesis and 1,172 (41%( did not, Figure 1. The 1,690 patients were approached to obtain their mobility outcomes. We initiated 1,352 phone calls and mailed questionaires to the 682 patients who did not answer and to 131 who requested a mailing. We conducted 256 telephone interviews with 243 being completed, and received 114 completed mailed questionaires. The total analysis sample included 357 patients (266 TT and 91 TF amputees).
Figure 1:
Strobe diagram depicting total numbers identified in the VA Corporate Data Warehouse, total numbers and reasons for exclusion, and final number enrolled.
*All transtibial and transfemoral amputations due to PAD or diabetes identified through the corporate Data Warehouse during March 2018-September 2020.
**Exclusions on left side of the strobe are in the order they were applied horizontally from left to right. Among the 4,930 excluded, the final exclusion was for those that did not receive a prescription (n= 1,172). leaving 1,690 with a prescription.
aPatients with spinal cord injury codes that did not have a hemi or quadriplegia code †Did not meet any other exclusion criteria; however, did not receive a prosthetic prescription.
††Prior to their 12-month mobility outcome follow-up window
**Those otherwise eligible who were past their follow-up window due to timing of identification or Covid-19 study delays.
***28 recruits fell out of contact window before questionnaires could be mailed.
Participant characteristics
The distributions of candidate predictors by AMPSIMM category are summarized in Table 2. For eligible patients who consented to participate, did not consent to participate, or died prior to entering their 12-month outcome window, a comparison summary of candidate predictors is provided in Supplementary Table 1. Of note, those eligble who did not consent to participate were very similar to those who consented. Those who died prior to reaching their mobility outcome window had a greater proportion of comorbidites.
Table 2.
Distribution of candidate predictors by AMPSIMM category for enrolled participants (N= 357)
| Variable | AMPSIMM 0-2 (n=72) | AMPSIMM 3 (n=63) | AMPSIMM 4 (n=110) | AMPSIMM 5-6 (n=112) |
|---|---|---|---|---|
|
| ||||
| n (%) or Mean (SD) | ||||
| Amputation Level | ||||
| TT | 52 (72%) | 41 (65%) | 73 (66%) | 100 (89%) |
| TF | 20 (28%) | 22 (35%) | 37 (34%) | 12 (11%) |
| Time to Prosthetic Prescription (days)* | 120.9 (80.8) | 110.3 (63.5) | 91.5 (66.4) | 93.3 (61.3) |
| Age (years)* | 67.1 (7.7) | 66.7 (6.4) | 65.8 (8.6) | 61.9 (8.7) |
| Race/Ethnicity | ||||
| Black | 18 (25%) | 22 (35%) | 24 (22%) | 16 (14%) |
| Hispanic | 4 (6%) | 3 (5%) | 2 (2%) | 5 (4%) |
| White | 39 (54%) | 37 (59%) | 76 (69%) | 84 (75%) |
| Other | 5 (7%) | 0 (0%) | 1 (1%) | 2 (2%) |
| Missing | 6 (8%) | 1 (2%) | 7 (6%) | 5 (4%) |
| Sex | ||||
| Male | 70 (97%) | 59 (94%) | 110 (100%) | 110 (98%) |
| Female | 2 (3%) | 4 (6%) | 0 (0%) | 2 (2%) |
| Marital Status | ||||
| Married | 35 (49%) | 35 (56%) | 48 (44%) | 45 (40%) |
| Not Married | 37 (51%) | 28 (44%) | 62 (56%) | 67 (60%) |
| Urban-Rural Classification | ||||
| Rural | 26 (36%) | 24 (38%) | 42 (38%) | 44 (39%) |
| Urban | 46 (64%) | 39 (62%) | 68 (62%) | 68 (61%) |
| BMI (kg/m2)* | 29.5 (7.5) | 29.9 (5.4) | 29.7 (7.0) | 30.8 (5.6) |
| Peripheral Artery Disease | 54 (75%) | 49 (78%) | 69 (63%) | 58 (52%) |
| Coronary Atherosclerosis | 48 (67%) | 44 (70%) | 50 (45%) | 48 (43%) |
| Heart Failure | 36 (50%) | 31 (49%) | 33 (30%) | 31 (28%) |
| Recent myocardial Infarction† | 5 (7%) | 6 (10%) | 7 (6%) | 10 (9%) |
| Diabetes | 64 (89%) | 61 (97%) | 91 (83%) | 106 (95%) |
| Kidney Dialysis | 8 (11%) | 11 (17%) | 5 (5%) | 3 (3%) |
| Chronic Obstructive Pulmonary Disease | 34 (47%) | 32 (51%) | 40 (36%) | 27 (24%) |
| Stroke | 20 (28%) | 18 (29%) | 19 (17%) | 15 (13%) |
| Chronic Liver Disease, alcoholic | 3 (4%) | 2 (3%) | 3 (3%) | 4 (4%) |
| Chronic Liver Disease, non-alcoholic | 9 (12%) | 6 (10%) | 11 (10%) | 9 (8%) |
| Chronic Liver Disease, any | 10 (14%) | 7 (11%) | 11 (10%) | 10 (9%) |
| Peripheral Neuropathy | 52 (72%) | 43 (68%) | 69 (63%) | 73 (65%) |
| Prior Revascularization (any)‡ | 34 (47%) | 25 (40%) | 44 (40%) | 20 (18%) |
| Ipsilateral (open) | ||||
| None | 52 (72%) | 46 (73%) | 93 (85%) | 103 (92%) |
| Distant | 17 (24%) | 15 (24%) | 14 (13%) | 8 (7%) |
| Recent | 3 (4%) | 2 (3%) | 3 (3%) | 1 (1%) |
| Ipsilateral (endo) | ||||
| None | 53 (74%) | 48 (76%) | 75 (68%) | 95 (85%) |
| Distant | 16 (22%) | 11 (17%) | 28 (25%) | 11 (10%) |
| Recent | 3 (4%) | 4 (6%) | 7 (6%) | 6 (5%) |
| Contralateral (open) | ||||
| None | 62 (86%) | 56 (89%) | 104 (95%) | 112 (100%) |
| Distant | 9 (12%) | 6 (10%) | 6 (5%) | 0 (0%) |
| Recent | 1 (1%) | 1 (2%) | 0 (0%) | 0 (0%) |
| Contralateral (endo) | ||||
| None | 60 (83%) | 58 (92%) | 93 (85%) | 111 (99%) |
| Distant | 9 (12%) | 4 (6%) | 16 (15%) | 1 (1%) |
| Recent | 3 (4%) | 1 (2%) | 1 (1%) | 0 (0%) |
| Mild Cognitive Impairment | 2 (3%) | 4 (6%) | 4 (4%) | 5 (4%) |
| Post-Traumatic Stress Disorder | 23 (32%) | 14 (22%) | 19 (17%) | 31 (28%) |
| Schizophrenia/Psychosis | 6 (8%) | 2 (3%) | 2 (2%) | 3 (3%) |
| Bipolar disorder | 5 (7%) | 6 (10%) | 5 (5%) | 5 (4%) |
| Anxiety disorder | 24 (33%) | 23 (37%) | 26 (24%) | 30 (27%) |
| Depression | ||||
| No Depression | 38 (53%) | 25 (40%) | 68 (62%) | 53 (47%) |
| Depressive Disorder (non-major) | 7 (10%) | 13 (21%) | 18 (16%) | 21 (19%) |
| Major Depressive Disorder (MDD) | 27 (38%) | 25 (40%) | 24 (22%) | 38 (34%) |
| AUDIT-C Category§ | ||||
| Mild | 67 (93%) | 59 (94%) | 103 (94%) | 104 (93%) |
| Moderate | 5 (7%) | 2 (3%) | 6 (5%) | 7 (6%) |
| Severe | 0 (0%) | 2 (3%) | 1 (1%) | 1 (1%) |
| Tobacco Use (ever)‖ | 54 (75%) | 38 (60%) | 77 (70%) | 59 (53%) |
| Cocaine Use Disorder | 5 (7%) | 6 (10%) | 2 (2%) | 2 (2%) |
| Opioid Use Disorder | 6 (8%) | 7 (11%) | 3 (3%) | 7 (6%) |
| Subsequent Ipsilateral Amputation | 10 (14%) | 13 (21%) | 16 (15%) | 16 (14%) |
Unless otherwise specified, all diagnoses represent any diagnosis prior to prosthetic prescription
At time of incident amputation
in past 6 months prior to prosthetic prescription
All revascularizations are based on time prior to prosthetic prescription (any – any prior ipsilateral or contra); distant = > 90 days, recent - ≤ 90 days)
most recent prior to prosthetic prescription (Mild - Female: < 3, Male: < 4; Moderate - Female: ≥ 3 & < 8, Male: ≥ 4 & < 8; Severe ≥ 8 for both Male and Female)
22 missing values. Multiple imputation techniques were used to generate values in the model building.
Outcomes
A total of 72 individuals (20.2%) reported AMPSIMM scores indicating wheelchair mobility (0-2), 63 (17.6%) indicating household ambulation (3), 110 (30.8%) basic community ambulation (4), and 112 (31.4%) advanced community ambulation (5-6). Among TT amputees, this distribution was 52 (19.5%), 41 (15.4%), 73 (27.4%), and 100 (37.6%), respectively. Among TF amputees, 20 (22.0%), 22 (24.2%), 37 (40.7%), and 12 (13.2%), respectively.
Risk prediction model
The estimated quantitative discrimination characteristics of the models developed under all variable selection approaches were similar. However, the grouped lasso model selected by the “min” criterion was associated with the best discrimination characteristics and was chosen as the final prediction model. This model included 23 predictors; coefficients for the predictors for each AMPSIMM category are provided in Table 3. Because the AMPSIMM outcome has 4 categories, the model coefficients do not correspond to log odds ratios (as is the case with a binary outcome) and interpretation of the individual coefficients is more complicated, particularly for quantitative predictors, which are modeled non-linearly. Generally, positive coefficients indicate increased probability for those outcome categories relative to outcome categories with lower coefficient values, and negative coefficients indicate decreased probability for those outcome categories relative to outcome categories with higher coefficient values. Lower mobility levels are more likely in the presence of PAD, coronary artery disease (CAD), CHF, dialysis, COPD, schizophrenia/psychosis, anxiety, cocaine and opioid use, revascularization, and for married individuals. The selected non-linear models for the quantitative predictors time to prosthetic prescription, age and BMI are illustrated graphically in Supplementary Material. The models predict that the likelihood of advanced community ambulation (AMPSIMM 5-6) declines with increasing age, and, in TF amputees, with increasing BMI. The likelihood of wheelchair mobility (AMPSIMM 0-2) increases and that of basic community ambulation (AMPSIMM 4) decreases with increased time to prosthetic prescription. Associations with other factors, such as diabetes, PTSD, depression, AUDIT-C score, and tobacco use, were more complex due to non-monotonicity of predictions across AMPSIMM categories.
Table 3.
AMPSIMM category prediction coefficients for the final model.
| Variable | APMSIMM 0-2 | AMPSIMM 3 | APMSIMM 4 | AMPSIMM 5-6 |
|---|---|---|---|---|
| Intercept | −0.304 | −0.635 | 1.022 | −0.083 |
| Time to Prosthetic Rx (days): | ||||
| x−2 | 6.206 | −2.494 | 8.097 | −11.809 |
| x−2 log x | 1.502 | −0.759 | 2.235 | −2.978 |
| x1/2 log x | 0.004 | 0.002 | −0.004 | −0.002 |
| x−2 10−5) | 5.009 | 1.212 | −3.969 | −2.252 |
| Age (years): | ||||
| x−2 (× 103) | −1.821 | −1.757 | −0.095 | 3.673 |
| Not Married | −0.014 | −0.043 | 0.021 | 0.037 |
| PAD | 0.062 | 0.097 | −0.063 | −0.096 |
| CAD | 0.163 | 0.201 | −0.248 | −0.116 |
| Heart Failure | 0.093 | 0.054 | −0.071 | −0.076 |
| Diabetes | −0.042 | 0.341 | −0.402 | 0.103 |
| Dialysis | 0.204 | 0.334 | −0.137 | −0.401 |
| COPD | 0.093 | 0.149 | −0.028 | −0.215 |
| Any Revascularization | 0.034 | 0.007 | 0.032 | −0.073 |
| Ipsilateral Revascularization: | ||||
| Open | 0.118 | 0.120 | −0.132 | −0.106 |
| Endo | −0.014 | −0.012 | 0.050 | −0.024 |
| Contralateral Revascularization: | ||||
| Open | 0.348 | 0.237 | −0.173 | −0.412 |
| Distant Endo (> 90d prior to pros. Rx) | 0.199 | −0.184 | 0.523 | −0.538 |
| Recent Endo (≤ 90d prior to pros. Rx) | 0.716 | −0.048 | 0.190 | −0.857 |
| PTSD | 0.080 | −0.046 | −0.122 | 0.087 |
| Schizophrenia/Psychosis | 0.355 | −0.050 | −0.152 | −0.153 |
| Anxiety | 0.069 | 0.108 | −0.047 | −0.130 |
| Depression: | ||||
| Major Depressive Disorder (MDD) | 0.007 | 0.152 | −0.223 | 0.063 |
| Depressive Disorder (non-MDD) | −0.043 | 0.102 | −0.113 | 0.055 |
| Severe AUDIT-C (≥ 8) | −0.110 | 0.114 | −0.043 | 0.039 |
| Ever Used Tobacco | 0.083 | −0.088 | 0.159 | −0.155 |
| Cocaine Use | 0.076 | 0.232 | −0.162 | −0.146 |
| Opioid Use | 0.090 | 0.131 | −0.182 | −0.039 |
| Interactions with TF Amputation Level: | ||||
| Age (years): | ||||
| (x 10−4) | −0.374 | 3.961 | 4.492 | −8.079 |
| x log x(× 10−4) | −0.429 | 4.577 | 5.185 | −9.334 |
| BMI: | ||||
| x3 log x(× 10−6) | 1.654 | 0.345 | 0.762 | −2.761 |
The risk score, Sj, corresponding to an individual having an outcome in AMPSIMM category j is the sum of the coefficients in that category (one column of the table) for all the components that apply to that individual (including various functional forms for continuous variables such as Age). The predicted probability of that outcome = exp(Sj) + [exp(S2) + exp(S2) + exp(S2) + exp(S4)]. See the supplemental material for a detailed example calculation.
Model validation
The discrimination summaries are shown in Table 4. Bootstrap estimates of the optimism for the various discrimination measures range from 0.06 to 0.13, indicating inflation in these estimated assessments when using the same data to develop and validate the model. The sensitivity analysis using cross-validation resulted in similar performance characteristics. After adjustment for optimism, the overall discrimination is modest (optimism adjusted M-index=0.67), but the ability to distinguish some individual mobility categories from others is better. The model is best able to distinguish an AMPSIMM score of 3 (household ambulation), from a score of 4 (basic community ambulation), and from scores 5-6 (advanced community ambulation); pairwise adjusted c-indices=0.76), and there is reasonable ability to distinguish AMPSIMM household mobility and lower (scores 0-3) from community ambulators (scores 4-6) and basic community ambulators or less (scores 0-4) from advanced community ambulators (scores 5-6), with hierarchical c-indices=0.72 and 0.74, respectively. Summaries of the characteristics of the competing models can be found in Supplementary Material.
Table 4.
Quantitative discrimination measures for the final model.
| Discrimination Measure | Model Estimate | Optimism Adjusted Estimate | |
|---|---|---|---|
| PDI | 0.55 | 0.42 | |
| M-index | 0.76 | 0.67 | |
| Pairwise c-indices | 0-2 versus 3 | 0.61 | 0.48 |
| 0-2 versus 4 | 0.73 | 0.62 | |
| 0-2 versus 5-6 | 0.79 | 0.69 | |
| 3 versus 4 | 0.83 | 0.76 | |
| 3 versus 5-6 | 0.84 | 0.76 | |
| 4 versus 5-6 | 0.76 | 0.68 | |
| Hierarchical c-indices | 0-2 versus 3-6 | 0.74 | 0.64 |
| 0-3 versus 4-6 | 0.80 | 0.72 | |
| 0-4 versus 5-6 | 0.80 | 0.74 |
Figure 2 shows boxplots of the modelled predicted probabilities for each AMPSIMM category, stratified by the observed outcomes. In each grouping of boxplots, the largest probabilities are predicted for individuals having the corresponding mobility level, indicating that the model is discriminating correctly. The greatest separation in boxplots occurs in predicted probabilities for individuals with AMPSIMM scores in the 0-3 range relative to those in the 5-6 range.
Figure 2:
Discrimination plot of predictions for the final model.
Distributions of predicted probabilities for each AMPSIMM category are stratified by the actually observed category. The red lines show the prevalence of each AMPSIMM category.
Model application
The AMPREKDICT PRO prediction model can be used to determine a LLP candidate’s probable future mobility level. The predicted probabilities may be used to aid in prosthetic prescription, focus rehabilitation treatment and impart patient mobility outcome expectations. In Table 5, both patient examples have had TT amputations and are presenting for a rehabilitation and prosthetic prescription evaluation. Patient #1 has diabetes and is younger, is challenged by obesity and depression, but does not have an extensive list of comorbidities. Patient #2 is older, does not have diabetes, but has numerous vascular and pulmonary comorbidities associated with a long history of smoking. Patient #1 has a high probability of being an independent community ambulator; therefore, the treatment plan should be directed to achieving these goals with a prosthetic prescription that optimizes performance at this functional level. In contrast, Patient #2 is likely to only achieve household or wheelchair mobility. In this context, the rehabilitation team could inform the patient of this outcome while focusing the rehabilitation treatment on wheelchair mobility, considering home modifications to accommodate a wheelchair.
Table 5.
Hypothetical patient examples of predicted probabilities for each of the 4 AMPSIMM categories.
| Predictors | Patient #1* | Patient #2† |
|---|---|---|
| Amputation level | Transtibial | Transtibial |
| Time to prosthetic prescription (days) | 45 | 92 |
| Age (years) | 59 | 75 |
| Married | Yes | No |
| Body mass index (kg/m2) | 32 | 24 |
| Peripheral artery disease | Yes | Yes |
| Coronary artery disease | No | Yes |
| Congestive heart failure | No | Yes |
| Diabetes | Yes | No |
| Currently on dialysis | No | No |
| Chronic obstructive pulmonary disease | No | Yes |
| Open ipsilateral revascularization | No | Distant |
| Endovascular ipsilateral revascularization | Recent | Distant |
| Open contralateral revascularization | No | Recent |
| Endovascular contralateral revascularization | No | No |
| Post-traumatic stress disorder | No | Yes |
| Schizophrenia/psychosis | No | No |
| Anxiety | No | Yes |
| Depression | Yes | Yes |
| Severe alcohol use disorder | No | Yes |
| Smoker (ever) | No | Yes |
| Cocaine use disorder (ever) | No | No |
| Opioid use disorder (ever) | No | No |
| Predicted mobility outcomes 12 months after initial prosthetic prescription | ||
| Wheelchair mobility | 9.9% | 42.4% |
| Household ambulator | 11.5% | 22.7% |
| Basic community ambulator | 30.0% | 24.3% |
| Advanced community ambulator | 48.5% | 10.7% |
Patient #1: A 59-year-old TT amputee secondary to CLTI 45 days post amputation. His BMI is 32 he is married, and has the following comorbidities: PAD, CAD, diabetes, depression and an ipsilateral endovascular revascularization approximately 1 month ago for claudication.
Patient #2: A 75-year-old TT amputee secondary to CLTI. He has a long history of cigarette smoking and hyperlipidemia. Because of some delayed healing, he is 92 days post amputation. His BMI is 24, he is not married, and has the following co-morbidities: PAD, CAD, CHF, COPD, and has had several ipsilateral and contralateral revascularizations. Depression, anxiety, PTSD, along with significant increased alcohol intake are additional medical diagnoses.
Discussion
The primary purpose of this study was to develop and validate a patient-specific multivariable model to predict 12-month prosthetic mobility at the time of the first prosthetic prescription at the TT or TF amputation level, based on a composite of predictors that are readily available in the electronic health record (EHR). The proposed benefits of the prediction model are to: 1) provide evidence at the time of prescription to facilitate matching the patient to the most appropriate prosthetic device, 2) assist the rehabilitation team to more appropriately develop a treatment plan that will focus on the individual patient’s future mobility level, and 3) assist in setting appropriate patient outcome expectations.
Prosthetic prescription is a complicated process for the most experienced amputee rehabilitation teams and is extremely challenging for practitioners with less experience. At the time of prosthetic prescription, the primary goal is to provide a prosthesis that will allow safe ambulatory mobility, but also allow the patient to accomplish their anticipated mobility goals. Because of a lack of scientific evidence to guide patient specific prosthetic prescription, a clinical estimation of a patient’s future mobility using K-levels is often used. The absence of evidence to support patient-specific prosthetic prescription has been identified by an Agency for Healthcare Research and Quality comparative effectiveness review as a significant limitation. It concluded that studies are not available that predict which LLAs would benefit most or least from a given component. Further high-quality evidence is needed to inform optimal matching of prosthetic components to patients.23 The absence of a VA or Medicare evidence-based process to predict future prosthetic mobility or matching an individual patient to the most appropriate prosthetic prescription remains a major challenge facing both the VA and the private sector. The AMPREDICT PROsthetics (aka AMPREDICT PRO) model has the potential to fill this gap.
The absence of evidence to help guide optimum prosthetic prescription has been identified as a concern by rehabilitation clinicians. Qualitative studies have shown that inadequate evidence has an adverse effect on clinician experience with concerns about potential bias in prosthetic prescription, and negative outcomes associated with inappropriate prosthetic prescription.24 These include the exposure of patients to risk through incorrect prescription, including potential falls and subsequent injury, or the provision of a prosthetic limb that may not be optimal, resulting in diminished ability to achieve mobility goals or discontinuation of prosthetic use.25
Personalized medicine has emerged to shift away from traditional “one size fits all” approaches to patient care, which have often been deemed ineffective, inefficient, and not driven by patient values. This approach is well underway in many medical disciplines, including oncology; however, the power of personalized medicine in rehabilitation is in its infancy.26 Prediction models are the ideal mechanism for implementing personalized medicine, including personalized rehabilitation, because they allow for data driven, individualized decisions regarding the interventions to be delivered.26
Prediction models that accurately establish future mobility may be used to help guide both rehabilitation treatment planning and prosthetic prescription. To our knowledge, only a single prediction model at the time of prosthetic prescription has previously been developed; 27 this model and its methods differ significantly from the AMPREDICT PRO model. A key difference is that the model was developed in “new” and “replacement” prosthetic users, with mobility follow-up times that were as short as less than a year and as long as over 5 years. Therefore, the patient population was very heterogenous and could include patients who had reamputations and other comorbidities that developed after their initial prosthetic prescription. AMPREDICT PRO was designed to be applied at the time of first prosthetic prescription, when the estimated mobility probability of a potential prosthetic user is most challenging. Furthermore, the published model predicted a continuous mobility score rather than a probability of achieving a specific level of mobility, as with AMPREDICT PRO. We elected the latter approach to provide clear guidance to the provider and patient on what mobility levels patients should expect to achieve.
The AMPREDICT PRO model includes 23 predictors across a wide range of domains. Our goal was to develop a model with predictors readily available in the medical record so that a future risk calculator with auto-population features can be developed. The advantage of this approach is that it will significantly reduce implementation obstacles, which were noted in our prior development of the AMPREDICT decision support tool (DST).28 The original AMPREDICT DST, which is currently available for online use (www.ampredict.org), requires manual data entry and some predictors not readily available in the EHR, requiring interview and evaluation of the patient, which increases the time and burden. Therefore, it is currently being updated and will be integrated in the EHR with autopopulation features.
When viewing the performance characteristics as a tool to differentiate individual mobility categories, the model performs effectively at distinguishing household ambulation from basic community ambulation and from advanced community ambulation. Similarly, it performs well at differentiating basic community ambulation from advanced community ambulation, but performs less effectively at differentiating those who will exclusively use a wheelchair for mobility from those that will only be able to perform limited in-home ambulation. This led to overall modest performance characteristics, especially after correcting for optimism in the internal validation. There are several potential reasons for this. The most compelling is that only predictors available at the time of prosthetic prescription can be included. We are unable to account for events that occur after prosthetic prescription that may affect future mobility (e.g., reamputation, contralateral amputation, or other significant comorbid events). However, the model is intended for prediction at the time of prosthetic prescription; therefore, accounting for these variables is not relevant for the model’s intended use. Further, the higher levels of mobility that were more effectively discriminated are of key clinical importance when estimating future prosthetic demands. The utility of the model in clinical care will require future prospective validation in addition to the existing bootstrap internal validation, but it holds the potential to support clinicians and rehabilitation teams as they prescribe prosthetic limbs and establish functional mobility goals in a shared communication with their patients.
Study Limitations
Several limitations of the current study are worthy of note. Because of the patient eligibility criteria, the results can only be applied to individuals who have undergone their first unilateral amputation at the TT level or higher due to diabetes or PAD, who do not have a diagnosis of dementia, and who survive a year after prosthetic prescription. The number of women in the development model was very small because of its development in a Veteran population. An additional potential limitation is the 21% response rate. Importantly, we found that the characteristics of patients who were eligible but not enrolled were very similar to those enrolled, reducing potential concerns of response bias.
Summary
We have successfully developed the AMPREDICT PRO model to be applied at the time of prosthetic prescription using predictors available in the medical record. This model will meet the perceived clinical need for assistance in determining future prosthetic mobility and the appropriate LLP to accomplish these goals. A more accurate prediction of mobility will assist rehabilitation teams in formulating appropriate rehabilitation goals and will improve the shaping of patient expectations.
Supplementary Material
Acknowledgments
This material is based upon work supported by the US Department of Veterans Affairs, Office of Research and Development, Rehabilitation Research and Development Grant number 1 I01 RX002919R-01.
Footnotes
Publisher's Disclaimer: This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
The opinions expressed are those of the authors and not necessarily those of the Department of Veterans Affairs or the United States Government.
References
- 1.Suckow BD, Goodney PP, Nolan BW, et al. Domains that Determine Quality of Life in Vascular Amputees. Ann VascSurg. 2015;29(4):722–730. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Chan KM, Tan ES. Use of lower limb prosthesis among elderly amputees. Ann Acad Med Singapore. 1990;19(6):811–816. [PubMed] [Google Scholar]
- 3.Asano M, Rushton P, Miller WC, Deathe BA. Predictors of quality of life among individuals who have a lower limb amputation. Prosthet Orthot Int. 2008;32(2):231–243. [DOI] [PubMed] [Google Scholar]
- 4.Chen MC, Lee SS, Hsieh YL, Wu SJ, Lai CS, Lin SD. Influencing factors of outcome after lower-limb amputation: a five-year review in a plastic surgical department. Ann Plast Surg. 2008;61(3):314–318. [DOI] [PubMed] [Google Scholar]
- 5.Pell JP, Donnan PT, Fowkes FG, Ruckley CV. Quality of life following lower limb amputation for peripheral arterial disease. Eur J Vasc Surg. 1993;7(4):448–451. [DOI] [PubMed] [Google Scholar]
- 6.Pernot HF, Winnubst GM, Cluitmans JJ, De Witte LP. Amputees in Limburg: incidence, morbidity and mortality, prosthetic supply, care utilisation and functional level after one year. Prosthet Orthot Int. 2000;24(2):90–96. [DOI] [PubMed] [Google Scholar]
- 7.Schaffalitzky E, Gallagher P, Maclachlan M, Ryall N. Understanding the benefits of prosthetic prescription: exploring the experiences of practitioners and lower limb prosthetic users. Disabil Rehabil. 2011;33(15-16):1314–1323. [DOI] [PubMed] [Google Scholar]
- 8.Singh R, Ripley D, Pentland B, et al. Depression and anxiety symptoms after lower limb amputation: the rise and fall. Clin Rehabil. 2009;23(3):281–286. [DOI] [PubMed] [Google Scholar]
- 9.Whyte AS, Carroll LJ. A preliminary examination of the relationship between employment, pain and disability in an amputee population. Disabil Rehabil. 2002;24(9):462–470. [DOI] [PubMed] [Google Scholar]
- 10.Palmetto. DMERC Medicare Advisory. Columbia, SC: Government Benefits Administrators; December 1994. [Google Scholar]
- 11.Das SR, Kinsinger LS, Yancy WS Jr., et al. Obesity prevalence among veterans at Veterans Affairs medical facilities. Am J Prev Med. 2005;28(3):291–294. [DOI] [PubMed] [Google Scholar]
- 12.Czerniecki JM, Thompson ML, Littman AJ, et al. Predicting reamputation risk in patients undergoing lower extremity amputation due to the complications of peripheral artery disease and/or diabetes. Br J Surg. 2019;106(8):1026–1034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Norvell DC, Thompson ML, Boyko EJ, et al. Mortality prediction following non-traumatic amputation of the lower extremity. Br J Surg. 2019;106(7):879–888. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Littman AJ, Tseng C-L, Timmons A, et al. Risk of ipsilateral reamputation following an incident toe amputation among US Military veterans with diabetes, 2005–2016. Diabetes Care. 2020;43(5):1033–1040. [DOI] [PubMed] [Google Scholar]
- 15.Norvell DC, Biggs WT, Bott J, Henderson AW, Moore KP, Czerniecki JM. The development and validation of a novel prosthetic component sophistication classification system Archives of Physical Medicine and Rehabilitation (under review). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Hoddinott SN, Bass MJ. The dillman total design survey method. Can Fam Physician. 1986;32:2366–2368. [PMC free article] [PubMed] [Google Scholar]
- 17.Norvell DC, Williams RM, Turner AP, Czerniecki JM. The development and validation of a novel outcome measure to quantify mobility in the dysvascular lower extremity amputee: the amputee single item mobility measure. Clin Rehabil. 2016;30(9):878–889. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. Journal of statistical software. 2010;33(1):1. [PMC free article] [PubMed] [Google Scholar]
- 19.Hastie T, Tibshirani R, Friedman JH, Friedman JH. The elements of statistical learning: data mining, inference, and prediction. Vol 2: Springer; 2009. [Google Scholar]
- 20.Wood AM, White IR, Royston P. How should variable selection be performed with multiply imputed data? Statistics in medicine. 2008;27(17):3227–3246. [DOI] [PubMed] [Google Scholar]
- 21.Van Calster B, Vergouwe Y, Looman CW, Van Belle V, Timmerman D, Steyerberg EW. Assessing the discriminative ability of risk models for more than two outcome categories. European journal of epidemiology. 2012;27(10):761–770. [DOI] [PubMed] [Google Scholar]
- 22.Team RC. R: A language and environment for statistical computing. 2013.
- 23.Balk EM, Gazula A, Markozannes G, et al. AHRQ Comparative Effectiveness Reviews. In: Lower Limb Prostheses: Measurement Instruments, Comparison of Component Effects by Subgroups, and Long-Term Outcomes. Rockville (MD): Agency for Healthcare Research and Quality (US); 2018. [PubMed] [Google Scholar]
- 24.Sansam K, O’Connor RJ, Neumann V, Bhakta B. Clinicians′ perspectives on decision making in lower limb amputee rehabilitation. J Rehabil Med. 2014;46(5):447–453. [DOI] [PubMed] [Google Scholar]
- 25.Sansam K, Neumann V, O′Connor R, Bhakta B. Predicting walking ability following lower limb amputation: a systematic review of the literature. J Rehabil Med. 2009;41(8):593–603. [DOI] [PubMed] [Google Scholar]
- 26.French MA, Roemmich RT, Daley K, et al. Precision Rehabilitation: Optimizing Function, Adding Value to Health Care. Arch Phys Med Rehabil. 2022. [DOI] [PubMed] [Google Scholar]
- 27.Anderson CB, Wurdeman SR, Miller MJ, Christiansen CL, Kittelson AJ. Development of a physical mobility prediction model to guide prosthetic rehabilitation. Prosthet Orthot Int. 2021;45(3):268–275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Norvell DC, Henderson A, Sukow B, Landry G, Twine C, Czierniecki J. The Development and Usability of the AMPREDICT Decision Support Tool: Translating Research to Shared Decision Making for Veterans Facing Lower Limb Amputation. European Journal of Endovascular and Vascular Surgery. 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Gual A, Segura L, Contel M, Heather N, Colom J. Audit-3 and audit-4: effectiveness of two short forms of the alcohol use disorders identification test. Alcohol and alcoholism. 2002;37(6):591–596. [DOI] [PubMed] [Google Scholar]
- 30.Royston P, Sauerbrei W. Multivariable model-building: a pragmatic approach to regression anaylsis based on fractional polynomials for modelling continuous variables. John Wiley & Sons; 2008. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.