The Predictive Validity of Functional Outcome Measures With Discharge Destination for Hospitalized Medical Patients

Abstract

Objective

To investigate the predictive validity for discharge to home or facility of 4 functional mobility outcome measures.

Design

Retrospective, observational study.

Setting

Urban, academic hospital in the United States.

Participants

Adult patients (N=3999) admitted to medical units between June 1, 2019, and February 29, 2020, with 2 or more recorded scores on each of 4 tools: Activity Measure for Post-Acute Care (AM-PAC) 6-Clicks Basic Mobility and Daily Activity, Henry Ford Mobility Level, and The Johns Hopkins Highest Level of Mobility.

Interventions

Not applicable.

Main Outcome Measures

Mobility scores and discharge destination.

Results

For the 3999 subjects, 51.4% went home at discharge and had higher mean scores on each measure than those not returning home. Both early (I) and later (II) time point for each measure had positive predictability for discharge home. AM-PAC 6-Clicks had the highest confidence intervals for early and later recorded scores. The c-statistic value for Basic Mobility I (cut point=16) was 0.74 and for II (cut point=18) was, 0.79. The value for Daily Activity I (cut point=18) was 0.75 and for Daily Activity II (cut point=18) was 0.80). The Johns Hopkins Highest Level of Mobility and Henry Ford Mobility Level measures were less discriminative at initial score (c-statistic 0.704 and 0.665, respectively) and final score (c-statistic 0.74 and 0.75, respectively).

Conclusions

Functional outcome measures have good predictive validity for discharge destination. The AM-PAC Basic mobility score appears to have a slightly higher confidence interval than the other tools in this study design.

Discharge disposition is an integral part of acute care hospital planning. The expectation is that patients who come from home will return home. Without early, accurate planning, those requiring post-acute services often have increased lengths of stay as this is sorted out.¹ Little a priori guidance exists regarding which patients will require advance care after discharge.² Health care professionals, patients, and family members need data and resources to understand medical decision making, prognosis, and outcomes. Functional mobility status has been identified as a predictor of the need for skilled nursing after hospital discharge.3, 4, 5 Physical decline and deconditioning are both direct and indirect consequences of inpatient hospital stays. Standardized outcome measure tools already exist and can be used to guide health care professionals with respect to discharge disposition. These tools were developed to meet specific needs spanning clinical care and research. Many were developed to support multi-disciplinary input and interpretation.6, 7, 8 Our goal is to develop a study model and compare the different tools with respect to predictive value. We hypothesize that mobility score is analogous to a diagnostic test. We are diagnosing the likelihood that the patient will go home at discharge. We therefore compared the different validated instruments for predictive accuracy of this outcome.

In this study, we investigated the predictive validity of 4 outcome measures used at our hospital. The Activity Measure for Post-Acute Care (AM-PAC) 6-Clicks was designed to measure the amount of assistance a patient requires for tasks. The tool is subdivided between mobility and activity focus.^4,9, 10, 11, 12, 13, 14 We studied both modules separately. The Henry Ford Mobility Level (HFML) was developed and validated for multi-stakeholder mobility evaluation.¹⁵ Its key value is multi-stakeholder buy-in. The Johns Hopkins Highest Level of Mobility (JH-HLM) tool was introduced in the literature in 2018.^7,16 Similar to AM-PAC, it has been studied as a stand-alone tool.^17,18 Little has been published on the real world head to head comparison of these various mobility tools.

Methods

Study design

This retrospective, observational study used data extracted from discrete flowsheets in the electronic health record. All patients were 18 years of age or older and admitted to the general medical unit or the medical intensive care unit of an urban, tertiary care hospital between January 1, 2018, and February 1, 2020. All had initial scoring performed within 48 hours of admission. Exclusion criteria included missing data from any 1 of the mobility scores, those that left against medical advice, expired in the hospital, entered hospice, or transferred to another acute care hospital. To reduce confounder of severity, patients were excluded if the length of stay was greater than 45 days. All patients received 2 or more physical or occupational therapy visits. Charlson Acuity Score¹⁹ and LACE index score²⁰ were collected to characterize the subject population. The LACE index uses four variables to predict the risk of death or nonelective 30-day readmission after hospital discharge among both medical and surgical patients: length of stay (L), acuity of the admission (A), comorbidity of the patient (C) and emergency department use in the duration of 6 months before admission.²⁰

This study was approved by the Institutional Review Board and met criteria for research. Informed consent was waived because of the retrospective nature of study.

Outcomes

The primary outcome variable was discharge destination. For this study design, discharge destinations were dichotomized to home (with or without home care services) and post-acute facility (inpatient rehabilitation facility, skilled nursing facility, or subacute rehabilitation facility). Secondary outcomes included comparison of initial and final measured data points for the different measurement instruments.

AM-PAC 6-Clicks Basic Mobility and Daily Activity

The AM-PAC 6-Clicks tools are reliable, validated outcome measures that use a 4-point scale that scores the amount of assistance (1=total assistance, 2=a lot of assistance, 3=a little assistance, and 4=no assistance) to complete each of the tasks.21, 22, 23 For each tool, the raw scores range from 6 to 24 and higher scores indicate better mobility.^{6,7,9,21,23,24} AM-PAC can be used as 2 separate tools: basic mobility and daily activity. Score were recorded by Physical or Occupational Therapists at first visit and last visit.

Henry Ford Mobility Level

The HFML was developed at Henry Ford Hospital, Detroit, Michigan in 2015 as an initiative to document mobility level across all inpatient units by all types of care providers including therapists, medical assistants, nurses, and physicians.^15,25,26 It is a 5-point mobility level scale (1=bed level, 2=dangle edge of bed, 3=standing to chair, 4=walk with assistance, and 5=walk independently). The raw scores range from 1 to 5. Score were recorded by Registered Nurse, Physical or Occupational Therapists at each visit or every 12 hours by nursing.

The Johns Hopkins Highest Level of Mobility

The JH-HLM is a 1-item scale with 8 ordinal responses scored based on a patient's observed highest level of activity, as follows: 1=only lying, 2=bed activities, 3=sitting at edge of bed, 4=transferring to chair, 5=standing for greater than or equal to 1 minute, 6=walking 10 or more steps, 7=walking approximately 25 feet or more, and 8=walking approximately 250 feet or more. Raw scores range from 1 to 8. Score were recorded by Physical or Occupational Therapists at first visit and last visit.

Statistical analysis

Descriptive statistics collected include demographics (age, sex, and race) and clinical characteristics including Medicare Severity Diagnosis Related Group (MS-DRG),^27,28 Charlson Acuity Score,¹⁹ LACE score,²⁰ hospital length of stay, medical intensive care unit length of stay, and rate of readmission within 30 days. Descriptive statistics and odds ratio were used to compare the 4 mobility scale results with respect to discharge disposition. Sensitivity, specificity, and positive predictive values were calculated for each mobility measure result regarding discharge destination (home vs post-acute facility). Odds ratios, standard error with 95% confidence interval (CI), and area under the curve were also determined. For each of our 4 models, we developed adjusted c-statistics, receiver operating characteristic (ROC) curves, and calibration plots. The Youden index is a measure of a diagnostic test's ability to balance sensitivity (detecting disease) and specificity (detecting health or no disease). In this case, the mobility scales are being used as a diagnostic test of discharge back to home. The calculated cut points of the Youden index are the values that provide the best trade-off between sensitivity and specificity.29, 30, 31 Subsequently, logistic regression models were fit to test for association between dichotomized measures (6-Clicks Basic Mobility, 6-Clicks Daily Activity, HFML, and JH-HLM) and chances of going home, adjusted for age, and sex. For each outcome measure, ROC curves were plotted and cut points were optimized using the Youden Index² in order to identify the most effective score for our purpose of predicting discharge destination of home.9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31 The Youden's J index combines sensitivity and specificity into a single measure (Sensitivity + Specificity - 1) and has a value between 0 and 1. For our statistical analyses, we used SAS software (SAS Institute, Inc, Cary, NC).^a

In this study, we used the first (I) and last (II) recorded values for all 4 mobility scales. The first was used because it represents the earliest measured value that can be applied to our model. The rationale for the last value (II) is that it is closest to discharge and therefore most accurately represents the actual functional and mobility ability of the individual at discharge. Additionally, comparing the predictive utility of both the first and last values has the potential to provide internal control to our study and additional insight on outcomes.

Results

The study population included 3999 patients on general medical units with complete scores for all measures and time points. Just over half of the patients went directly home after discharge (51.4%). The demographic and clinical characteristics of the population are provided in table 1. The group had a median age of 63.5 years (interquartile range 48-79). Men represented 50% of the population. The racial prolife was 48% African-American, 40% White, and 12% other. In descending order, the most common MS-DRG categories were neurologic (23%), respiratory (15%), circulatory (19%), hepatobiliary (10%), infectious disease/sepsis (10%), and musculoskeletal (5%). The median length of stay was 10.3 days (SD 7.2). Within the population, 45% required an intensive care unit stay for an average of 3.1 (SD 5.3) days. Table 2 shows that the mean scores for patients discharged to home were statistically significantly higher than those who went to post-acute facilities, for all 4 outcome measures and both time points.

Table 1.

Demographic and medical description of the study population

Description	Study Population (N=3999)
Age at admission, mean y (SD)	63.5 (15.5)
Sex, male, n (%)	1989 (50)
Race, n (%)
African-American	1929 (48)
White	1607 (40)
Other	453 (12)
Top 6 Major MS-DRG, n (%)
Neurological	903 (23)
Circulatory	741 (19)
Respiratory	587 (15)
Hepatobiliary	400 (10)
Infectious, sepsis	397 (10)
Musculoskeletal	201 (5)
Charlson score, mean ± SD (n=3679)	7.5 (3.5)
LACE score, mean ± SD	11.2 (2.9)
Hospital length of stay (d), mean ± SD	10.3 (7.2)
ICU length of stay (d), mean ± SD	3.1 (5.3)
Readmission within 30 d, n (%)	1080 (27)
Discharge destination, home, n (%)	2054 (51.4)

Abbreviation: ICU, intensive care unit.

Table 2.

Mobility score comparison for discharge home and post-acute facility

Outcome Measure and Time Point	Home (n=2054)	Post-Acute Facility (n=1945)	P Value
6-Clicks Basic Mobility I, mean ± SD	16.8 (4.1)	12.7 (4.6)	<.001*
6-Clicks Basic Mobility II, mean ± SD	19.7 (3.9)	13.8 (4.3)	<.001*
6-Clicks Daily Activity I, mean ± SD	16.8 (4.2)	13.1 (4.1)	<.001*
6-Clicks Daily Activity II, mean ± SD	19.5 (3.9)	14.3 (3.6)	<.001*
HFML I, mean ± SD	3.0 (1.5)	2.3 (1.3)	<.001*
HFML II, mean ± SD	4.1 (0.9)	3.0 (1.2)	<.001*
JH-HLM I, mean ± SD	5.3 (1.7)	4.1 (1.7)	<.001*
JH-HLM II, mean ± SD	6.4 (1.3)	4.9 (1.8)	<.001*

Abbreviations: I, first recorded value after admission; II, last value closest to discharge.

^⁎Statistical significance at P<.01.

We also used the data to understand changes in mobility measurement during the hospital course for the 2 populations. Patients who went home had a mean score of 16.8 on both 6-Clicks measures (Mobility and Activity) recorded early in their hospital stay. This increased a mean of 2.8 points for this group by discharge. The post-acute facility group scored 12.7 and 13.1 on the Basic Mobility and Daily Activity measures initially and only increased a mean of 1.1 points during their hospital course. Length of stay was similar for the groups. Similar pattern was noted for the other 2 tools. The 8 cut points (4 mobility tools and 2 timepoints) were determined (table 3). Figure 1 shows the plotted ROC curves used to develop the Youden index J statistic. Youden index J statistic ranges from 0 to 1 and is a measure of a diagnostic test's ability to balance sensitivity and specificity at that cut point.^30,31 In general, above 0.70 is considered a good diagnostic test. For AM-PAC 6-Clicks Daily Activity, the cut points for I and II were different, 16 and 18, respectively. For all other scales, the time points I and II were identical.

Table 3.

Statistical analysis and Youden Index calculation of the mobility scales results

Outcome Measure and Time Point	Cut Point	OR (95% CI)	Adjusted C-Statistic
6-Clicks Basic Mobility I	16	5.8 (5.32-6.32)	0.7443
6-Clicks Basic Mobility II	18	11.2 (10.2-12.3)	0.7923
6-Clicks Daily Activity I	18	4.97 (4.56-5.43)	0.7507
6-Clicks Daily Activity II	18	10.3 (9.35-11.3)	0.804
HFML I	4	2.46 (2.26-2.68)	0.6654
HFML II	4	8 (7.25-8.83)	0.7525
JH-HLM I	6	3.72 (3.26-4.24)	0.7048
JH-HLM II	6	6.89 (5.94-8)	0.7426

Abbreviations: I, first recorded value after admission; II, last value closest to discharge; OR, odds ratio.

Fig 1.

ROC curves for the mobility scales. Graphic representation of the Youden Index determination. I is the first recorded value after admission. II is the last value closest to discharge.

Using the calculated cut points, we compared the odds ratio for the groups discharged to home vs post-acute facility (table 3). The odds of going home were higher for patients who scored above the individual cut points for all scales at both time points. For each mobility scale, timepoint IIs were consistently higher odds ratio than the corresponding cut point for timepoint Is. For timepoint I, the highest odds ratio were associated with both AM-PAC tools followed by JH-HLM and HFML

Discussion

This study found that, to varying degrees, each of the measures of functional mobility studied can identify differences in populations destined to return home and those who will require post-acute inpatient care. Not surprisingly, the late timepoint data were more accurate than the data collected closer to hospital admission. Well-designed mobility tools can communicate a snapshot of patient ability to cognitively and physically perform a set of actions. This information provides a data point regarding the big picture on the overall well-being of a hospitalized patient. For the vast majority of the patients discharged to post-acute care, deconditioning is a major factor. Hospitalized patients, especially those in the intensive care unit, spend most of the time in bed. It should not be unexpected that almost half cannot return home. This requirement for additional supervised care is a burden that the patient, family, and health system need to accommodate. The earlier in the hospital course such information is available, the more actionable it is. Early on in the hospital course, mobility and deconditioning are definitely not the priority. Treatment of the specific condition or disease are the focus. As the hospitalization progresses and the clinical problems are treated and controlled, the focus shifts from disease-specific data points to more global data points. These global data points, including functional mobility status, give a more holistic view of patient's overall wellbeing and ability to resume their premorbid life. A reliable global data point, early in the hospital stay, could increase predictability and decrease stress. A well-designed mobility assessment may serve this purpose. Our study confirms the ability of the various mobility tools to predict or diagnose eventual discharge disposition.

The strongest predictability of discharge destination were models using the last measured values (timepoint II). This is the value closest to the time of discharge. Although statistically impressive, it has the least clinical value. For the early timepoint, the 2 AM-PAC tools both performed well compared with the other 2 tools. We believe this is the first head to head comparison of the 4 tools to be published. Further work is needed to further develop and refine these results.

Both AM-PAC tools are based on the amount of assistance required to complete the activities. These instruments can be thought of as tests of functional independence. Patients with an AM-PAC score of 16 or higher, clinically indicating patients who need little to no assistance to perform basic mobility and daily activities, are more likely to return home at discharge than those with scores of 15 or lower. The cut point of 16, both clinically and statistically, makes sense as a benchmark level of independence demonstrated at the bedside.

We also saw a small but measurable change in both mean AM-PAC scores from timepoint I to II. This is interesting because it may display 1 characteristic of the home group has increased potential to counteract the deconditioning during the acute care stay. Even though the body is focused on recovering from the medical condition that necessitated acute care admission, there is enough reserve to allow for general functional improvement with therapy intervention. Understanding this will give us a better understanding of resource allocation. We cannot ignore the most debilitated patient, but we should focus time and effort on those who will benefit the most. This may even provide insight into how to convert high-risk post-acute patients into discharge home patients.

Our results agree with previous large-center studies of the predictive utility of the AM-PAC instruments. Warren et al studied AM-PAC predictive ability with respect to discharge destination and found area under the curve values of 0.80 (95% CI, 0.80-0.81) and 0.81 (95% CI, 0.80-0.82) for Basic Mobility and Daily Activity, respectively. They used a cut point of 16 for Basic Mobility and 19 for Daily Activity. They found that patients with a Basic Mobility score below the threshold were 7.8 (95% CI, 6.83-8.91) times more likely to be discharged to a skilled nursing facility than home, and those with a Daily Activity score below the threshold were 8.87 (95% CI, 7.9-9.95) times more likely to be discharged to a skilled nursing facility. Odds ratios for discharge to an inpatient rehabilitation facility were 7.54 (95% CI, 6.28-8.91) and 11.44 (95% CI, 9.68-13.51) for Basic Mobility and Daily Activity, respectively.¹⁰ These results corroborate what we found based on our own odds ratios and provide more evidence supporting the predictive utility of AM-PAC scores.

Pfoh et al also studied initial AM-PAC Basic Mobility scores taken within 48 hours of hospital admission. The authors concluded that initial scores had a moderate ability to predict discharge destination, with an odds ratio of 0.78, similar to what was found. Warren et al, using a cut point of 12, found a positive predictive value of 0.59 and a negative predictive value of 0.81 for discharge home. Indicating that a score less than 12 could identify patients who required non-home discharge 80% of the time.¹² Taken together, these results suggest that initial AM-PAC scores may be more useful for predicting the need for post-acute care as opposed to the likelihood of discharge to home. Our data showed increased odds ratio from timepoint I to II (table 3). The implication is that at timepoint I, some patients may be incorrectly categorized as going home. Obviously, timepoint I cannot anticipate the entire hospital course. Many patients can get sicker and more debilitated during their hospital stay. More research is needed to understand to what degree this uncertainty can be identified and controlled for.

For the outcome tools that quantify the unassisted level of mobility, there are no published comparable studies. The HFML is particularly insensitive at level 4. This may be due to built-in ambiguity regarding the quality and quantity of assistance required at this level. The JH-HML tool was designed to assess needs and track progress during hospital stay.¹⁶ It may not have the granularity necessary for a diagnostic test in this model.

Study limitations

Although only a single institution's experience, the large number of patients and the hundreds of care providers used for patient assessments should imply broad applicability. Further limitations include the retrospective nature of the study design and selection bias. Actual discharge destination can be multi-factorial, including social determinants of health, individual preferences, and financial constraints. The timing of scoring was not standardized and could have influenced results.

Conclusions

Functional outcome measures have good predictive value for discharge destination from the acute care setting. Functional mobility assessment using AM-PAC Basic Mobility, AM-PAC Daily Activity, HFML, or JHHML tools may help identify patients who are most likely to go home. Assessments early in the hospital stay can be considered reliable and allow care givers to understand patients at a more holistic level.

Suppliers

• a.SAS software (SAS Institute, Inc., Cary, NC).