Abstract
Objective
Although Medicare assessment files will include Standardized Patient Assessment Data Elements from 2016 forward, lack of uniformity of functional data prior to 2016 impedes longitudinal research. The purpose of this study was to create crosswalks for postacute care assessment measures and the basic mobility and daily activities scales of the Activity Measure for Post-Acute Care (AM-PAC) and to test their accuracy and validity in development and validation datasets.
Methods
This cross-sectional study is a secondary analysis of AM-PAC, the Inpatient Rehabilitation Facility Patient Assessment Instrument, the Minimum Data Set, and the Outcome and Assessment Information Set data from 300 adults receiving rehabilitation recruited from 6 health care networks in 1 metropolitan area. Rasch analysis was used to co-calibrate items from the 3 measures onto the AM-PAC metric and to create look-up tables to create estimated AM-PAC (eAM-PAC) scores. Mean scores and correlation and agreement between actual and estimated scores were examined in the development dataset. Scores were estimated in a cohort of Medicare beneficiaries with hip, humerus and radius fractures. Correlations between eAM-PAC and Functional Independence Measure motor scores were examined. Differences in mean eAM-PAC scores were evaluated across groups of known differences (age, fracture type, dementia).
Results
Strong correlations were found between actual and eAM-PAC scores in the development dataset. Moderate to strong correlations were found between the eAM-PAC basic mobility and Functional Independence Measure motor scores in the validation dataset. Differences in basic mobility scores across known groups were statistically significant and appeared to be clinically important. Differences between mean daily activities scores were statistically significant but appeared not to be clinically important.
Conclusion
Although further testing is warranted, the basic mobility crosswalk appears to provide valid scores for aggregate analysis of Medicare postacute care data.
Impact
This study reports on a method to take data from different Medicare administrative data sources and estimate scores on 1 scale. This approach was applied separately for data related to basic mobility and to daily activities. This may allow researchers to overcome challenges with using Medicare administrative data from different sources.
Postacute care is an important area of health services research, accounting for a substantial proportion of Medicare spending.1 Questions about comparative effectiveness, dosage, timing and sequencing of alternative treatments, and the extent and impacts of disparities in postacute care are active topics of inquiry.2–5 Extensive data are available from Centers for Medicare and Medicaid Assessment files to address these and other important questions. However, investigation into the role of physical functioning in postacute research has been limited by the lack of uniform functional assessment instruments across postacute settings. The Improving Medicare Post-Acute Care Transformation Act of 2014 mandated the development and use of Standardized Patient Assessment Data Elements (SPADE) to address this problem.6
The 3 data sources available in the Medicare postacute assessment files up to this point include: (1) the Inpatient Rehabilitation Facility Patient Assessment Instrument, which includes items from the Functional Independence Measure (FIM)7; (2) the Minimum Data Set (MDS), used in skilled nursing facilities8; and (3) the Outcome and Assessment Information Set (OASIS), which is used in home health care settings.9 These measures vary in their coverage of domains of functioning and in their approach to measurement. Although SPADE implementation will support research across settings using these new data elements, it does not solve the problem for analysis of data prior to 2016.
One solution is to use statistical methods to create crosswalks connecting scores from different settings’ assessment-based functional scales. Prior research using item response theory to put the items from 2 separate measures onto the same hierarchical scale showed potential for this approach.10–13 However, no crosswalk has been developed that includes multiple PAC settings. This project focuses on physical functioning, including basic mobility, which includes walking, moving around and carrying; and daily activities, which includes eating, dressing, bathing, and self-care activities.14,15,16 Our overarching objective was to create crosswalks between the data from the postacute care assessment files (inpatient rehabilitation: FIM; skilled nursing: MDS activities of daily living (ADL) scale; home health: OASIS) and the basic mobility and daily activities scales of the AM-PAC, and to test their accuracy and validity using rigorous statistical methods.
To develop the crosswalks, we took advantage of prior research in which detailed data across rehabilitation settings were collected to develop short forms for the Activity Measure for Post-Acute Care (AM-PAC).14 FIM, MDS, and OASIS items in addition to the AM-PAC items were prospectively collected from a sample of adults undergoing rehabilitation, which allowed us to develop statistical crosswalks between these instruments and the AM-PAC basic mobility and Daily Activity scales from which we generated estimated AM-PAC scores (eAM-PAC). We tested the accuracy of the eAM-PAC against the actual AM-PAC scores. We then tested the construct validity of the eAM-PAC in a national sample of Medicare beneficiaries who had a hip, proximal humerus, or distal radius/ulna fracture by assessing the degree to which the scores were consistent with known characteristics of this population.
Methods
Crosswalk Development Dataset
We identified data necessary to build a crosswalk for FIM, MDS, and OASIS to AM-PAC scores in a prior study to develop short forms for the AM-PAC.14 The dataset from the prior AM-PAC study included 485 adults participating in rehabilitation services who were recruited from 6 health care networks in the Boston area. Individuals were inpatient or in skilled nursing or home care settings and provided data at 1 time point for a core set of AM-PAC questions and additional setting-specific questions. For this study, we used the data from 300 individuals for whom FIM, MDS, or OASIS data were available. There were 108 people in inpatient rehabilitation for which FIM data were collected, 89 people in skilled nursing facilities for whom MDS data were completed, and 103 people in home care whose data were recorded for the OASIS.
The sample was categorized into 3 groups: (1) neurological conditions, including stroke, multiple sclerosis, Parkinson disease, spinal cord injury, brain injury, and neuropathy; (2) musculoskeletal conditions, including orthopedic surgeries, fractures, joint replacement, and non-surgical musculoskeletal conditions; and (3) medically complex conditions (cardiopulmonary conditions and other diseases).
Measures
The AM-PAC scales were developed using item response theory to measure patient functional status across postacute settings.14–16 In creating the AM-PAC item banks, content from existing setting-specific postacute care instruments (FIM, MDS, and OASIS) and newly developed items were used to develop 3 scales. The final basic mobility, daily activities, and applied cognitive item banks included 101, 67, and 59 items and were calibrated based on the partial credit model.17,18 AM-PAC logit scores are converted to T scores with a mean of 50 and SD of 10, with higher scores representing better function. In this study, we focused on basic mobility and daily activity scales.
The FIM, used in inpatient rehabilitation settings, includes 18 items with scores on each individual item ranging from 1 (total dependence) to 7 (complete independence).7 The FIM is administered within 36 hours of admission and discharge. Five mobility and 6 self-care items were available for inclusion in this study.
The MDS19 serves clinical, quality improvement, and payment determination functions for nursing home settings and includes over 200 items administered within 2 weeks of admission and then every 3 months. In this study, 8 mobility and 10 self-care and instrumental items were used.
The OASIS9 was developed to allow measurement and payment of care provided in the home care setting. Patients are assessed using OASIS at admission, when there is a significant change in status, and every 60 days. Five mobility and 8 self-care and instrumental items were used in this study.
Crosswalk Creation Methods
The crosswalks were created using methods reported in Velozo et al.11 We used the item parameters from the full AM-PAC basic mobility and daily activities item banks from the prior study14 to estimate person scores for participants with available FIM, MDS and OASIS items (n = 300) and to create the look-up tables for application to the validation dataset. For polytomous items, a sample size of 50 is sufficient to achieve calibration error within ±1 logit with 99% confidence.20–22 Therefore, based on the number of items in each scale and the sample size for each assessment instrument (AM-PAC and FIM: n = 108; AM-PAC and MDS: n = 89; AM-PAC and OASIS: n = 103), we estimated that the sample sizes were sufficient to provide stable calibration estimates. The newly developed crosswalks were used to create look-up tables that translate responses to FIM, MDS, or OASIS assessment data to AM-PAC basic mobility or daily activities scores that can be used across the data sources.
eAM-PAC Score Accuracy Testing
Accuracy was evaluated by comparing actual with estimated scores using correlation analyses and testing differences between means and score distributions. eAM-PAC score accuracy was assessed by comparing the estimated with the actual AM-PAC in the crosswalk development dataset, testing proportion at the highest score (ceiling) and the lowest score (floor) and by comparing mean scores and distributions for all participants and stratified by setting (IRF, SNF, home care). The setting-specific testing was necessary to identify whether performance varied by source data (FIM, MDS, OASIS). Paired t tests were conducted to compare means, and the Kolmogorov-Smirnov test23 was used to compare score distributions. Ceiling and floor values were defined as a score ≤3 points from the maximum or minimum score, respectively.
The actual and estimated scores were also evaluated by calculating the Pearson and intraclass correlation coefficients (ICC [3,1])24 between the AM-PAC and eAM-PAC and by inspection of score differences using Bland-Altman plots.25 Absolute score differences were calculated for each participant as AM-PAC minus eAM-PAC such that a positive score difference indicated underestimation and a negative score indicated overestimation of the AM-PAC score. We categorized the absolute value of the score difference as <5 points, between 5 and 10 points, and ≥10 points and calculated the proportion of the sample within each category.
eAM-PAC Validity Testing
One of the challenges when evaluating measures of physical function is the absence of a gold standard to use as the criterion measure. The recommended approach is to test whether the measure exhibits expected relationships with other constructs, which is called “construct validity” testing.26 To do so, we test hypotheses about correlations and differences in means and distributions using data from a cohort of Medicare beneficiaries with hip, proximal humerus, or distal radius/ulna fracture.
Validation Sample Construction
Validity of the eAM-PAC scores was assessed in a cohort of Medicare beneficiaries with fragility fracture identified from national Medicare fee-for-service claims. The methods used to develop the cohort were previously published, including methodological appendices.27 Briefly, beneficiaries between the ages of 66 and 99 years were included if they were enrolled in Medicare Parts A and B for at least 1 year prior to and after sustaining an index fracture occurring between 2007 and 2010. Fractures were identified using a combination of diagnosis and fracture treatment procedure codes using Medpar, Carrier, and Outpatient Hospital files. For those who sustained fractures and were not admitted to the hospital, claims for radiographs were required in addition to the fracture diagnosis code. Claims for the prior year were examined to ensure that the fracture was new. Diagnosis codes were used to exclude beneficiaries with fractures related to cancer or multi-trauma and to exclude follow-up care for prior fracture.
Demographic and clinical variables were extracted, and the latter were used to calculate Charlson Comorbidity scores28 and categorize them into the following categories: 0, 1, and ≥2 comorbidities. For the validity analysis, data from those with hip, proximal humerus (“humerus”), and distal radius (“radius”) fractures that had postacute assessment data at index and at least 1 postacute assessment were used. Postacute care assessment files were merged to provide FIM, MDS, and OASIS physical functioning data for the assessment closest to the index fracture date and the assessment closest to 1 month after fracture. The crosswalk items were identified in the relevant postacute file, and responses were recoded as needed and summed to obtain a raw score. The look-up tables were used to convert the raw scores from FIM, MDS, and OASIS to the eAM-PAC basic mobility and daily activity scores based on a T score distribution with mean of 50 and SD of 10; higher scores represented better function. Demographic variables were summarized using means and percent. Mean scores were calculated across age and fracture groups, comorbidity levels, and by dementia status.
Construct Validity Analysis
We tested the construct validity of eAM-PAC basic mobility scores in the fracture cohort members from the inpatient rehabilitation setting who had FIM scores, which focus on basic mobility activities. We hypothesized that the eAM-PAC basic mobility scores would be associated with FIM motor scores. We used only FIM in this analysis because the MDS and OASIS measures combine content across basic mobility and daily activities constructs.
Additional hypotheses were developed regarding the construct validity of the eAM-PAC basic mobility and daily activities scores by identifying groups within the validation dataset of older adults. We hypothesized that mean scores would be different across subgroups and that older age, more severe fractures, more comorbidities, and the presence of dementia would be associated with lower eAM-PAC scores.
We tested at baseline (as close as possible to the fracture event) and at 1 month after fracture, which allowed for stabilization from surgical and pain management interventions and more consistency in expected functional status given recent fracture. We tested the following hypotheses: (1) eAM-PAC_FIM basic mobility scores will be moderately correlated (r = 0.4–0.5) with FIM motor scores at baseline and 1 month; (2) at 1 month, mean eAM-PAC basic mobility and daily activities scores will be higher for younger age groups and those with fewer comorbidities; and (3) mean eAM-PAC basic mobility and daily activities scores will be significantly lower for those with dementia. Pearson correlation coefficients were calculated for eAM-PAC_ FIM basic mobility scores and actual FIM motor scores. The t tests and analysis of variance were conducted to evaluate mean differences in eAM-PAC scores across fracture, age, comorbidity, and dementia status groups and to test for interactions between variables.
Role of the Funding Source
The funders played no role in the design, conduct, or reporting of this study.
Results
Crosswalk Development Dataset
In the AM-PAC short-form development study, items were administered to 485 adults who were receiving rehabilitation services. Data were available for a core set of AM-PAC items for everyone in the sample and setting-specific measures depending on the patient’s rehabilitation setting. There were 108 people from inpatient rehabilitation for whom FIM data were collected, 89 people in skilled nursing facilities for whom MDS basic mobility data and 87 for whom daily activities data were collected, and 103 people in home care whose data were recorded for the OASIS.
The sample was described previously.14 To summarize, the mean age was 67.2 (range = 19–94) years. Eighty-nine percent of the sample was White, and 63% was female. Twenty-nine percent of the sample was classified as having neurological, 34% orthopedic, and 37% medically complex conditions. The majority of the inpatient rehabilitation sample had neurological conditions (56%). The nursing home sample was predominantly classified as medically complex (48%), and the home care sample had 44% orthopedic and 39% medically complex conditions. A detailed summary of sample characteristics is provided in Table 4.
Table 4.
Characteristics of Development Sample
| Characteristic | Full Sample (n = 300) | Inpatient Rehabilitation (n = 108) | Nursing Home (n = 89) | Home Care (n = 103) |
|---|---|---|---|---|
| Age (mean, SD, range, No.) | 67.24 (16.21), 18.57 ~ 93.73, 299 | 61.07 (17.87), 19.24 ~ 91.25, 108 | 69.71 (14.4), 22.22 ~ 92.47, 89 | 71.6 (13.82), 18.57 ~ 93.73, 103 |
| Female (No., %) | 185 (62.67) | 55 (50.93) | 58 (65.17)) | 72 (69.9) |
| White (No., %) | 268 (89.33) | 86 (79.63) | 83 (93.26) | 99 (96.12) |
| Married (No., %) | 114 (38.26) (missing = 2) | 40 (37.74) (missing = 2) | 33 (37.08)) | 41 (39.81) |
| High school or less (No., %) | 151 (51.19) (missing = 5) | 53 (49.53) (missing = 1) | 33 (38.37) (missing = 3) | 65 (63.73) (missing = 1) |
| Patient group | ||||
| Neurological | 87 (29) | 60 (55.56) | 9 (10.11) | 18 (17.48) |
| Orthopedic | 101 (33.67) | 19 (17.59) | 37 (41.57) | 45 (43.69) |
| Medically complex | 112 (37.33) | 29 (26.85) | 43 (48.31) | 40 (38.83) |
Comparison of eAM-PAC Scores With Actual AM-PAC Scores
Table 1 summarizes the distributions of actual AM-PAC scores and eAM-PAC scores for basic mobility and daily activities for the overall sample (n = 300) and for the subsamples in each rehabilitation setting. The differences between actual and estimated mean scores were within approximately 1 point for both the basic mobility and the daily activities scales. The t tests of mean difference and Kolmogorov-Smirnov tests revealed statistically significant differences between actual and estimated score for the full sample for basic mobility and daily activities. The Kolmogorov-Smirnov test revealed a statistically significant difference between mean AM-PAC and eAM-PAC score for the home care sample for daily activities. Table 2 summarizes Pearson correlation and agreement (ICC) results for AM-PAC and eAM-PAC scores for the full crosswalk development sample and stratified by setting. All correlations were statistically significant. Correlation coefficient estimates ranged from r = 0.71 to 0.80 for basic mobility and r = 0.75 to 0.78 for daily activities. ICCs ranged from 0.65 to 0.82 and 0.68 to 0.76, respectively.
Table 1.
Comparison of Actual AM-PAC and eAM-PAC Mean Scores in the Full Crosswalk Development Sample and Stratified by Settinga
| AM-PAC b Actual Mean Score (SD) | eAM-PAC c Score Mean (SD) | T Test of Mean Difference: P | Kolmogorov-Smirnov 2-Sample Test | Absolute Score Difference <5 (%) | Absolute Score Difference 5–10 (%) | Absolute Score Difference ≥10 (%) | Ceiling d n (%) | Floor d n (%) | |
|---|---|---|---|---|---|---|---|---|---|
| Basic mobility | |||||||||
| All (n = 300) | 48.94 (9.67) | 49.82 (12.47) | .04 | 0.01 | 54.7 | 30.2 | 15.77 | 3 (1) | 5 (1.67) |
| IRF (n = 108) | 45.79 (10.78) | 46.42 (12.62) | .35 | 0.63 | 58.33 | 30.56 | 11.11 | 1 (0.93) | 5 (4.63) |
| SNF (n = 89) | 49.38 (6.82) | 50.58 (10.11) | .12 | 0.11 | 55.06 | 29.21 | 15.73 | 0 | 0 |
| Home (n = 103) | 51.86 (9.62) | 52.73 (13.39) | .30 | 0.01 | 49.51 | 30.10 | 20.39 | 2 (1.94) | 0 |
| Daily activities | |||||||||
| All (n = 298) | 49.79 (9.12) | 50.92 (12.07) | .01 | 0.03 | 61.07 | 25.84 | 13.09 | 14 (4.7) | 3 (1.01) |
| IRF (n = 108) | 46.15 (8.11) | 47.30 (11.11) | .10 | 0.63 | 70.36 | 21.3 | 8.33 | 4 (3.7) | 0 |
| SNF (n = 87) | 50.94 (7.27) | 52.05 (11.61) | .19 | 0.21 | 56.32 | 27.59 | 16.09 | 5 (5.75) | 0 |
| Home (n = 103) | 52.65 (10.26) | 53.77 (12.58) | .16 | 0.09 | 55.34 | 29.13 | 15.53 | 5 (4.85) | 3 (2.91) |
a Bolded items indicate a statistically significant difference. AM-PAC = Activity Measure for Post-Acute Care; eAM-PAC = estimated Activity Measure for Post-Acute Care; FIM = Functional Independence Measure; IRF-PAI = Inpatient Rehabilitation Facility Patient Assessment; MDS = Minimum Data Set; OASIS = Outcome and Assessment Information Set; SNF = skilled nursing facility.
b AM-PAC score based on patient responses to the full item bank.
c AM-PAC score estimated based on items available from IRF-PAI (FIM), MDS, or OASIS data.
d Percentage of ceiling and floor were calculated as percentage of participants who responded to all items at the lowest or highest available categories in FIM, MDS, or OASIS data.
Table 2.
Pearson and ICC Between AM-PAC and eAM-PAC Scores for the Full Crosswalk Development Sample by Settinga
| Test | Pearson Correlation b | ICC (3,1) (95% CI) |
|---|---|---|
| Basic mobility | ||
| AM-PAC (n = 300) | 0.80 | 0.78 (0.73–0.81) |
| IRF (n = 108) | 0.83 | 0.82 (0.75–0.87) |
| SNF (n = 89) | 0.71 | 0.65 (0.51–0.75) |
| Home (n = 103) | 0.78 | 0.75 (0.63–0.81) |
| Daily activities | ||
| Full sample (n = 298) | 0.78 | 0.74 (0.69–0.79) |
| IRF (n = 108) | 0.77 | 0.73 (0.63–0.81) |
| SNF (n = 87) | 0.75 | 0.68 (0.55–0.78) |
| Home (n = 103) | 0.77 | 0.76 (0.66–0.83) |
a AM-PAC = Activity Measure for Post-Acute Care; ICC = intraclass correlation coefficient; IRF = inpatient rehabilitation facility; SNF = skilled nursing facility.
b P < .0001.
The proportion of participants exceeding the 95% limits of agreement in Bland-Altman plots was as follows: basic mobility: eAM-PAC_ FIM = 3.70 (1.82), eAM-PAC_MDS = 5.62 (2.44), eAM-PAC_OASIS = 6.73 (2.47); daily activities: eAM-PAC_ FIM = 4.63 (2.02), eAM-PAC_MDS = 3.41 (1.95), eAM-PAC_OASIS = 3.85 (1.89). The Bland-Altman plots are provided in Supplementary Appendix 1. The percent at the ceiling and floor were <5% for all estimates.
Crosswalk Validation Dataset
Among 885,000 beneficiaries with an index fracture between 2007 and 2010, there were 171,350 Medicare beneficiaries with baseline and 1-month assessment data available Supplementary Appendix 2. The mean ages by fracture type were 83.3 years for hip (n = 120,235), 80.8 years for humerus (n = 20,256), and 81 years for radius fractures (n = 30,859). Mean eAM-PAC scores and distribution are summarized in Table 3. The mean basic mobility score for all beneficiaries with fracture was 43.2, which is approximately 7 points below the population mean of 50. The mean score for daily activities was 40.6 points.
Table 3.
eAM-PAC Mean Scores Across Age, Charlson Comorbidity Score, Dementia Status, Setting, and Fracture Type for the Validation Sample: Medicare Beneficiaries 1 Month Postfracture
| Basic Mobility | Daily Activities | ||||
|---|---|---|---|---|---|
| N | Mean | SD | Mean | SD | |
| All fractures | |||||
| All beneficiaries | 82,548 | 43.17 | 9.61 | 40.57 | 7.02 |
| Age (y) 65–74 | 10,736 | 46.30 | 11.83 | 42.37 | 9.10 |
| Age (y) 75–84 | 32,673 | 44.02 | 10.03 | 40.93 | 7.35 |
| Age (y) 85+ | 39,139 | 41.60 | 8.17 | 39.78 | 5.86 |
| Charlson 0 | 16,502 | 46.29 | 10.80 | 42.49 | 8.18 |
| Charlson 1 | 17,290 | 43.08 | 9.48 | 40.49 | 6.90 |
| Charlson 2+ | 48,756 | 42.14 | 8.99 | 39.96 | 6.49 |
| Dementia: no | 65,924 | 44.23 | 9.91 | 41.34 | 7.22 |
| Dementia: yes | 16,624 | 38.95 | 6.87 | 37.53 | 5.11 |
| By setting | |||||
| Nursing facility | 55,615 | 40.55 | 5.11 | 40.12 | 4.62 |
| Inpatient rehabilitation | 6298 | 42.23 | 9.09 | 41.52 | 7.94 |
| Home care | 20,635 | 50.52 | 14.18 | 41.53 | 10.93 |
| By fracture type | |||||
| Hip | 70,701 | 42.62 | 9.23 | 40.58 | 6.75 |
| Proximal humerus | 6430 | 45.85 | 10.87 | 40.32 | 7.92 |
| Distal radius | 5417 | 47.16 | 11.25 | 40.77 | 9.00 |
The correlation between FIM motor and eAM-PAC basic mobility scores was r = 0.87 and r = 0.40 at baseline and 1 month, respectively. The basic mobility scores for known groups at 1 month are illustrated in Figure 1. Consistent trends are evident, with higher mean scores for those who did not have dementia, were younger, and had fewer comorbidities. Across groups, mean scores were best for wrist, followed by humerus, and hip fracture.
Figure 1.
Comparison of mean eAM-PAC basic mobility scores in the validation sample at 1 month for the fracture cohort by age, fracture type, Charlson Comorbidity score, and presence of dementia.
Results of tests of differences using ANOVA for the basic mobility scores indicated that mean score differences were significant across age groups (F2, 82539 = 384.9, P < .0005) and fracture type (F2, 82539 = 356.5, P < .0005). Differences in scores by age were influenced by fracture type (interaction for age and diagnosis [F4, 82539 = 4.47, P = .0013]): mean score differences were smaller between humerus and radius fracture groups among those who were the youngest. Mean basic mobility scores were different by Charlson score (F2, 82539 = 297.8, P < .0005) and fracture type (F2, 82539 = 370.6, P < .0005). There was borderline significance for interaction between Charlson score and fracture type (F4, 82539 = 2.4, P = .0469) such that mean score differences between humerus and radius fracture groups were smaller for those with the fewest comorbidities. Significant differences were also found for dementia (F1, 82542 = 732.0, P < .0005) and fracture type (F2, 82542 = 204.7, P < .0005), with no evidence of interaction between these 2 variables (F2, 82542 = 1.7, P = .1880).
Although statistically significant differences were found in means for the daily activities scores, they varied less than basic mobility scores with age, fracture type, comorbidity, and dementia status (Fig. 2).
Figure 2.
Comparison of mean eAM-PAC daily activities scores in the validation sample at 1 month for fracture cohort by age, fracture type, Charlson Comorbidity score, and presence of dementia.
Crosswalk look-up tables and instructions are provided in Supplementary Appendix 3.
Discussion
This study found that a crosswalk based on AM-PAC calibration data showed promise to allow analysis of basic mobility from FIM, MDS, and OASIS data. This is important because a major challenge to understanding the impact of physical therapy interventions has been limited by the lack of uniform functional assessment instruments across postacute settings. The magnitude of the problem is evident in the mandated development and use of standardized assessment data collection across postacute settings required by the Improving Medicare Post-Acute Care Transformation Act of 2014. The results of this study provide an important step in allowing comparability across functional data prior to the implementation of the Improving Medicare Post-Acute Care Transformation Act of 2014 Act.
The basic mobility scores demonstrated expected relationships in the development and validation datasets. Strong correlations between actual and eAM-PAC scores were demonstrated in the development dataset. Moderate to strong correlations were found between the AM-PAC basic mobility and FIM motor scores at baseline and 1 month in the validation dataset, supporting the validity of the crosswalked scores. Differences in basic mobility scores across known groups were statistically significant and appeared to be clinically important. The minimal detectable change is 4.02 for the basic mobility scale and daily activities: 3.72.29 For example, the mean difference in the validation sample between those with and without dementia was 5.28 points, those with ≥2 versus no comorbidities was 4.15 points, and hip versus radius fracture was 4.54. Since AM-PAC scales are based on T scores, the magnitude of a 5-point difference can also be represented as one-half of a standard deviation. The results were less promising for daily activities, where differences between groups were statistically significant but appeared not to be clinically important. Actual and eAM-PAC mean scores in the development dataset were similar. However, there was some divergence: approximately 50% of scores were less than a 5-point difference, and 10% had more than a 10-point difference.
It is important to note that there were little to no ceiling or floor effects, since patients with fragility fractures have a range of functional levels at the time of fracture and undergo various recovery trajectories.
The finding of significant interaction between age and fracture type for the eAM-PAC scores may provide additional support for the validity of the crosswalks. Although we did not specify hypotheses for the direction of the influence, smaller functional differences between humerus and radius fracture groups among those who were the youngest fits with evidence that those with upper extremity fractures are younger and in better health than older adults with hip fracture. However, we would expect a similar pattern for the interaction between Charlson score and fracture type, which was not the case, since mean score differences were smaller for proximal humerus and wrist fractures for those with none or 2 or more comorbidities. Further research is needed to better understand these relationships.
The current study builds on work by Velozo et al,11–13 which created crosswalks between the FIM and the MDS motor and cognitive scales in a sample of patients from Veterans Affairs facilities. The motor crosswalk demonstrated good internal consistency (Cronbach alpha = 0.94), and the distribution of item calibrations was consistent with the literature on functional items.11 Wang et al13 conducted a validation study of the crosswalk among 2130 Veterans Affairs patients. This study found that mean difference in scores between actual and crosswalk-derived scores was small (1.3 pts), and correlations with other functional measures were strong (−0.80). Their analyses tested the derived scores on 3 levels: (1) individual, (2) functional-related groups, and (3) facility level. They concluded that the results supported “population equivalence” and that crosswalk-based scores may be sufficient for group-level analyses in large samples. The crosswalk developed by Velozo et al did not include items from the home health assessment, the OASIS, limiting its usefulness for investigating physical therapy questions across settings that include community-based services. In addition, because Velozo et al used existing VA data, they were limited in their ability to collect data from the FIM and MDS for the same patients. They selected data from patients who were assessed over a very short period of time, making the assumption that their function was essentially the same for both administrations. This was a potential source of measurement error. We aimed to overcome these limitations by using a dataset from a prospective study that includes items from FIM, MDS, OASIS, and AM-PAC.
Limitations
Limitations of this study include a relatively small sample for co-calibration. Although our validation sample was large and highly relevant, there were limited data available with which to test validity hypotheses. All crosswalks are limited by the source data (FIM, MDS, and OASIS), and the error introduced by estimating scores. However, they have potential to provide useful information when applied judiciously. Although the large, nationally representative sample is considered a strength of this study, we investigated the use of the AM-PAC crosswalks in older adults with fragility fracture.
Future Research
In this study, we aimed to provide an approach to address the problem of lack of comparable functional data in postacute Medicare files. Although the results indicate that the basic mobility crosswalk shows promise, there are additional steps that need to be taken to support full comparability across datasets. The crosswalk and validation sample were conducted using the MDS 2.0 item sets. MDS 3.0 contains some differences in the instructions, item definitions, and coding, such as the addition of a new code 7: activity occurred only once or twice. Further work will need to be completed to validate a revised crosswalk to MDS 3.0. In addition, the Medicare Post-Acute Care Transformation Act of 2014 made SPADE available across settings.6 SPADE includes similar items as those included in the FIM, MDS and OASIS. Therefore, it is likely that our approach could be successfully applied to SPADE functional data, allowing crosswalks across data before and after 2016. The SPADEs would also be critical to the development of any unified payment system across the PAC continuum.30,31 As the functional assessment SPADES continue to be added to and validated in each of the institutional PAC and home health settings, these crosswalks could be used to help test cross setting PAC quality or payment models. Finally, future research should investigate the measurement properties in samples other than fragility fractures and across a range of time points. It is possible that this population provided more variation in basic mobility that in daily activities. Therefore, future research should assess the characteristics of the daily activities crosswalk in conditions that most affect those functional activities.
Although further testing is warranted for both crosswalks in Medicare populations with different conditions, the basic mobility crosswalk appears to provide valid scores for analysis of Medicare PAC data.
Conclusions
Although further testing is warranted for both crosswalks in Medicare populations with different conditions, the basic mobility crosswalk appears to provide valid scores for analysis of Medicare PAC data.
Supplementary Material
Contributor Information
Christine M McDonough, Department of Orthopaedics, Geisel School of Medicine at Dartmouth, Hanover, New Hampshire; Department of Health Law, Policy and Management, Boston University School of Public Health, Boston, Massachusetts; and Department of Physical Therapy, School of Health and Rehabilitation Sciences and Department of Orthopaedic Surgery, University of Pittsburgh School of Medicine, 100 Technology Drive Pittsburgh, PA 15219 (USA).
Donald Carmichael, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth.
Molly E Marino, Department of Health Law, Policy and Management, Boston University School of Public Health. Now with: Department of Quality Measurement and Health Policy, Research Triangle (RTI) International, Waltham, Massachusetts.
Pengsheng Ni, Department of Health Law, Policy and Management, Boston University School of Public Health.
Anna N A Tosteson, Department of Orthopaedics, The Dartmouth Institute for Health Policy and Clinical Practice, and the Department of Medicine, Geisel School of Medicine at Dartmouth.
Julie P W Bynum, The Dartmouth Institute for Health Policy and Clinical Practice and the Department of Medicine, Geisel School of Medicine at Dartmouth. Now with: Department of Internal Medicine, University of Michigan Medical School, and the Institute for Health Policy and Innovation, University of Michigan, Ann Arbor, Michigan.
Author Contributions
Concept/idea/research design: C.M. McDonough, M.E. Marino, A.N.A. Tosteson, J.P.W. Bynum
Writing: C.M. McDonough, M.E. Marino, A.N.A. Tosteson, J.P.W. Bynum
Data collection: C.M. McDonough, D. Carmichael, J.P.W. Bynum
Data analysis: C.M. McDonough, D. Carmichael, M.E. Marino, P. Ni, J.P.W. Bynum
Project management: C.M. McDonough, J.P.W. Bynum
Fund procurement: C.M. McDonough, A.N.A. Tosteson, J.P.W. Bynum
Ethics Approval
This study was approved by the Dartmouth Committee for Protection of Human Subjects.
Funding
This research was supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases at the National Institutes of Health (P60-AR062799) (A. Tosteson, principal investigator) and a Center on Excellence in Health Services and Health Policy Research and Training (CoHSTAR) grant from the Foundation for Physical Therapy Research.
Disclosures
The authors completed the ICMJE Form for Disclosure of Potential Conflicts of Interest and reported no conflicts of interest.
References
- 1. Chandra A, Dalton MA, Holmes J. Large increases in spending on postacute care in Medicare point to the potential for cost savings in these settings. Health Aff. 2013;32:864–872. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Deutsch A, Granger CV, Fiedler RC, et al. Outcomes and reimbursement of inpatient rehabilitation facilities and subacute rehabilitation programs for Medicare beneficiaries with hip fracture. Med Care. 2005;43:892–901. [DOI] [PubMed] [Google Scholar]
- 3. Freburger JK, Holmes GM, Ku LJ. Postacute rehabilitation care for hip fracture: who gets the most care? J Am Geriatr Soc. 2012;60:1929–1935. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Ottenbacher KJ, Karmarkar A, Graham JE, et al. Thirty-day hospital readmission following discharge from postacute rehabilitation in fee-for-service Medicare patients. JAMA. 2014;311:604–614. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Qu H, Shewchuk RM, Chen Y, Deutsch A. Impact of Medicare prospective payment system on acute rehabilitation outcomes of patients with spinal cord injury. Arch Phys Med Rehabil. 2011;92:346–351. [DOI] [PubMed] [Google Scholar]
- 6. Centers for Medicare and Medicaid Services . IMPACT Act Standardized Patient Assessment Data Elements. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Post-Acute-Care-Quality-Initiatives/IMPACT-Act-of-2014/-IMPACT-Act-Standardized-Patient-Assessment-Data-Elements.html. 2019. Accessed April 27, 2020.
- 7. Keith RA, Granger CV, Hamilton BB, Sherwin FS. The functional independence measure: a new tool for rehabilitation. Adv Clin Rehabil. 1987;1:6–18. [PubMed] [Google Scholar]
- 8. Mor V. Improving the quality of long-term care with better information. Milbank Q. 2005;83:333–364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Shaughnessy PW, Crisler KS, Schlenker RE. Outcome-based quality improvement in home health care: the OASIS indicators. Qual Manag Health Care. 1998;7:58–67. [DOI] [PubMed] [Google Scholar]
- 10. Buchanan JL, Andres PL, Haley SM, Paddock SM, Zaslavsky AM. Evaluating the planned substitution of the minimum data set-post acute care for use in the rehabilitation hospital prospective payment system. Med Care. 2004;42:155–163. [DOI] [PubMed] [Google Scholar]
- 11. Velozo CA, Byers KL, Wang YC, Joseph BR. Translating measures across the continuum of care: using Rasch analysis to create a crosswalk between the functional independence measure and the minimum data set. J Rehabil Res Dev. 2007;44:467–478. [DOI] [PubMed] [Google Scholar]
- 12. Wang YC, Byers KL, Velozo CA. Rasch analysis of minimum data set mandated in skilled nursing facilities. J Rehabil Res Dev. 2008;45:1385–1399. [PubMed] [Google Scholar]
- 13. Wang YC, Byers KL, Velozo CA. Validation of FIM-MDS crosswalk conversion algorithm. J Rehabil Res Dev. 2008;45:1065–1076. [DOI] [PubMed] [Google Scholar]
- 14. Haley SM, Andres PL, Coster WJ, Kosinski M, Ni P, Jette AM. Short-form activity measure for post-acute care. Arch Phys Med Rehabil. 2004;85:649–660. [DOI] [PubMed] [Google Scholar]
- 15. Haley SM, Coster WJ, Andres PL, et al. Activity outcome measurement for postacute care. Med Care. 2004;42:I49–I61. [DOI] [PubMed] [Google Scholar]
- 16. Coster WJ, Haley SM, Andres PL, Ludlow LH, Bond TL, Ni PS. Refining the conceptual basis for rehabilitation outcome measurement: personal care and instrumental activities domain. Med Care. 2004;42:I62–I72. [DOI] [PubMed] [Google Scholar]
- 17. Linacre JM, Wright BD. A User's Guide to WINSTEPS. Chicago, IL, USA: Mesa Press; 1999. [Google Scholar]
- 18. Wright BD, Masters GN. Rating Scale Analysis: Rasch Measurement. Chicago, IL, USA: Mesa Press; 1982. [Google Scholar]
- 19. Morris JN, Hawes C, Fries BE, et al. Designing the national resident assessment instrument for nursing homes. Gerontologist. 1990;30:293–307. [DOI] [PubMed] [Google Scholar]
- 20. Linacre JM. Sample size and item calibration stability. Rasch Measurement Transactions. 1994;7:328. [Google Scholar]
- 21. Wright B, Douglas G. Best Test Design and Self-Tailored Testing. Chicago, IL: Department of Education University of Chicago; 1975. [Google Scholar]
- 22. Wright B, Panchapakesan N. A procedure for sample-free item analysis. Educ Psychol Meas. 1969;29:23–48. [Google Scholar]
- 23. Conover W. Practical Nonparametric Statistics. New York, NY, USA: John Wiley; 1971. [Google Scholar]
- 24. Shrout PE, Fleiss J. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428. [DOI] [PubMed] [Google Scholar]
- 25. Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. Statistician. 1983;32:307–317. [Google Scholar]
- 26. Mokkink LB, Prinsen CA, Bouter LM, Vet HC, Terwee CB. The COnsensus-based standards for the selection of health measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther. 2016;20:105–113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Bynum JPW, Bell JE, Cantu RV, et al. Second fractures among older adults in the year following hip, shoulder, or wrist fracture. Osteoporos Int. 2016;27:2207–2215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–383. [DOI] [PubMed] [Google Scholar]
- 29. Jette AM, Haley SM, Tao W, et al. Prospective evaluation of the AM-PAC-CAT in outpatient rehabilitation settings. Phys Ther. 2007;87:385–398. [DOI] [PubMed] [Google Scholar]
- 30. Medicare Payment Advisory Commission . MedPAC Report to the Congress: Medicare and the Health Care Delivery System. http://www.medpac.gov/docs/default-source/reports/june-2016-report-to-the-congress-medicare-and-the-health-care-delivery-system.pdf?sfvrsn=0. 2016. Accessed August 13, 2020.
- 31. Medicare Payment Advisory Commission . Report to the Congress: Medicare and the Health Care Delivery System. http://www.medpac.gov/docs/default-source/reports/jun18_medpacreporttocongress_rev_nov2019_note_sec.pdf?sfvrsn=0. 2018. Accessed August 13, 2020.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.