Geographic and Facility Variation in Inpatient Stroke Rehabilitation: Multilevel Analysis of Functional Status

Abstract

Objective

Examine geographic and facility variation in cognitive and motor functional outcomes following post-acute inpatient rehabilitation in patients with stroke.

Design

Retrospective cohort design using Centers for Medicare and Medicaid Services (CMS) claims files. Records from 1,209 rehabilitation facilities in 298 hospital referral regions (HRRs) were examined. 145,460 inpatient records from 2006 through 2009 were analyzed using linear mixed models. Multilevel models were used to calculate the variation in outcomes attributable to facilities and geographic regions.

Setting

Inpatient rehabilitation units and facilities in the United States.

Intervention

Not applicable.

Main Outcome Measure

Cognitive and motor functional status at discharge measured by items in the CMS Inpatient Rehabilitation Facility-Patient Assessment Instrument.

Results

Variation profiles indicated that 19.1% of rehabilitation facilities were significantly below the mean functional status rating (81.58, sd=22.30) with 221 facilities (18.3%) above the mean. Total discharge functional status ratings varied by 3.57 points across regions. Across facilities functional status values varied by 29.2 points with a 9.1 point difference between the top and bottom deciles. Variation in discharge motor function attributable to hospital referral region was reduced 82% after controlling for cluster effects at the facility level.

Conclusions

Our findings suggest that variation in motor and cognitive function at discharge following post-acute rehabilitation in patients with stroke is accounted for more by facility than geographic location.

The majority of studies examining geographic variation in health care outcomes have been conducted in acute care facilities^{1–3, 4;56–89;10} and explore factors that contribute to the use of diagnostic and surgical procedures.^11;12 Researchers have reported that differences in procedures and surgical rates are related to a lack of clinical practice guidelines or agreement among physicians.

Few studies have focused on geographic variation associated with post-acute care settings. Kane and colleagues assessed geographic variation in post-acute care use from 1996 to 1998.¹³ They found a 12% difference in the use of post-acute care attributed to practice style variation, the type of rehabilitation services across regions, and diversity in administrative policy at the local level. Geographic variation has been reported in stroke and orthopedic procedures (e.g., joint replacement) commonly associated with physical rehabilitation.^14–17 Little information is available at a national level regarding variation in functional status outcomes following post-acute stroke rehabilitation.

Following the Affordable Care Act,¹⁸ the Institute of Medicine (IOM) investigated geographic variation including payment structures, access to care, efficiency, and quality of care. The IOM report indicates that 73% of variation in Medicare spending is due to post-acute care use, including rehabilitation.¹⁹ The IOM findings suggest that some regions are more efficient than others at delivering quality care. The report indicates that Medicare reimbursement policies are a driver of variation and excessive healthcare use, because the payment system rewards intensity as opposed to the value and quality of care.¹⁹

Reducing variation is often reported as a method to slow healthcare spending and improve care.^20;2122–24 The IOM report argues that reducing variation is only meaningful if it focuses on inefficient or low quality healthcare delivery.¹⁹ Variation in quality of care and the measures that reflect quality of care are important components related to healthcare policy development and implementation.

Regulators use quality measures within policies and programs to facilitate performance at the facility level. For example, the readmission reduction program is specifically designed to facilitate continuity of care at discharge for acute care. The Federal Register 2015 Final Rule for Medicare Inpatient Rehabilitation Facility Prospective Payment System, proposes functional status at discharge as a quality measure and indicator for rehabilitation.²⁵

Few studies have evaluated geographic variation in quality measures for post-acute care rehabilitation. The purpose of this study was to explore geographic variation in functional status, a CMS proposed quality measure for inpatient stroke rehabilitation. Specifically, we examined regional and facility variation in cognitive and motor function using CMS records of beneficiaries who were discharged from inpatient rehabilitation facilities during 2006 to 2009. Based on previous research¹³ and the recent IOM report,¹⁹ we hypothesized that geographic regions would contribute to differences in cognitive and motor functional quality measures in patients receiving stroke rehabilitation.

METHODS

Research Design

We conducted a retrospective cohort study including secondary analyses of Medicare fee-for-service beneficiaries with stroke who received post-acute inpatient rehabilitation.

Data Sources

We used four CMS claims files: Beneficiary Summary, Medicare Provider Analysis and Review (MedPAR), Inpatient Rehabilitation Facility-Patient Assessment Instrument (IRF-PAI), and the Provider of Service (POS). The Beneficiary Summary, MedPAR, and IRF-PAI were linked using the encrypted beneficiary identification.

Sample

The sample included Medicare fee-for-service beneficiaries 66 years and older with rehabilitation impairment category codes for stroke defined by the CMS (01.1, 01.2, 01.3, and 01.9). The eligible sample included patients who were discharged to inpatient rehabilitation facilities (IRFs) from acute hospitals between January 2006 and June 2009. All patients included in the study were living in the community prior to their stroke (N=55,758). We excluded persons who received previous stroke rehabilitation treatment (n=6,667), those with an atypical length of stay (i.e., ≤3 days in rehabilitation: n=3,232), or who died during the rehabilitation stay (n=61). Patients from rehabilitation facilities that treated less than 10 patients with stroke over the study period (n=338) were also excluded. The final sample included 145,460 patient records from 1,209 inpatient rehabilitation facilities, which is 93% of the eligible sample (Figure 1).

Figure 1.

Study STOBE flow diagram

* Medicare fee for service beneficiaries discharged from acute hospitals between January 2006 – June 2009 with rehab impairments group codes for storke, and who were living in the commnuity prior to hospitalization

Variables

Primary outcome variables were: motor and cognitive function, and total functional status ratings at discharge from post-acute inpatient rehabilitation. Motor and cognitive function were assessed using items from the IRF-PAI.²⁶ The IRF-PAI is administered within three days of admission and three days of discharge, and includes 18 items derived from the Functional Independence Measure (FIM instrument).²⁷ The motor subscale contains 13 items: eating, grooming, bathing, upper body dressing, lower body dressing, toileting, bladder management, bowel management, bed chair and wheelchair transfers, toilet transfers, tub and shower transfers, walking/wheelchair locomotion, and stairs. The cognitive subscale includes 5 items: comprehension, expression, social interaction, problem solving, and memory. Total functional status contained the sum of the 18 FIM Instrument items. All items in the IRF-PAI are rated on a seven-point scale from 1=total assistance to 7=complete independence. The reliability and validity of the items have been studied extensively in patients with stroke and other disabilities and found to be adequate.²⁸

Patient level covariates included demographic variables known to influence rehabilitation outcomes, including age, gender, and race/ethnicity.²⁹ Age was entered as a continuous variable. Race/ethnicity was categorized as non-Hispanic white, non-Hispanic black, Hispanic, and other. Clinical covariates included functional status (IRF-PAI) ratings at admission, length of stay, and comorbidity tier level. Medicare classifies patients upon admission to an inpatient rehabilitation facility into comorbidity tier levels: tier 1, 2, 3 and non-tier.³⁰ Each tier represents expected service utilization during rehabilitation and results in an adjustment in the prospective payment system with tier 1 comorbidities receiving the largest adjustment.^31;32

Facility information was entered in the analysis based on the identification code contained in the MedPAR and POS. There are approximately 1200 IRF facilities nationally dispersed unevenly across the country.³³ Rehabilitation facilities include both freestanding hospitals/centers and units within a hospital.

The geographic unit of analysis was the hospital referral region (HRR) developed by the Dartmouth Atlas group.⁴ There are 306 HRRs in the United States. To construct the HRR variable, we used the Dartmouth Atlas crosswalk linked to the facility level ZIP code in the POS. Eight HRR's did not contain an IRF resulting in 298 HRRs in the analyses described below.

Data Analysis

Functional status outcomes (cognitive, motor, total functional status) were evaluated with multilevel analyses using linear mixed models. In the analyses, patients were nested within facilities, and facilities were nested within HRRs. We estimated the percentage of variance for each outcome attributable to the facilities and HRRs with intraclass correlation coefficients (ICC).³⁴

We used a series of multilevel models to examine the variation due to geographic regions and facilities. Both null models without covariates, and adjusted models controlling for patient characteristics (i.e., age, race/ethnicity, gender, comorbidity, functional status rating at admission, and length of stay) were built to case-mix adjust for factors known to contribute to stroke rehabilitation outcomes.³⁵ We constructed nine multilevel models that included two-level models of patients within facilities, two level models of patients within HRRs, and three level models of patients within facilities within HRRs for cognitive, motor, and total functional status outcomes.

We calculated ICCs for each level (e.g. facility, regions), which reflect the proportion of variation in function attributable to the level, for all two and three-level models. The only difference between two and three-level models is the introduction of the additional level. This allows us to use a ratio of the corresponding three and two-level ICCs to determine the relative reduction in variation due to facilities and HRRs, [1−(3 level ICC/2 level ICC)]. This approach is based on a method for comparing reductions in explained variance within multilevel model proposed by Snijders and Bosker.³⁶

We also constructed variation profile graphs by facilities and regions. The facility and region specific predictors were derived from both two-level and three-level models. Patient characteristics were adjusted for using the mean for continuous covariates and frequencies for categorical variables to determine the facility and HRR value representing the average age LOS, admission FIM rating and the distribution of gender race and comorbidity tiers across the study sample. These facility and region specific values were plotted by rank. We repeated our analysis for each outcome using all facilities (e.g., including facilities with less than 10 records) to assure that the effects were not due to facilities with low numbers of stroke patients. The results were similar, data not shown.

We examined statistical assumptions for linear mixed models, whether the error terms at each level were normally distributed, independent and homoscedastic, through residual plots. We didn’t find serious violation within and between facilities or regions. We also calculated Mahalanobis distances to detect multivariate outliers at the high level units. Our results were robust for the models without extreme outliers. All analyses were performed with SAS version 9.2 (SAS Inc. Cary, NC).

The study was approved by the Institutional Review Board. Medicare files were obtained from the CMS. A data use agreement was completed meeting CMS requirements.

RESULTS

Our sample of 145,460 patient records was 51.6% female with a mean age of 78.40 years (sd=7.20). The sample was 80% white, 12% black, 5% Hispanic and 3% other. The typical patient had 8.24 comorbidities. Eighty percent of the comorbidities were non-tier level based on the CMS criteria with 17% low, 2% medium and 1% high-tier.^37;38 The sample mean admitting FIM cognitive function rating was 20.30 (sd=7.60), FIM motor rating was 36.30 (sd=13.60), and total FIM rating was 56.7 (sd=18.6). At discharge, the mean cognitive function rating was 24.60 (sd=7.00), motor rating was 56.90 (sd=22.3), and total FIM rating was 81.5 (sd=22.3). The mean LOS was 16.27 days (sd=8.50), and 69% of the patients were discharged to the community. Of the 1,209 facilities, 44.1% (n=569) had > 100 stroke patients during the study period. One-hundred and twelve (37.6%) of the 298 HRRs included 100 to 300 stroke patients. Table 1 includes unadjusted patient level demographic and clinical factors across the 1,209 facility and 298 HRRs.

Table 1.

Patient level descriptive statistics and clinical factors (unadjusted)

Age (mean, sd)	78.4 (7.2)	Adm Function Rating* (mean, sd)
Female (%)	51.6	Cognitive	20.3(7.6)
Race/Ethnicity (%)		Motor	36.3(13.6)
White	80.0	Total	56.7(18.6)
Black	11.8	Disch Function Rating (mean, sd)
Hispanic	4.9	Cognitive	24.6(7.0)
Other	3.3	Motor	56.9(17.7)
Comorbidity Level (%)		Total	81.5(22.3)
Non-tier	80.0	Length of Stay (mean, sd)	16.2(8.5)
Low-tier	17.5
Medium-tier	1.7
High-tier	0.8

^*Function rating from Inpatient Rehabilitation Facility-Patient Assessment Instrument (IRF-PAI)

Table 2 presents the results of the multilevel analyses. In the null multilevel models of patients within facilities without covariates (Table 2, Panel 1), facility accounted for 6.2% of the variance in total functional status ratings, with facility accounting for 8.5% and 5.9% of FIM cognition and motor function at discharge. When we entered demographic and clinical covariates into the models, the amount of variation for cognitive and motor outcome ratings was 8.7% and 9.5%.

Table 2.

Multilevel models of cognitive, motor, and total functional status ratings. Panel 1 shows 2-level patient-facility. Panel 2 shows patient-region. Panel 3 shows 3-level model of patient-facility-region. Covariate models include patient level demographic and clinical factors.

Panel 1: Two-Level Model: Patients within Facilities
	ICC (%)- Null Model	ICC (%)– With Covariates*
Cognition at Discharge	8.5 %	8.7 %
Motor at Discharge	5.9 %	9.5 %
Total Function Rating	6.2 %	8.8 %
Panel 2: Two-Level Model: Patient within Hospital Referral Regions (HRR)
	ICC - Null Model	ICC – With Covariates*
Cognition at Discharge	4.4 %	4.3 %
Motor at Discharge	3.7 %	4.3 %
Total Function Rating	3.7 %	3.8 %
Panel 3: Three-Level: Patients within Facilities within Regions
	ICC - Null Model	ICC– With Covariates*
Cognition at Discharge	HRR = 0.98 % Facility = 7.5 %	HRR = 0.95 % Facility = 7.8 %
Motor at Discharge	HRR = 1.6 % Facility = 4.3 %	HRR = 0.78 % Facility = 8.7 %
Total Function Rating	HRR = 1.4 % Facility = 4.8 %	HRR = 0.49 % Facility = 8.3 %

FIM = function rating from the Inpatient Rehabilitation Facility-Patient Assessment Instrument (IRF-PAI)

ICC = Intraclass correlation coefficient

^*Adjusted for age, gender, race, admission function rating, comorbidity, and length of stay

In the two level models of patients within HRR's (Panel 2), the null model ICCs indicated 3.7% of the variance in total discharge functional status rating was attributable to the hospital referral region. ICCs for the HRRs accounted for 4.3% of the variance in FIM cognitive and motor discharge ratings after adjusting for age, gender, race/ethnicity, admission functional status, comorbidity level, and length of stay. For the three level models (Panel 3), the ICCs differed, with facility accounting for a larger portion of the variance compared to HRRs in the null model as well as the model with demographic and clinical factors.

Examining FIM motor function at discharge using the three level and two level models [e.g., 1− (3 level facility ICC/ 2 level facility ICC)=1− (8.7/9.5)=8%], suggests that the variation of discharge motor functional status attributed to facility is reduced 8% after accounting for geographic region. Conversely, the explained variation in discharge FIM motor function attributed to the HRR level was reduced 82% after accounting for facility [e.g., 1− (3 level HRR ICC/ 2 level HRR ICC) = 1−(0.78/4.3) = 82%].

To further examine the difference in explained variation between facilities and HRRs we constructed variation profiles. Figure 2 shows facility level variation profiles and Figure 3 displays the HRR level variation profiles. The figures include the two level models (top panel) and three level models (bottom panel). Each vertical line in the graphs represents the adjusted mean of discharge total functional status rating with its 95% prediction interval for one facility (Figure 2) or one HRR (Figure 3). The facility level variation profile for the three level model (Figure 2), indicates that 231 (19.1%) facilities were significantly above the overall mean discharge functional status rating, and 221 (18.3%) of the facilities were significantly below the overall mean. In the HRR level variation profile for the three level model (Figure 3), one HRR was significantly above the overall mean discharge functional status rating and one HRR was significantly below the overall mean.

Figure 2.

Facility level variation profiles for the two level model with covariates (patients – facilities: top), and the three level model with covariates (patients-facilities-regions: bottom). Horizontal lines present the overall adjusted mean of total function rating across facilities (81.75 point for the two level model and 81.58 for the three level model). Each vertical line represents the adjusted mean of total function rating with its 95% confidence interval for one of the 1209 facilities nationally. Black and dark gray lines reflect the 231 IRFs (19.1%) below and the 221 (18.3%) above the overall average in the three-level model (292 and 264 IRFs were below and above the overall average in the two level models).

FIM = function rating from the Inpatient Rehabilitation Facility-Patient Assessment Instrument (IRF-PAI)

Figure 3.

Region level variation profiles for the two level model with covariates (patients –regions: top), and the three level model with covariates (patients-facilities-regions: bottom). Each line represents an HRR*. Horizontal lines present the overall adjusted mean of total function rating⁺ across HRR regions (81.50 point for the two level model and 81.58 for the three level model). Black lines represent the one HRR below the overall average and the dark gray line reflects the one HRR significantly above average in the three level model (80 HRRs below the overall average and 66 HRRs above the overall average in the two level models).

FIM = function rating from the Inpatient Rehabilitation Facility-Patient Assessment Instrument (IRF-PAI), HRR = Hospital Referral Region

Examination of deciles of total functional status ratings support the finding that facilities demonstrate more explained variation in functional status outcomes than geographic region (Table 3). Table 3 shows facility and HRR level variation from the multilevel model alongside unadjusted function ratings. Based on the three level model, total discharge functional status ratings for the HRR level were within a 3.57 point range. Variation in functional status ratings at the facility level ranged from 70.1 to 99.3 (a 29.2 point range), after adjusting for patient level demographic and clinical factors.

Table 3.

Facility and region level variation in discharge functional status by percentile. With the multilevel models for total discharge functional status facilities varied 29.2 compared to only 3.57 points across regions.

Facility Level (N=1209)	Discharge Functional Status (unadjusted)	Discharge Functional Status (3 level model)*
Minimum	61.8	70.1
10th percentile	74.6	77.3
25th percentile	78.6	79.3
50th percentile	82.6	81.2
75th percentile	86.8	83.7
90th percentile	90.5	86.5
Maximum	104.7	99.3
Region Level (N=298)
Minimum	69.2	79.6
10th percentile	76.9	81.2
25th percentile	79.3	81.4
50th percentile	82.4	81.6
75th percentile	85.3	81.8
90th percentile	88.7	82.1
Maximum	102.8	83.2

^*Adjusted for age, gender, race, admission function rating, comorbidity, and length of stay

DISCUSSION

We examined variation in patient functional status following stroke rehabilitation at the levels of the facility and the hospital referral region. Our findings suggest facility effects account for more variation in functional status following inpatient stroke rehabilitation than geographic region. Each of our multilevel models supports this interpretation. In the two level models, which were null models of patients within facilities, we identified substantial explained variation in cognitive and motor outcomes. When we introduced common patient covariates known to affect functional status, the amount of variation at the facility level increased as a function of the ICC (Table 2, Panel 1). It is common to find that the ICC decreases with conditioning for individual characteristics compared to a null model without individual covariates. This would occur, for example, if differences between facilities in case-mix accounts for some of the differences between facilities in discharge outcomes.³⁹ To the contrary, our data show that variation in FIM motor function increased, and, to a lesser extent, variation in FIM cognitive function also increased when individual covariates were entered into the models. This result suggests that case-mix masks variation in improvement in functional outcome at different facilities. This could happen if systematic selection of patients at admission suppressed real differences between facilities in funtional increases. This result suggests that admission practices or patterns of care across facilities influence functional outcomes compared to the unadjusted data. This facility level effect is also supported in the variation profiles depicting the 1,209 facilities (Figure 2) and the 298 HRRs (Figure 3).

Our results provide potential support for the application of Donebidian’s⁴⁰ model for health service evaluation and quality of care in stroke rehabilitation. Researchers have identified difficulties in measuring structure level variables in stroke rehabilitation due to differences in stroke guidelines and treatment approaches.⁴¹ Our data suggest that facility level variation in functional status outcomes is potentially linked to delivery of care issues within IRF settings.

As CMS moves forward with identifying and implementing functional status as a quality measure for rehabilitation, it is critical that facility administrators, policy makers, and researchers explore facility strategies that influence variation in outcomes of care. The IOM report highlighted the need to focus efforts on improving the quality of rehabilitation among low performing regions.¹⁹ Our study of functional status at discharge suggests that improvements that minimize regional variation in post-acute care cannot focus solely on regional reimbursement adjustments.

Stroke rehabilitation is a complex process. Research indicates that facilities with structured teams guided by organizational procedures and management tools yield superior stroke outcomes.^42–44 Facilities that use discharge planning instruments and algorithms that incorporate stroke practice guidelines with functional status have improved outcomes.^43;45 Other research has shown that facilities with dedicated stroke units and accredited programs produce enhanced outcomes.⁴⁶ Additional research is needed to examine structure variables at the facility level, such as space, equipment, staff availability and training. At the process level, we included length of stay in our models. Additional facility level process factors such as discharge planning and continuity of care protocols should be examined. Facility guidelines for completing functional assessments and coding practices should also be explored. These and other process measures are possible targets for improving healthcare quality,^23;40;47;48 in inpatient rehabilitation and other post-acute settings. A logical next step in identifying effective stroke rehabilitation structures, processes and interventions will be to conduct facility level studies examining the 221 inpatient rehabilitation facilities above the mean functional status rating in our sample compared to those used by facilities below the mean functional status.

Our study has limitations. We examined records for Medicare fee-for-service beneficiaries, and our findings may not generalize to beneficiaries enrolled in the Medicare Advantage (HMO) program. We did not have access to information regarding other post-acute-care venues such as home health, long-term acute care, and skilled nursing facilities. We used the CMS tier level comorbidity classifications and admission functional status as covariates for severity in our models,⁴⁹ which limits our ability to evaluate the effect of select comorbidities on regional and facility variation. Although we adjusted for patient level characteristics, there are numerous patient factors which may influence facility and region level variation including primary language and educational level that should be further explored.

Our data have no specific information on the selection process for rehabilitation and future studies should explore selection bias related to IRF admission. Our models also lacked important measures of facility and geographic factors such as urban and rural designations, or the number of healthcare workers per 100,000 population in a HRR. Finally, the alignment of the Dartmouth Atlas HRR algorithm based on acute care hospitals may not be appropriate for post-acute rehabilitation facilities and could contribute to the lower level of variation we found by geographic region.

CONCLUSION

In contrast to research in other areas of healthcare examining geographic variation, our results indicate that variation in cognitive and motor function at discharge in persons receiving post-acute stroke rehabilitation is7more strongly related to differences in facilities compared to variation in geographic region. Continued research to understand how facility level structure and process variables impact functional outcomes should provide insights regarding how to improve the quality of care for patients with stroke receiving post-acute inpatient rehabilitation.

Abbreviations

CMS

Centers for Medicare and Medicaid Services

FIM™ instrument

Functional Independence

HRR

Hospital Referral Region

IRF

Inpatient Rehabilitation Facility

IRF-PAI

Inpatient Rehabilitation Facility-Patient Assessment Instrument file

IOM

Institute of Medicine

ICC

Intraclass Correlation Coefficients

MedPAR

Medicare Provider Analysis and Review file

POS

Provider of Service file