Abstract
Objectives
To explore accuracy of rehabilitation service use reports by older adults and variation in accuracy by demographic characteristics, time since use, duration, and setting (inpatient, outpatient, home).
Design
We calculate the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of survey-based measures from an observational panel study, the National Health and Aging Trends Study (NHATS), relative to measures developed from linked Medicare claims.
Participants
Community-dwelling sample of Medicare fee-for-service beneficiaries in 2015 NHATS who were enrolled in Medicare Parts A and B for 12 months prior to their interview (N=4,228).
Main Outcome Measures
Respondents were asked whether they received rehabilitation services in the last year and the duration and location of services. Healthcare Common Procedure Coding System codes and Revenue Center codes were used to identify Medicare-eligible rehabilitation service.
Results
Survey-based reports and Medicare claims yielded similar estimates of rehabilitation use over the last year. Self-reported measures had high sensitivity (77%) and PPV (80%) and even higher specificity and NPV (approaching 95%). However, in adjusted models sensitivity was lower for Black enrollees, the very old, and those with lower education levels.
Conclusions
Survey-based measures of rehabilitation accurately captured use over the past year but differential reporting should be considered when characterizing rehabilitation use in certain subgroups of older Americans.
Keywords: survey research, measurement, aging
As the U.S. population continues to age, increasing numbers of older adults are using physical rehabilitation services to restore, maintain, improve or slow deterioration of functioning related to illness or injury. Nationally-focused studies of rehabilitation in later life typically rely on administrative sources to study a single condition within a single setting or to examine post-acute rehabilitation following hospitalization.1-3 Yet older adults receive rehabilitation for many different conditions and in many different settings (e.g. at home, medical offices, hospitals, nursing homes, and inpatient rehab facilities), often without prior hospitalization. Consequently, little is known about the basic epidemiology of rehabilitation use in the U.S. at a national level, particularly for older adults.
To address this gap, in 2015 the National Health and Aging Trends Study (NHATS), a longitudinal observational study of over 8,000 adults ages 65 and older in the U.S., included a new series of questions about rehabilitation.4 The survey questions were added to a validated protocol administered annually to measure the physical and cognitive capacity of older adults; their physical, social, technological, and service environment; accommodations such as use of assistive technology and help from another person; and activity limitations and restrictions in valued activities.5 Asking about rehabilitation in the context of a national panel survey on disability and aging has several distinct advantages. Characteristics of beneficiaries may be compared across broad types of settings; functional assessments from interviews conducted before and after use can be compared; and detailed differences between beneficiaries who receive and do not receive rehabilitation services can be examined. Moreover, non-sensitive data are made publicly available and are easier to access than administrative sources
Health services researchers have long recognized the value and challenges of asking adults about their utilization of acute and ambulatory services in a survey context.6 Unlike administrative sources, surveys provide utilization for all respondents, not just a subset linkable to claims. Yet, adults’ ability to recall utilization can vary for different types of services and decay with time, making it challenging to collect high-quality data. A common approach for evaluating accuracy of self-reports is to compare responses to an administrative source, typically medical records or insurance/billing claims, and calculate the agreement between the two. A systematic review concluded that self-reported questionnaires were a valid method for assessing health care use generally, although accuracy varied by visit type.7 For instance, for a national sample of older adults in the U.S., there was high agreement for inpatient stays but underreporting of office visits.8 Subsequent studies have clarified that accuracy decreases with the duration of the recall period and varies with the saliency of the event.9-11 To date, however, the literature has not provided an assessment focused on agreement between administrative sources and patient reports of rehabilitation.
Recent analysis of NHATS has found that one in five older adults reported use of rehabilitation services in the prior year and that use was significantly lower among black individuals and higher among those with greater levels of completed education.12 Given that such conclusions may be sensitive to under- or over-reporting, it is of interest to investigate not only how accurately older adults report their experience with rehabilitation services but also whether accuracy varies with key demographic characteristics. The main objective of this paper is to explore how well older adults in the 2015 NHATS report their experience with rehabilitation service use in the last year and how accuracy varies with demographic characteristics of older adults. We also explore accuracy by time since last service use, duration of service use, and type of setting.
METHODS
Study Design, Setting, and Participants
The first round of NHATS gathered information in person from a nationally representative sample of over 8,245 adults aged 65 and older in 2011. The enrollment file for the Medicare program, the federal health insurance program covering 97% of older adults in the U.S., served as the sample frame.13 Oversamples at older ages and of Black enrollees were included and sample weights were provided that allow analysts to take into account differential probabilities of selection and nonresponse.14
Annual in-person reinterviews have been conducted with NHATS participants in their homes to document change over time. The sample was replenished in 2015 to maintain its ability to represent the older Medicare population. In about 6% of cases in 2015, the sample person was unable to respond for themselves (most often the result of dementia, illness, or a hearing impairment) and a proxy familiar with the respondent’s daily routine provided answers. NHATS protocols were approved by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board.
This analysis draws upon the community-dwelling sample of Medicare beneficiaries in 2015 who were asked about their experience with rehabilitation services in the last year (N=7,499). We restricted the sample to 4,235 individuals (56%) who were enrolled in Medicare’s fee-for-service program (both Parts A and B) for 12 months prior to their 2015 interview, so that claims from all sites were available for a full year. We also excluded 7 individuals reporting “don’t know” to the survey question about rehabilitation use. The final analytic sample consisted of 4,228 individuals. Weighted cross-tabulations suggest that compared to those omitted, the analytic sample is older, more likely to be non-Hispanic white, more highly educated, and more likely to report rehabilitation use.
Survey-based Measures of Rehabilitation
Beginning in 2015, NHATS asked about use of rehabilitation services in the last year. Respondents were told “Physical rehabilitation or ‘rehab’ services can help you improve function and the ability to carry out daily activities. Services include physical therapy, occupational therapy, and speech therapy. Rehab can be received in different settings. For instance, while you are staying in the hospital, after a hospital stay in a nursing home or rehab facility, at a doctor’s or therapist’s office or clinic, or at home. In the last year, {have you/has [name]} received any rehab services?” Respondents who replied “yes” were asked follow-up questions including the duration of services (number of months, and if less than one month, number of weeks) and where services were received (e.g. as an overnight patient, as an outpatient, at home).
Claims-based Measures of Rehabilitation
We examined both institutional and non-institutional claims files for 2014 and 2015. A 1-year time frame was defined for each respondent by setting their 2015 interview date to the 15th of the month (for May to November interviews) and subtracting 365 days. For noninstitutional claims, dates of service were used. For institutional home health and outpatient claims, the claim was retained if any visit date was within the window. For institutional skilled nursing facility and inpatient claims, any claim with a date of service range that fell within the window was included.
For noninstitutional outpatient and home health services we used a subset of the Center for Medicare and Medicaid Services 2015 therapy Healthcare Common Procedure Coding System (HCPCS) codes that indicated provision of therapy (as opposed to evaluation or testing). HCPCS codes included: 92526 treatment of swallowing dysfunction; 92507 treatment of speech, language, voice, communication, and/or auditory processing disorder; 97110 therapeutic exercises to develop strength and endurance, range of motion, flexibility; 97112 neuromuscular reeducation; 97113 aquatic therapy; 97116 gait training; 97124 massage therapy; 97140 manual therapy techniques; 97150 group therapy; 97530 dynamic activities; and 97535 self-care/home management training. We also required the use of a discipline modifier (GP, GN, GO), which indicates that the provider was a physical therapist, occupational therapist, or speech/language pathology provider, respectively. For institutional inpatient, skilled nursing facility, and home health agency services, we used the following revenue center codes: 0420-0424, 0429 physical therapy; 0430-0434, 0439 occupational therapy; and 0440-0444, 0449 speech language pathology. For institutional outpatient services, both HCPCS (and modifiers) and revenue center codes were available and used.
Finally, we required individuals to have at least two dates of service or claims to count as receiving rehabilitation services. For cases with only institutional claims (N=132), most of the single claims were hospital-based evaluation sessions prior to discharge, which we assumed respondents would not recognize as receiving rehabilitation services. For cases with only non-institutional claims, only a small number had exactly one therapy visit (N<20) in the 1-year window of interest.
Statistical Approach
We calculated for the full sample four measures of agreement, each suggesting a distinctive error: sensitivity (% responding yes on the survey among those with claims indicating rehabilitation service use), specificity (% responding no among those without such claims), positive predicted value (PPV; % with claims among those responding yes) and negative predicted value (NPV; % without claims among those responding no).15,16 To facilitate comparisons with prior studies, we also calculated the percentage agreement and κ (a measure of agreement beyond chance).17
We then calculated sensitivity, specificity, PPV and NPV by: primary race (nonHispanic black, nonHispanic white, other), sex (male, female), age group (65-74, 75-84, 85+), respondent type (self, proxy), completed education level (less than high school, high school graduate or equivalent, more than high school), and geographic region (Northeast, South, Midwest, West). We also examined sensitivity by months since rehabilitation service use (according to the claims) and PPV by duration of rehabilitation use (according to the survey; < 1 week; 1-2 weeks; 3-4 weeks; 1-3 months; 4-5 months; >=6 months). We also calculated each measure for the full sample by location of service (inpatient, outpatient, home).
Finally, we estimated four logistic regression models with outcomes (yes on the survey, no on the survey, yes based on claims, no based on claims) aligned with samples (yes based on claims; no based on claims; yes on the survey; no on the survey) so that they corresponded to sensitivity, specificity, PPV and NPV, respectively. We included in these models demographic variables and for sensitivity and PPV models relevant timing variables (months since use for sensitivity and duration for PPV). For the models, we assigned the modal response to the small number of cases responding don’t know or refused to education or duration (<3%). In sensitivity analyses we dropped these cases and found results were essentially unchanged (not shown). Goodness of fit tests confirmed no lack of fit.18
All tabulations and models were estimated using analytic weights and standard errors take into account NHATS’ complex sample design.
RESULTS
Among older adults living in the community and enrolled in Medicare fee-for-service plans, 21.8% (95% CI 20.0-23.5) reported having used rehabilitation services in the last year and 22.6% (95% CI 20.6-24.5) had claims indicating that rehabilitation services were received (see Table 1). More than three-fourths (76.9%) of older adults who received rehabilitation according to the claims analysis reported having received them in the last year and 94.3% of those who did not receive rehabilitation according to the claims analysis reported that they did not receive rehabilitation services in the last year. The PPV and NPV are both relatively high, 79.7% (95% CI 76.4-83.1) and 93.3% (95% CI 92.4-94.2), respectively. Overall agreement (90.4%) is high and κ (.72) is substantial (not shown).
Table 1.
Percentage Using Rehabilitation in the Last Year by Source and Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV) of Survey-based Measure, by Demographic Characteristics (95% CI)
| Weighted % (n) | %-Survey | %-Claims | Sensitivity | Specificity | PPV | NPV | |
|---|---|---|---|---|---|---|---|
| All | 100.0 (4228) | 21.8 (20.0-23.5) | 22.6 (20.6-24.5) | 76.9 (74.5-79.3) | 94.3 (93.3-95.3) | 79.7 (76.4-83.1) | 93.3 (92.4-94.2) |
| Non-Hispanic White | 82.5 (3105) | 22.1 (20.1-24.0) | 22.2 (19.9-24.4) | 79.2 (76.3-82.2) | 94.2 (93.0-95.3) | 79.5 (75.5-83.5) | 94.1 (93.1-95.1) |
| Non-Hispanic Black | 6.9 (753) | 17.5 (14.5-20.6) | 23.6 (20.6-26.6) | 63.2 (56.3-70.1) | 96.6 (94.3-98.8) | 85.1 (76.2-93.9) | 89.5 (87.1-91.8) |
| Other Race | 10.6 (370) | 22.1 (17.9-26.2) | 25.0 (19.5-30.6) | 69.5 (63.4-75.6) | 93.8 (90.5-97.1) | 78.9 (66.8-90.9) | 90.2 (86.8-93.6) |
| Male | 44.3 (1784) | 19.4 (17.1-21.7) | 19.5 (17.0-22.0) | 75.2 (71.5-78.8) | 94.1 (92.4-95.7) | 75.4 (69.3-81.5) | 94.0 (92.8-95.2) |
| Female | 55.7 (2444) | 23.6 (21.5-25.7) | 25.0 (22.9-27.1) | 78.0 (74.5-81.4) | 94.5 (93.2-95.8) | 82.6 (79.0-86.2) | 92.8 (91.5-94.1) |
| Age 65-74 | 53.1 (1469) | 19.5 (17.3-21.8) | 18.7 (16.2-21.3) | 78.6 (75.0-82.3) | 94.1 (92.7-95.4) | 75.4 (70.4-80.3) | 95.0 (93.8-96.3) |
| Age 75-84 | 32.6 (1693) | 23.5 (21.0-26.0) | 24.3 (21.9-26.8) | 78.3 (74.1-82.6) | 94.2 (92.7-95.7) | 81.3 (76.6-85.9) | 93.1 (91.7-94.5) |
| Age 85+ | 14.3 (1066) | 26.1 (23.3-29.0) | 32.7 (29.8-35.7) | 70.8 (64.6-76.9) | 95.6 (94.0-97.2) | 88.7 (84.7-92.6) | 87.1 (84.1-90.0) |
| Self-Respondent | 95.8 (3980) | 21.3 (19.6-23.0) | 21.9 (20.0-23.8) | 77.3 (74.8-79.9) | 94.4 (93.3-95.4) | 79.4 (75.8-82.9) | 93.7 (92.8-94.6) |
| Proxy Respondent | 4.2 (248) | 31.5 (24.0-38.9) | 37.7 (29.2-46.2) | 71.1 (62.4-79.9) | --a | --a | 84.1 (77.9-90.3) |
| < High School | 14.0 (793) | 17.0 (13.2-20.8) | 21.6 (17.9-25.2) | 64.9 (57.9-71.9) | 96.2 (94.0-98.4) | 82.4 (75.8-89.1) | 90.9 (88.0-93.7) |
| High School Graduate | 25.2 (1092) | 22.2 (19.5-25.0) | 23.8 (20.5-27.2) | 74.0 (69.0-79.1) | 94.0 (92.3-95.6) | 79.4 (75.1-83.6) | 92.0 (89.7-94.4) |
| More than High School | 58.0 (2274) | 22.2 (20.1-24.3) | 21.7 (19.4-24.0) | 81.4 (78.0-84.8) | 94.3 (92.7-95.8) | 79.7 (74.5-84.9) | 94.8 (93.7 -96.0) |
| Northeast | 17.9 (672) | 24.7 (19.1-30.4) | 26.7 (21.0-32.3) | 79.3 (74.8-83.8) | 95.1 (92.0-98.3) | 85.5 (77.8-93.3) | 92.7 (90.2-95.2) |
| Midwest | 22.0 (980) | 19.4 (14.9-23.9) | 21.3 (16.4-26.3) | 72.6 (66.0-79.2) | 95.1 (92.7-97.5) | 80.0 (71.3-88.6) | 92.7 (90.4-95.1) |
| South | 40.8 (1852) | 23.1 (20.6-25.6) | 22.8 (19.9-25.6) | 78.8 (75.1-82.5) | 93.4 (91.8-94.9) | 77.8 (72.6-83.1) | 93.7 (92.4-95.0) |
| West | 19.4 (724) | 19.0 (16.1-21.9) | 19.7 (16.2-23.1) | 74.6 (67.5-81.6) | 94.6 (92.4-96.8) | 77.2 (68.3-86.2) | 93.8 (91.8-95.9) |
Suppressed due to cell size.
Underestimates of use in the last year varied with the characteristics of older adults and were more substantial for enrollees who were Black, age 85 or older, with a proxy respondent, with less than a high school education, and lived in the Northeast. Lower usage for Black enrollees that was evident in the survey responses was not replicated in the fee-for-service claims and age gradients appeared steeper based on claims than on survey responses.
Sensitivity varied by all characteristics considered here and was especially low for enrollees who were Black and those who completed less than a high school education. In addition, sensitivity declined from nearly 90% for use in the previous 3 months to 60% for 9-12 months ago (Figure 1). Specificity and PPV were more robust across demographic groups, although adults ages 85 and older had a PPV of 88.7%—substantially higher than other age groups. PPV also increased with the duration of rehabilitation use and reached above 80% after 3-4 weeks (Figure 2). NPV was in the 85%-95% for all groups, but was lower for those ages 85 and older and those with a proxy respondent.
Figure 1.
Sensitivity of Rehabilitation Use in the Last Year by Months Since Use
Figure 2.
Positive Predicted Value of Rehabilitation Use in the Last Year by Reported Duration
In fully controlled models (see Table 2), the odds ratio for sensitivity was lower for enrollees who were Black vs. White (OR=0.47, 95% CI 0.30-0.73), ages 85 or older vs. ages 65-74 (OR=0.61, 95% CI 0.39-0.94), for those with a high school degree (OR=0.63, 95% CI 0.44-0.90) or less education (OR=0.43, 95% CI 0.25-0.73), and by months since rehabilitation (0.83, 95% CI 0.79-0.88). Sensitivity was also higher for women than for men (OR=1.45, 95% CI 1.06-1.99). Results for specificity and PPV were similar to descriptive findings; for the latter, duration of rehabilitation use was an especially strong predictor. NPV varied significantly with race (OR=0.53, 95% CI 0.37-0.77 for Black vs. White enrollees), age group (OR=0.71, 95% CI 0.51-0.99 for ages 75-84 and 0.39, 95% CI 0.25-0.61 for ages 85 and older and vs. ages 65-74) and respondent type (OR=0.52, 95% CI 0.29-0.92 for proxy vs. self-respondents).
Table 2.
Odds Ratios (95% CI) for Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV) of Survey-based Measure of Rehabilitation Use in the Last Year
| Sensitivity | Specificity | PPV | NPV | |||||
|---|---|---|---|---|---|---|---|---|
| NonHispanic White | Ref. | Ref. | Ref. | Ref. | ||||
| NonHispanic Black | 0.47 | (0.30-0.73) | 1.79 | (0.89-3.60) | 1.58 | (0.72-3.44) | 0.53 | (0.37-0.77) |
| Other | 0.69 | (0.39-1.24) | 0.90 | (0.44-1.83) | 1.12 | (0.43-2.91) | 0.61 | (0.36-1.05) |
| Male | Ref. | Ref. | Ref | Ref. | ||||
| Female | 1.45 | (1.06-1.99) | 1.10 | (0.75-1.60) | 1.47 | (0.98-2.19) | 0.88 | (0.66-1.16) |
| Age 65 - 74 | Ref. | Ref. | Ref. | Ref. | ||||
| Age 75 - 84 | 1.01 | (0.70-1.46) | 1.00 | (0.75-1.34) | 1.42 | (0.98-2.06) | 0.71 | (0.51-0.99) |
| Age 85+ | 0.61 | (0.39-0.94) | 1.38 | (0.85-2.25) | 2.61 | (1.50-4.56) | 0.39 | (0.25-0.61) |
| Self-respondent | Ref. | Ref. | Ref | Ref. | ||||
| Proxy respondent | 0.83 | (0.45-1.53) | 0.70 | (0.27-1.78) | 1.07 | (0.41-2.80) | 0.52 | (0.29-0.92) |
| < High school | 0.43 | (0.25-0.73) | 1.52 | (0.77-2.99) | 1.06 | (0.57-2.01) | 0.77 | (0.47-1.26) |
| High school Graduate | 0.63 | (0.44-0.90) | 0.83 | (0.56-1.24) | 0.82 | (0.53-1.29) | 0.72 | (0.49-1.05) |
| > High school | Ref. | Ref. | Ref. | Ref. | ||||
| Northeast | 1.40 | (0.88-2.24) | 1.01 | (0.47-2.18) | 1.61 | (0.73-3.55) | 1.05 | (0.67-1.65) |
| Midwest | Ref. | Ref. | Ref. | Ref. | ||||
| South | 1.35 | (0.87-2.11) | 0.69 | (0.41-1.16) | 0.98 | (0.53-1.80) | 1.28 | (0.87-1.86) |
| West | 0.98 | (0.64-1.50) | 0.93 | (0.50-1.72) | 0.87 | (0.44-1.72) | 1.19 | (0.76-1.87) |
| Months since rehab use | 0.83 | (0.79-0.88) | - | - | - | |||
| < 1 month rehab use | - | - | Ref. | - | ||||
| 1-3 months rehab use | - | - | 2.13 | (1.21-3.76) | - | |||
| 4+ Months | - | - | 2.02 | (1.08-3.78) | - | |||
|
| ||||||||
| (N) | 1056 | 3172 | 935 | 3293 | ||||
Odds ratios shown in bold are statistically significant at p<.05; Ref.=Reference group
Finally, estimates of rehabilitation service use by place type from survey-based responses are similar to (only 1-2 percentage points lower than) claims-based measures (Table 3). However, sensitivity was substantially lower for inpatient rehabilitation than for other locations (55.5% for inpatient). Specificity and NPV were higher for site-specific estimates, but PPV, like sensitivity, was lower, ranging from 69.4 for inpatient to 75.2 for home-based services.
Table 3.
Percentage Using Rehabilitation in the Last Year by Source and Sensitivity, Specificity, Positive Predictive Value and Negative Predictive Value of Survey-based Measure, by Location of Service (95% CI)
| %-Survey | %-Claims | Sensitivity | Specificity | PPV | NPV | |
|---|---|---|---|---|---|---|
| Any Place | 21.8 (20.0-23.5) | 22.6 (20.6-24.5) | 76.9 (74.5-79.3) | 94.3 (93.3-95.3) | 79.7 (76.4-83.1) | 93.3 (92.4-94.2) |
| Inpatient | 7.0 (6.1-7.9) | 8.7 (7.7-9.8) | 55.5 (50.0-61.0) | 97.7 (97.0-98.3) | 69.4 (63.3-75.5) | 95.8 (95.0-96.6) |
| Outpatient | 14.5 (13.0-16.0) | 15.1 (13.4-16.8) | 69.1 (65.1-73.1) | 95.2 (94.3-96.1) | 71.9 (67.5-76.3) | 94.5 (93.5-95.5) |
| Home | 8.1 (7.1-9.1) | 9.1 (7.9-10.3) | 67.1 (62.1-72.0) | 97.8 (97.3-98.3) | 75.2 (69.6-80.8) | 96.7 (96.1-97.3) |
| All (N) | 4228 | 4228 | 1056 | 3172 | 935 | 3293 |
| Inpatient, Outpatient, Home | 4228, 4228, 4228 | 4228, 4228, 4228 | 444, 622, 508 | 3784, 3606, 3720 | 307, 553, 410 | 3921, 3675, 3818 |
DISCUSSION
This study found that self-reports by older adults and Medicare claims yielded similar estimates of rehabilitation use over the last year. The self-reported measures have high sensitivity (77%) and PPV (80%) and even higher specificity and NPV (approaching 95%). Moreover, agreement was high, and Kappa was substantial and higher than previously reported for outcomes such as office visits.8
We also found that sensitivity declined as the time elapsed since use increased. Previous studies have suggested that reference period matters for other types of care,11 but this is the first study we know of to demonstrate a steady decay in sensitivity for rehabilitation use over a one-year look back period. Although sensitivity is adequate for the annual measurement period, longer look back periods may not provide acceptable accuracy.
There are also differences in reporting errors by race, for the very old, and by education that researchers should be aware of if they are interested in documenting subgroup differences in rehabilitation use. In particular, in fully controlled models, Black respondents and those with less than a high school education who have experienced rehabilitation according to Medicare claims are less likely than other groups to answer that they have used rehabilitation in the last year. Why there are different patterns of errors by race is not altogether clear from our analysis, although we did not find support for age, education levels or proxy status as possible reasons. Two other possible sources of differences in sensitivity that we did not explore but may be fruitful for future research are differences in intensity of rehabilitation and geographic-based differences in utilization beyond those captured by regional distinctions.
With respect to setting, inpatient rehabilitation appears to be under-reported relative to other places. In this study inpatient rehabilitation encompasses a mixture of settings including hospital, skilled nursing homes, and inpatient rehabilitation facilities. It may be that in hospital settings in particular, older adults may have more difficulty sorting out different types of services they receive. In contrast, sensitivity is higher for rehabilitation received in outpatient settings and at home, perhaps because going to an appointment or having a therapist in one’s home (as opposed to receiving services while an inpatient) is more salient.
Limitations
Several limitations of this study are worth noting. The generalizability of findings is limited to Medicare beneficiaries ages 65 and older who enroll in fee-for-service (both Parts A and B) Medicare for a full 12 months prior to their interview. We found differences with respect to age, racial composition and education between this group and those excluded because they were enrolled in Medicare Advantage or in only Part A. Prior studies also suggest that those enrolled in fee for service tend to have higher utilization than those in managed care plans.19 Despite these differences, we have no basis for speculating that the extent of reporting inconsistencies differs between those enrolled continuously in fee-for-service Medicare and the overall Medicare population. In addition, we did not have information on whether participants were covered by an employer-provided plan or were private pay patients. In these cases, an individual might receive and therefore report rehabilitation services but since Medicare is a secondary payer, Medicare claims would be absent (leading to a downward bias in PPV). By restricting the sample to persons with both Parts A and B, we decreased the likelihood of including persons with these payment sources. Moreover, the age patterns for PPV were reassuring; that is, PPV was lowest for those ages 65-74 and increased significantly with age (as the chances of having private primary insurance decrease). Finally, we did not attempt to distinguish type and/or number of rehabilitation disciplines nor did we distinguish type of inpatient setting. With respect to the latter, we suspect that individuals in inpatient rehabilitation facilities may be more likely than those in hospital settings or nursing homes to accurately report rehabilitation services, but the number of Medicare beneficiaries who use such facilities is relatively small.20
CONCLUSIONS
This study suggests that older adults as a group can report whether they used rehabilitation services in the last year with acceptable accuracy. At the same time, researchers should be cautious in drawing conclusions about rehabilitation use with self-reported data for some subgroups. Asking valid questions about rehabilitation use on a national panel survey of older adults’ functioning substantially expands researchers’ ability to study the basic epidemiology of late-life rehabilitation use in the U.S. Instead of focusing on a single condition within a single setting or examining post-acute rehabilitation following hospitalization, researchers may now explore on a national level the role of rehabilitation in late-life disability onset, recovery, and trajectories as well as disability trends.
Highlights.
Among older adults in the U.S., survey-based measures accurately capture rehabilitation service use over the past year.
Under-reporting errors were higher for Black enrollees, the very old, and those with lower education levels.
Agreement between survey responses and claims decreases with months since use and increases with duration of use up to 3-4 weeks.
Acknowledgments
This research was funded by the National Institutes of Health U01-AG-032947. We thank John Mulcahy for his expert programming assistance.
Abbreviations
- NHATS
National Health and Aging Trends Study
- PPV
Positive Predictive Value
- NPV
Negative Predictive Value
- HCPCS
Healthcare Common Procedure Coding System
- OR
Odds Ratio
- CI
Confidence Interval
Footnotes
Disclosures: The authors are National Health and Aging Trends Study investigators.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Vicki A. Freedman, Institute for Social Research, University of Michigan
Judith D Kasper, Department of Health Policy and Management, Johns Hopkins University Bloomberg School of Public Health
Alan Jette, Center for Interprofessional Studies & Innovation, MGH Institute of Health Professions
References
- 1.Granger CV, Markello SJ, Graham JE, et al. The uniform data system for medical rehabilitation: report of patients with lower limb joint replacement discharged from rehabilitation programs in 2000-2007. American Journal of Physical Medicine and Rehabilitation. 2010;9(10):781–794. doi: 10.1097/PHM.0b013e3181f1c83a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Granger CV, Reistetter TA, Graham JE, et al. The Uniform Data System for Medical Rehabilitation: report of patients with hip fracture discharged from comprehensive medical programs in 2000-2007. American Journal of Physical Medicine and Rehabilitation. 2011;90(3):177–189. doi: 10.1097/PHM.0b013e31820b18d7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ottenbacher K, Graham JE. The state of the science: access to postacute care rehabilitation services. Archives of Physical Medicine and Rehabilitation. 2007;88:1513–1521. doi: 10.1016/j.apmr.2007.06.761. [DOI] [PubMed] [Google Scholar]
- 4.Kasper JD, Freedman VA. Findings from the 1st Round of the National Health and Aging Trends Study (NHATS): Introduction to the Special Issue. Journals of Gerontology Series B: Psychological Sciences and Social Sciences. 2014;69(Suppl 1):S1–S7. doi: 10.1093/geronb/gbu125. [DOI] [PubMed] [Google Scholar]
- 5.Freedman VA, Kasper JD, Cornman JC, et al. Validation of new measures of disability and functioning in the National Health and Aging Trends Study. Journals of Gerontology Series A: Biological and Medical Sciences. 2011;66(9):1013–1021. doi: 10.1093/gerona/glr087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Andersen R, Kasper J, Frankel MR, et al. Total Survey Error. San Francisco, CA: Jossey-Bass; 1979. [Google Scholar]
- 7.Leggett LE, Khadaroo RG, Holroyd-Leduc J, et al. Measuring Resource Utilization: A Systematic Review of Validated Self-Reported Questionnaires. Medicine (Baltimore) 2016;95(10):e2759. doi: 10.1097/MD.0000000000002759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Zuvekas SH, Olin GL. Validating Household Reports of Health Care Use in the Medical Expenditure Panel Survey. Health Services Research. 2009;44(5 Pt 1):1679–1700. doi: 10.1111/j.1475-6773.2009.00995.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Petrou S, Murray L, Cooper P, Davidson LL. The accuracy of self-reported healthcare resource utilization in health economic studies. International Journal of Technology Assessment in Health Care. 2002;18(3):705–710. doi: 10.1017/s026646230200051x. [DOI] [PubMed] [Google Scholar]
- 10.Lubeck DP, Hubert HB. Self-report was a viable method for obtaining health care utilization data in community-dwelling seniors. Journal of Clinical Epidemiology. 2005;58(3):286–290. doi: 10.1016/j.jclinepi.2004.06.011. [DOI] [PubMed] [Google Scholar]
- 11.Roberts RO, Bergstralh EJ, Schmidt L, et al. Comparison of self-reported and medical record health care utilization measures. Journal of Clinical Epidemiology. 1996;49(9):989–995. doi: 10.1016/0895-4356(96)00143-6. [DOI] [PubMed] [Google Scholar]
- 12.Gell NM, Mroz TM, Patel KV. Rehabilitation Services Use and Patient-Reported Outcomes Among Older Adults in the United States. Archives of Physical Medicine and Rehabilitation. 2017:30216–2. doi: 10.1016/j.apmr.2017.02.027. Epub ahead of print. [DOI] [PubMed] [Google Scholar]
- 13.Cubanski J, Swoope C, Boccuti C, Jacobson G, Casillas G, Griffin S, Neuman T. A Primer on Medicare: Key facts about the Medicare program and the People it covers. [November 24, 2017];Kaiser Family Foundation Report. 2015 http://files.kff.org/attachment/report-a-primer-on-medicare-key-facts-about-the-medicare-program-and-the-people-it-covers.
- 14.DeMatteis J, Freedman VA, Kasper JD. NHATS Technical Paper #14. Baltimore: Johns Hopkins University School of Public Health; 2016. [June 11, 2017]. National Health and Aging Trends Study Development of Round 5 Survey Weights. Available at www.NHATS.org. [Google Scholar]
- 15.Altman DG, Bland JM. Diagnostic tests 1: sensitivity and specificity. British Medical Journal. 1994;308:1552. doi: 10.1136/bmj.308.6943.1552. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Altman DG, Bland JM. Diagnostic tests 2: predictive values. British Medical Journal. 1994;303:102. doi: 10.1136/bmj.309.6947.102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
- 18.Hosmer DW, Lemeshow S. Applied Logistic Regression. New York: Wiley; 2013. [Google Scholar]
- 19.Landon BE, Zaslavsky AM, Saunders RC, et al. Analysis Of Medicare Advantage HMOs compared with traditional Medicare shows lower use of many services during 2003-09. Health Affairs. 2012;31(12):2609–2617. doi: 10.1377/hlthaff.2012.0179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.MedPAC. Report to Congress: Medicare Payment Policy. Chapter 10. [June 26, 2017];Inpatient rehabilitation facility services. 2017 Available at http://www.medpac.gov/-documents-/reports.