Impact of Nonresponse on Medicare Current Beneficiary Survey Estimates

Abstract

The Medicare Current Beneficiary Survey (MCBS) has been used by policymakers and research analysts to provide information on a wide array of topics about the Medicare Program. Nonresponse bias is potentially one of the most important threats to the validity of the estimates from the MCBS. In this article we present results of our methodological study that analyzes the impact of nonresponse on MCBS estimates, including initial round unit nonresponse, panel attrition, and item nonresponse. Our findings indicate that for most of the measures studied, the bias caused by differences between nonrespondents and respondents in the MCBS was substantially reduced or eliminated by the nonresponse procedures currently employed.

Introduction

The MCBS is a continuous, multipurpose survey of a representative national sample of the Medicare population, conducted by CMS. The central goals of the MCBS are to determine expenditures and sources of payment for all services used by Medicare beneficiaries, including copayments, deductibles, and non-covered services; to ascertain all types of health insurance coverage and relate coverage to sources of payment; and to trace processes over time, such as changes in health status, spending down to Medicaid eligibility, and the impacts of program changes. The MCBS is the most important survey of Medicare beneficiaries, and has been used by policymakers and research analysts to provide information on a wide array of topics about the Medicare Program (Kautter and Pope, 2004).

The MCBS operates as a rotating panel survey. New panels are selected from the population of beneficiaries eligible for Medicare as of January 1 of the year of induction.¹ Initial interviews for each new supplement are conducted in the fall interview round. MCBS beneficiaries are interviewed three times per year, and each interview round is administered over a 4-month period. Beneficiaries remain in the sample for 4 years; each fall approximately one-fourth of the sample is replaced.²

Because of its key role in informing Medicare policymakers, obtaining accurate estimates from the MCBS is of critical importance. Like virtually all surveys, the MCBS is subject to several forms of nonresponse. These include unit nonresponse, in which the sampled beneficiary is not interviewed, and item nonresponse, in which the interviewed beneficiary does not answer a certain question. In addition, in longitudinal surveys like the MCBS, there is the potential for beneficiaries to answer one or more rounds of the survey and stop participating (panel attrition). Consequences of nonresponse include: (1) biases in point estimators, (2) inflation of the variances of point estimators, and (3) biases in customary estimators of precision (Dillman et al., 2002). Nonresponse bias is potentially one of the most important threats to the validity of the estimates from surveys like the MCBS.³

In this article we present results of a methodological study initiated by CMS that analyzed the impact of nonresponse on MCBS estimates (Kautter et al., 2003). We present results of our analyses of initial round unit nonresponse, panel attrition, and item nonreponse. After providing a definition of nonresponse bias and its components, we provide a brief overview of the MCBS survey design. Then we explain our methodology for analyzing initial round unit nonresponse in the MCBS and our analytic findings for initial round unit nonresponse, and similarly, for panel attrition and item nonresponse. Finally, we offer conclusions.

Definition of Nonresponse Bias and its Components

The concern about survey nonresponse is that nonrespondents will differ from respondents with regard to the survey variables, in which case the survey estimates based on the respondents alone will be biased estimates of the overall population parameters (Kalton, 1983). If there is no difference between respondents and nonrespondents regarding survey variables, then there is no potential for bias due to nonresponse.

To illustrate this point, suppose the aim of a survey is to estimate a population mean. In the case of a survey that fails to collect data for the nonrespondents, the sample statistic used to estimate the population mean is the respondent sample mean. The bias arising from using the respondent sample mean as an estimator for the population mean is:

$$\mathrm{\beta }={\mathrm{\mu }}_{\text{R}}-\mathrm{\mu }$$

where (β is the bias, μ_R is the population mean for respondents, and μ is the overall population mean. This expression shows that the nonresponse bias is a function of the difference in the population mean for respondents and the overall population mean.

The nonresponse bias can also be expressed in terms of the response rate and the difference in the respondent and nonrespondent means:

$$\mathrm{\beta }=(1-\text{RR})({\mathrm{\mu }}_{\text{R}}-{\mathrm{\mu }}_{\text{NR}})$$

where (β is the bias, RR is the response rate in the population, and μ_R and μ_NR are the population means for respondents and nonrespondents (Kalton, 1983). This last expression shows that the nonresponse bias is a function of two quantities: (1) the response rate in the population; and (2) the difference in the population means for respondents and nonrespondents. Note that if there is a 100-percent response rate (RR = 1), or if there is no difference between the respondent and nonrespondent means (μ_R − μ_NR = 0), then the nonresponse bias is zero. For a given difference in the respondent and nonrespondent means, the nonresponse bias falls as the response rate increases, and for a given response rate, the nonresponse bias falls as the difference in the respondent and nonrespondent means falls.

MCBS Survey Design

A basic understanding of the MCBS survey design is important for understanding our methodology for analyzing nonresponse. We provide an overview of the MCBS sample design and survey weighting procedures.

Sample Design

The sample design for the MCBS is a stratified area probability design with three stages of selection. The induction sample is based on a multistage sample using clusters of counties (primary sampling units [PSUs]) at the first stage, and ZIP Code clusters at the second stage; this clustering helps control data collection costs because face-to-face personal interviews are conducted to collect the data. The final stage of sampling is at the level of beneficiaries in the 5 percent sample with addresses in the selected ZIP-Code clusters. At the final stage of selection, the beneficiary sample is stratified within seven age categories. The target sample size for the continuing annual sample is 12,000 responding beneficiaries. Beneficiaries eligible for Medicare by disability (under age 65), as well as the oldest old (85 or over), are oversampled (Apodaca, Judkins, and Lo, 1992). Thus, when analyzing the MCBS, weights must be employed for each respondent to account for the differential sampling probabilities.

Survey Weights

Like many complex surveys, the MCBS uses survey weights to account for differential probabilities of selection and to adjust for nonresponse (Judkins and Lo, 1993). In this section we describe the steps used to create the survey weights for a panel of beneficiaries in the MCBS (Westat, Inc., 2001).

Base Weights

For a panel in their initial round of the MCBS, to account for differential probabilities of selection, base weights are computed from their inverse of probability of selection.

Poststratification Weights

After base weights are created for a panel in their initial round of the MCBS, poststratification adjustments are applied to ensure consistency between the characteristics of sampled beneficiaries, properly weighted, and the national Medicare population.

Nonresponse Adjusted Weights

The post-stratified weights for a panel in their initial round of the MCBS are then adjusted for nonresponse at the initial round. Potential predictors of initial round unit response include Medicare and Medicaid entitlement status, Medicare managed care enrollment, medical reimbursements, physicians' fee ratios and practice cost indices, and demographic, socioeconomic, and geographic variables. Cells for adjusting weights for nonresponse are based either on chi-square tests of association or models of response propensity. The resulting weights are the panel's initial round nonresponse weights.⁴ For the panel's second survey year, nonresponse weights are created by adjusting the initial round nonresponse weights to account for conditional nonresponse in the panel's second year of the survey. The conditional response rate in year two is defined as respondents divided by eligibles, where eligibles are restricted to initial round respondents who are alive on January 1 of year two. Similar procedures are used to derive nonresponse weights for the panel's third and fourth years of the survey. In addition to administrative data, survey data provided in prior survey years can be used to adjust for panel attrition. Candidate variables for panel attrition adjustment include health, functioning, demographic, geographic, utilization, and interview variables.

Initial Round Unit Nonresponse

Methodology

Our sample for analyzing initial round unit nonresponse is confined to MCBS eligibles (respondents and nonrespondents) in their initial round of the survey. For this study, incoming eligibles are pooled across 3 years of MCBS data (1997-1999) to maximize sample size for our analyses. Medicare administrative records, including claims, provide a unique opportunity to analyze the impact of nonresponse on MCBS estimates, since they provide data on respondents and nonrespondents alike. Claims for services received by persons enrolled in managed care are not available from these records. The data for beneficiaries in long-term care facilities are typically provided by facility staff rather than the beneficiary, and response rates are close to 100 percent. Beneficiaries who are eligible for Medicare by end stage renal disease (ESRD) are a small unique subpopulation. For these reasons, the analysis sample for studying potential bias was limited to community-based, non-ESRD beneficiaries enrolled in traditional Medicare fee-for-service (FFS). Our 3-year merged analytic file for analyzing MCBS initial round unit nonresponse has a sample size of 14,315.

Proxy measures, defined as variables known for both respondents and nonrespondents that serve as proxies for studying the effects of nonresponse, were identified and used to compare respondents and nonrespondents. Postratification survey weights for the selected sample were obtained for this exercise; the weights incorporated a poststratification adjustment to align the selected sample with the complete frame, but no adjustments for nonresponse. For the initial round, proxy measures were based primarily on administrative record data.

The proxy measures used in comparing respondents and nonrespondents were of two types based on the time that they become available. The first type included demographic measures that are available from the sampling frame and could be (or were) utilized in the weight adjustment process. The second type were CMS administrative record data that only became available for both respondents and nonrespondents some time after the survey had been completed and may not have been available for application in the weight adjustment process for the current round. This second type included selected diagnoses, counts of services received by type, expenditures for health care, and hierarchical condition categories (HCC) diagnostic cost groups (DCG) risk scores, or HCC-DCG risk scores, which were developed for risk adjustment of Medicare managed care capitation payments (Pope et al., 2004).

The HCC-DCG risk score is an expenditure-weighted index of a beneficiary's diagnoses that predicts the relative risk of future Medicare expenditures. A beneficiary's HCC-DCG risk score is calculated by dividing the beneficiary's predicted expenditures by per capita expenditures for the entire Medicare FFS population. An HCC-DCG risk score above 1.0 indicates that a beneficiary is predicted to have greater future medical expenditures than the average Medicare FFS beneficiary (i.e., is sicker than average), whereas an HCC-DCG risk score below 1.0 indicates the beneficiary is predicted to have lower than average future health care costs, i.e., is healthier than average. In short, the HCC-DCG risk score is a summary index of a beneficiary's diagnostic disease profile or burden, incorporating both numbers and severity of serious disorders. Multiple diseases are aggregated into a single index score using the metric of their impact on future medical expenditures.

To measure nonresponse bias, estimates were compared based on the respondent sample and the eligible sample (respondents and nonrespondents). Comparisons were first based on the poststratification weights (before nonresponse adjustment) and then recomputed using the nonresponse adjusted weights (after nonresponse adjustment) for the respondent data and the poststratification weight for the eligible sample. The evaluation of the statistical significance of bias estimates was performed individually on a large number of estimates. No corrections for multiple comparisons were applied, because the real interest was in the individual comparisons. But if one wished to assess the overall impact on bias, a few statistically significant results among a large number of measures would be likely by chance even if the overall impact on bias was low or negligible.

We did not attempt to directly evaluate nonresponse bias for variables available for survey respondents only. For these variables, we cannot compare survey respondents to nonrespondents or to eligibles. However, to the extent that survey-only and administrative variables are correlated, it is reasonable to infer that the degree of bias in administrative and survey variables is related. That is, a large bias for administrative variables implies the potential for a large bias among survey variables. Conversely, if little bias is observed among administrative variables, our confidence of lack of significant bias among survey variables is increased.

Findings

Table 1 presents response rates overall and by selective demographic and eligibility characteristics. The overall 1997-1999 initial round MCBS response rate for our analysis sample is 82.6 percent. This response rate is roughly comparable to what is expected for large national health surveys administered in person (Aday, 1996). Response rates by subcategory are relatively consistent without many large variations among the groups, the largest difference in response rates being the lower response in metropolitan versus non-metropolitan areas. While the variations in response rates are relatively small, vulnerable groups associated with poorer health status respond at an equal or slightly higher rate to the MCBS. For example, Medicaid enrollees have a response rate of 85.7 percent compared with 82.0 percent for those without Medicaid. Similarly, the sickest beneficiaries with the highest HCC-DCG risk scores have a higher response rate than the healthiest beneficiaries with low scores (85.6 and 79.1 percent, respectively).

Table 1. MCBS Initial Round Unit Response Rates by Demographic, Eligibility, and Health Status Characteristics¹.

Characteristic	Eligibles	Respondents	Response Rate	Statistical Significance2
	N	N	Percent
All Sample	14,315	11,817	82.6	—
Age
Under 65 Years	2,609	2,174	83.3	—
65-74 Years	5,391	4,442	82.4	—
75-84 Years	4,664	3,837	82.3	—
85 Years or Over	1,651	1,364	82.6	—
Sex				***
Male	6,280	5,268	83.9	—
Female	8,035	6,549	81.5	—
Race				**
White	12,079	9,938	82.3	—
Black	1,463	1,244	85.0	—
Other	773	635	82.2	—
Original Reason for Medicare Entitlement				**
Aged	13,477	11,096	82.3	—
Disabled	831	715	86.0	—
Medicaid Status				***
No Medicaid	12,127	9,941	82.0	—
Medicaid	2,188	1,876	85.7	—
Current Reason for Medicare Entitlement				*
Aged	11,706	9,643	82.4	—
Disabled	2,609	2,174	83.3	—
Metropolitan Area Status				***
Non-Metropolitan	4,114	3,675	89.3	—
Metropolitan	10,201	8,142	79.8	—
Census Regions				**
North East	2,917	2,336	80.1	—
North Central	3,565	2,906	81.5	—
South	5,385	4,540	84.3	—
West	2,216	1,833	82.7	—
Others3	232	202	87.1	—
HCC-DCG Risk Score Quintiles4				**
0-20% (Lowest Score)	2,892	2,288	79.1	—
20-40%	2,840	2,286	80.5	—
40-60%	2,864	2,356	82.3	—
60-80%	2,858	2,439	85.3	—
80-100% (Highest Score)	2,861	2,448	85.6	—
Mortality
Died in the Year Following Initial Round	890	739	83.0	—
Survived the Year Following Initial Round	13,425	11,078	82.5	—

^*p<0.1.

^**p<0.05.

^***p<0.01.

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded.

²Statistical significance testing for distribution.

³Other includes Puerto Rico and other territories.

⁴Diagnosis-based health status index computed from provider bills (claims). A higher hierarchical condition categories-diagnostic cost group (HCC-DCG) score indicates poorer health.

NOTES: MCBS is Medicare Current Beneficiary Survey. Data unweighted.

SOURCE: RTI analysis of the 1997-1999 MCBS.

Consistently, multiple logistic regression analysis of response (Table 2) showed that males, Medicaid enrollees, Black persons, southerners, non-metropolitan residents, younger beneficiaries, and those in poorer health (upper quintiles of the HCC-DCG risk score) were more likely to respond to the initial round of the MCBS. The higher response rate of many sicker, more vulnerable groups is surprising, and is contrary to findings from recent nonresponse analyses of other major Medicare surveys, such as the Health Outcomes Survey (Khatutsky et al., 2002). We speculate that these differences may arise from the different modes of administration of the surveys, inperson for the MCBS versus mail with telephone follow up for the Health Outcomes Survey.

Table 2. Logistic Regression Model Estimating Likelihood of MCBS Initial Round Unit Response¹.

Characteristic	Estimate	Standard Error	Odds Ratio	Statistical Significance
Age
Under 65 Years	0.00	0.07	1.00	—
65-74 Years (Omitted)	—	—	—	—
75-84 Years	-0.15	0.06	0.86	***
85 Years or Over	-0.24	0.09	0.79	***
Sex
Male (Omitted)	—	—	—	—
Female	-0.15	0.05	0.86	***
Medicaid
Non-Enrolled (Omitted)	—	—	—	—
Enrolled	0.14	0.08	1.15	*
Race
White (Omitted)	—	—	—	—
Other	-0.06	0.10	0.94	—
Black	0.21	0.09	1.23	**
Current Reason for Medicare Entitlement
Originally Entitled to Medicare by Age (Omitted)	—	—	—	—
Originally Entitled to Medicare by Disability	-0.02	0.10	0.98	—
Census Regions
North East (Omitted)	—	—	—
North Central	-0.02	0.06	0.98	—
South	0.11	0.06	1.12	*
West	0.07	0.07	1.07	—
Other	0.69	0.21	1.99	***
Metropolitan Area Status
Non-Metropolitan (Omitted)	—	—	—	—
Metropolitan	-0.75	0.06	0.47	***
HCC-DCG Risk Score Quintiles2
Up to 20% (Omitted)	—	—	—	—
20-40%	0.03	0.08	1.04	—
40-60%	0.26	0.07	1.30	***
60-80%	0.48	0.07	1.61	***
80-100%	0.53	0.08	1.70	***
Panel
1997(Omitted)	—	—	—	—
1998	0.01	0.05	1.01	—
1999	0.08	0.06	1.08	—

^*p<0.1.

^**p<0.05.

^***p<0.01.

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded.

²Diagnosis-based health status index computed from provider bills (claims). A higher hierarchical condition categories-diagnostic cost group (HCC-DCG) score indicates poorer health.

NOTES: MCBS is Medicare Current Beneficiary Survey. Data weighted by poststratification weight adjusted for sampling design, but not for nonresponse. N=14,308.

Table 3 compares MCBS initial round eligibles, respondents, and nonrespondents by demographic, enrollment, and health status characteristics. Nonrespondents are further decomposed into refusals and other nonrespondents, where other nonrespondents represent Medicare beneficiaries who are unlocatable, physically or mentally incompetent without available proxy, out of area, etc. Initial round MCBS nonrespondents are significantly healthier than respondents, by 18 percent in terms of lower current Medicare expenditures ($3,526 on average for nonrespondents versus $4,309 for respondents) and by 11 percent in terms of the HCC-DCG risk score (0.86 on average for nonrespondents versus 0.97 on average for respondents). However, as is the case with sociodemographic characteristics, as a result of the high initial round response rate, the differences between eligibles and respondents⁵ with respect to health status is much less pronounced than the differences between nonrespondents and respondents (e.g., 3 percent difference in current Medicare expenditures—$4,172 for eligibles versus $4,309 for respondents—versus the 18 percent difference between nonrespondents and respondents).

Table 3. Comparison of MCBS Initial Round Eligibles, Respondents, and Nonrespondents, by Selected Characteristics¹.

Characteristic	Eligibles	Respondents	All	Nonrespondents Refusals	Others	Statistical Significance3
All Sample	N=14,315	N=11,817	N=2,498	N=1,852	N=646
	Percent
Age
Under 65 Years	13.6	13.9	12.4	8.2	26.2	—	**
65-74 Years	44.0	43.9	44.2	46.5	36.7	—	**
75-84 Years	32.6	32.4	33.5	36.0	25.2	—	**
85 Years or Over	9.9	9.8	9.9	9.3	11.9	—	—
Sex
Male	43.4	44.2	39.8	37.6	47.2	*	**
Female	56.6	55.8	60.2	62.4	52.8	*	**
Race
White	86.0	85.6	87.5	91.7	73.5	*	**
Black	8.8	9.1	7.2	5.3	13.6	*	**
Other	5.3	5.3	5.3	3.0	12.9	—	**
Original Reason for Entitlement
Aged	94.0	93.7	95.1	96.1	91.8	*	**
Disabled	6.0	6.3	5.0	3.9	8.3	*	**
Medicaid Status
No Medicaid	87.2	86.6	89.8	94.3	75.0	*	**
Medicaid	12.8	13.4	10.2	5.7	25.0	*	**
Current Reason for Entitlement
Aged	86.4	86.2	87.6	91.8	73.8	—	**
Disabled	13.6	13.9	12.4	8.2	26.2	—	**
Metropolitan Area Status
Non-Metropolitan	27.5	29.8	16.7	15.8	19.9	*	**
Metropolitan	72.5	70.2	83.3	84.2	80.1	*	**
Census Regions
North East	20.3	19.7	23.1	23.4	22.2	*	—
North Central	25.5	25.2	26.8	29.1	19.3	—	**
South	37.5	38.4	33.6	32.8	36.3	*	—
West	15.0	14.9	15.3	14.5	18.1	—	**
Other4	1.6	1.8	1.2	0.2	4.2	*	**
Mortality
Died in the Year Following Initial Round	6.0	6.0	6.0	4.9	9.5	—	**
Survived the Year Following Initial Round	94.0	94.0	94.1	95.2	90.5	—	**
Mean HCC-DCG Score5	0.95	0.97	0.86	0.80	1.05	* —	** —
Total Medicare Expenditures (Dollars)	4,172	4,309	3,526	2,935	5,458	*	**
% Users	88.5	89.8	85.0	87.5	77.0	*	**
Expenditures for Inpatient Services (Dollars)	2,025	2,092	1,708	1,371	2,811	*	**
% Users	17.6	18.3	14.5	13.2	18.8	*	**
Expenditures for Part B Services (Dollars)	1,252	1,285	1,093	1,019	1,336	*	**
% Users	87.9	88.7	84.3	87.1	75.5	*	**

^*Statistically significant difference between respondents and all nonrespondents (p<0.05).

^**Statistically significant difference between refusals and other nonrespondents (p<0.05).

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded.

²Includes out of area, unlocatable, physically and mentally impaired without a proxy, and other types of nonrespondents.

³Statistical significance testing on eligible and respondent differences is equivalent to statistical significance testing on all nonrespondent and respondent differences.

⁴Other includes Puerto Rico and other territories.

⁵Diagnosis-based health status index computed from provider bills (claims). A higher hierarchical condition categories-diagnostic cost group (HCC-DCG) score indicates poorer health.

NOTES: MCBS is Medicare Current Beneficiary Survey. Data weighted by post-stratification weights not adjusted for nonresponse.

SOURCE: RTI analysis of the 1997-1999 MCBS.

Our analysis (Table 3) also demonstrates that nonrespondents are not a homogeneous group. Refusals, which represent approximately three-quarters of all nonrespondents, are substantially healthier than other nonrespondents, and account for the overall better health of nonrespondents relative to respondents.⁶

Overall, the results in Table 3 suggest that although statistically significant differences occur between nonrespondents and respondents on such demographic characteristics as sex, race, and geographic distribution, and although nonrespondents overall appear to be healthier than respondents, because of the high initial MCBS response rate, the magnitude of the differences between eligibles and respondents is relatively small, and thus unlikely to create a major potential for bias.

Table 4 analyzes the effect of existing MCBS unit nonresponse weighting adjustments on selected variables by comparing pre- and post- nonresponse adjustment estimates. The differences in eligible and respondent means (proportions), each adjusted by only poststratification weights are shown in this table. The differences are estimates of nonresponse bias before MCBS nonresponse adjustment weighting. Correspondingly, the table presents the differences in eligible and respondent means (proportions), each adjusted by not only poststratification weights, but by nonresponse weights as well. The differences are the estimates of nonresponse bias after MCBS nonresponse adjustment weighting.

Table 4. Effect of MCBS Initial Round Nonresponse Adjustment on Selected Characteristics¹.

Characteristic	Respondents Not Adjusted for Nonresponse				Respondents Adjusted for Nonresponse
	Weight		Difference (R-E)	Statistical Significance	Weight		Difference (R-E)	Statistical Significance
	Poststratification Eligibles (E)	Poststratification Respondents (R)			Poststratification Eligibles (E)	MCBS Initial Nonresponse Respondents (R)
Age (%)
Under 65 Years	13.6	13.9	0.25	*	13.6	13.8	0.17	—
65-74 Years	44.0	43.9	-0.05	—	44.0	44.2	0.18	—
75-84 Years	32.6	32.4	-0.19	—	32.6	32.3	-0.29	—
85 Years or Over	9.9	9.8	-0.01	—	9.9	9.8	-0.06	—
Female (%)	56.6	55.8	-0.76	***	56.6	56.4	-0.19	—
White (%)	86.0	85.6	-0.31	**	86.0	85.7	-0.28	—
Enrolled in Medicaid (%)	12.8	13.4	0.55	***	12.8	13.2	0.38	***
Metropolitan Area Status	72.5	70.2	-2.27	***	72.5	72.0	-0.51	—
Original Reason for Medicare Entitlement: Disability (%)	6.0	6.3	0.23	**	6.0	6.2	0.12	—
Mean HCC-DCG Score2	0.95	0.97	0.02	***	0.95	0.96	0.01	**
Total Medicare Expenditures (Dollars)	4,172	4,309	137	***	4,172	4,257	85	*
% Users	88.5	89.8	1.33	***	89.0	89.4	0.46	—
Expenditure for Inpatient Services (Dollars)	2,025	2,092	67	**	2,025	2,072	47	—
% Users	17.6	18.3	0.66	***	17.6	17.9	0.34	**
Expenditures for Part B Services (Dollars)	1,252	1,285	34	***	1,252	1,273	21	—
% Users	87.9	88.7	0.76	***	87.9	88.3	0.38	—

^*p<0.1.

^**p<0.05.

^***p<0.01.

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded.

²Diagnosis-based health status index computed from provider bills (claims). A higher hierarchical condition categories-diagnostic cost group (HCC-DCG) score indicates poorer health.

NOTES: MCBS is Medicare Current Beneficiary Survey.

SOURCE: RTI analysis of the 1997-1999 MCBS.

As shown in Table 4, current MCBS nonresponse adjustments align the distribution of respondents across sociodemographic characteristics to be far more consistent with the eligible sample. All comparisons of respondents with eligibles are statistically significant before nonresponse adjustment. Only four comparisons (enrollment in Medicaid, health status, total Medicare expenditures, and percent utilizing inpatient services) remain statistically significant after nonresponse adjustment and, even in those cases, the magnitude of the bias measure is reduced. Although initial round nonresponse bias is small and is further reduced by MCBS nonresponse weights, it is not entirely eliminated. For example, after nonresponse weights are applied, the estimated bias in mean Medicare total expenditures falls from $137 (3.3 percent) to $85 (2.0 percent),⁷ for a 38 percent reduction in the estimated bias.

Panel Attrition

Methodology

Longitudinal surveys include any type of survey for which at least some of the units are measured more than once. These include traditional panel surveys, with fixed or rotating panels, retrospective longitudinal surveys, and cohort followups (Duncan and Kalton, 1987). Although the increased availability of longitudinal survey data has been one of the most important developments in applied social science research over the last few decades, the most potentially damaging threat to the value of longitudinal survey data is the presence of biasing attrition, i.e., attrition that is selectively related to outcome variables of interest (Fitzgerald, Gottschalk, and Moffitt, 1998).

In addition to analyzing initial round unit nonresponse, an analysis of panel attrition in the MCBS is performed. Two pairs of MCBS file years, i.e., 1997–1998 and 1998–1999, are pooled to construct three separate panel attrition analysis samples:

• Second Year Panel Attrition Analysis Sample—First year respondents eligible in the second year, used to analyze panel attrition between the first and second MCBS survey years.

• Third Year Panel Attrition Analysis Sample—Second year respondents eligible in the third year, used to analyze panel attrition between the second and third MCBS survey years.

• Fourth Year Panel Attrition Analysis Sample—Third year respondents eligible in the fourth year, used to analyze panel attrition between the third and fourth MCBS survey years.

Similar to our initial round unit nonresponse sample, our sample for analyzing panel attrition is restricted to beneficiaries residing in the community, enrolled in traditional FFS Medicare, and not eligible for Medicare by ESRD. The sample sizes for our second, third, and fourth year panel attritions are, respectively, 7,544, 6,345, and 5,437.

The conditional response rate for the second (third, fourth) year panel attrition analysis sample is defined as second (third, fourth) year attrition sample respondents divided by eligibles, where eligibles are restricted to first (second, third) year respondents who are alive on January 1 of the second (third, fourth) year. Conditional nonresponse bias can be decomposed into the conditional response rate and the difference in conditional population means for respondents and nonrespondents. For variables available for both respondents and nonrespondents, we first calculate conditional response rates for each panel attrition analysis sample, stratifying by sociodemographics. Then we estimate differences in respondent and nonrespondent conditional means and proportions for demographic, enrollment, health status, and service utilization measures. Note that unlike for the initial round unit nonresponse analysis, for the panel attrition analysis, in addition to administrative data, survey data provided in prior survey years could also be used as proxy measures, e.g., self-reported general health status, prescription drug expenditures, etc.

In addition to analyzing the components of conditional nonresponse bias, we analyze bias directly by comparing means (proportions) for respondents and eligibles. We estimate bias before and after applying MCBS adjustments for panel attrition. Each set of response analyses (i.e., second, third, and fourth year) was conditional on the response in the prior MCBS year. Consequently, the largest potential for bias occurred in the initial round, because the conditional response rates in subsequent years remained relatively high and increased by year as the more reluctant respondents were removed from the pool of persons sampled for each subsequent year.

The cumulative response rate for a panel in their second (third, fourth) MCBS year was approximated by calculating the product of their first year response rate and their conditional response rates through their second (third, fourth) survey year. Finally, in order to assess the cumulative impact of nonresponse over the 4 survey years, the cumulative nonresponse bias at each year was approximated by summing the estimated biases up to and including that year.

Findings

As shown in Table 5, the conditional response rates for our second, third, and fourth year panel attrition analysis samples are 88.9, 94.7, and 96.8 percent, respectively, yielding progressively declining attrition rates of 11.1, 5.3, and 3.2 percent. Declining attrition rates are common in longitudinal surveys like the MCBS (U.S. Bureau of the Census, 1998).

Table 5. MCBS Conditional Response Rates for Panel Attrition Analysis Samples, by Demographics, Eligibility, and Health Status Characteristics¹.

Characteristic	Second Year Attrition Sample	Statistical Significance2	Third Year Attrition Sample	Statistical Significance2	Fouth Year Attrition Sample	Statistical Significance2
	N=7,544		N=6,345		N=5,437
			Percent
All Sample	88.9	—	94.7	—	96.8	—
Age				**
Under 65 Years	89.0	—	93.1	—	95.8	—
65-74 Years	89.9	—	94.9	—	96.6	—
75-84 Years	88.1	—	94.9	—	97.5	—
85 Years or Over	87.5	—	96.1	—	97.1	—
Sex
Male	89.4	—	94.9	—	96.9	—
Female	88.5	—	94.6	—	96.7	—
Living Alone				*
Yes	88.3	—	94.4	—	96.7	—
No	89.2	—	95.5	—	97.1	—
Income Categories				*
Under $15,000	89.1	—	95.5	—	97.4	—
$15,001-$30,000	90.6	—	94.4	—	96.7	—
$30,001-$50,000	90.0	—	93.6	—	95.4	—
Over $50,000	88.3	—	94.5	—	96.5	—
Race
White	88.8	—	94.7	—	96.7	—
Black	89.7	—	95.0	—	96.4	—
Other	89.9	—	93.6	—	99.0	—
Medicaid Status		***		**
No Medicaid	88.3	—	94.5	—	96.7	—
Medicaid	92.5	—	96.1	—	97.5	—
Current Reason for Entitlement				***
Aged	88.9	—	95.1	—	97.0	—
Disabled	89.0	—	93.1	—	95.8	—
Metropolitan Area Status		***		**		**
Non-Metropolitan	92.3	—	95.7	—	97.7	—
Metropolitan	87.4	—	94.2	—	96.4	—
HCC-DCG Risk Score Quintiles3		*
0-20 % (Lowest Score)	88.3	—	94.3	—	96.1	—
20-40 %	89.5	—	94.6	—	97.7	—
40-60 %	90.3	—	95.4	—	97.2	—
60-80 %	89.3	—	95.0	—	96.7	—
80-100 % (Highest Score)	87.1	—	94.2	—	96.5	—
General Health				***
Excellent	89.9	—	95.3	—	96.5	—
Very Good	88.9	—	95.5	—	97.2	—
Good	89.4	—	95.5	—	97.0	—
Fair	88.4	—	93.3	—	97.0	—
Poor	87.9	—	91.9	—	95.8	—
With ADL Difficulties
None	89.1	—	94.7	—	96.8	—
1-2	89.2	—	95.0	—	97.0	—
3-4	87.8	—	94.4	—	96.1	—
5-6	87.2	—	93.0	—	98.1	—
With IADL Difficulties
None	88.8	—	95.2	—	97.0	—
1-2	89.4	—	94.0	—	96.4	—
3-4	87.5	—	94.5	—	96.6	—
5-6	90.7	—	94.9	—	98.2	—

^*p<0.1.

^**p<0.05.

^***p<0.01.

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded. The conditional response rate for the second (third, fourth) year panel attrition analysis sample is defined as second (third, fourth) year attrition sample respondents divided by eligibles, where eligibles are restricted to first (second, third) year respondents who are alive on January 1 of the second (third, fourth) year.

²Statistical significance testing for distribution.

³Diagnosis-based health status index computed from provider bills (claims). A higher hierarchical condition categories-diagnostic cost group (HCC-DCG) score indicates poorer health.

NOTES: MCBS is Medicare Current Beneficiary Survey. ADLs are activities of daily living. IADLS are instrumental activities of daily living. Data for second (third, fourth) year attrition sample weighted by first (second, third) year nonresponse adjusted weights.

SOURCE: RTI analysis of the 1997-1999 MCBS.

Response rates by subcategory are relatively consistent without many large variations among the groups. As shown in Table 5, one factor affecting response rate remains prominent in all three attrition samples: the statistically significant lower response rate in metropolitan areas versus non-metropolitan areas. In addition, the higher response rate for Medicaid enrollees compared with non-Medicaid enrollees is statistically significant in two of the three attrition samples.

Although Medicaid eligibility is often correlated with poor health status, we do not find any consistent evidence that conditional response rates are different for those in poor health versus those in better health. However, our first year nonresponse analysis indicated that sicker beneficiaries have a higher propensity to respond to the MCBS (Tables 1 and 2). These two findings suggest that different beneficiary characteristics affect first year nonresponse and panel attrition.

Our findings also suggest that beneficiary characteristics affecting the response propensity vary for each attrition sample. For example, as shown in Table 6, nonrespondents in the second year attrition sample were not statistically different from respondents in terms of self-reported general health status. Nonrespondents in the third year attrition sample, however, assessed their own health significantly differently than respondents. More than 12 percent of nonrespondents reported their health as poor (versus 7.9 percent for respondents), and 23.9 percent reported their health as fair (versus 17.8 percent for respondents). Finally, in the fourth year attrition sample, again no difference in self-reported general health status is found between respondents and nonrespondents.

Table 6. Comparison of MCBS Eligibles, Respondents, and Nonrespondents for Panel Attrition Analysis Samples, by Selected Characteristics¹.

Characteristic	Second Year Attrition Sample				Third Year Attrition Sample				Fourth Year Attrition Sample
	Eligible	Respondent	Nonrespondent	Statistical Significance	Eligible	Respondent	Nonrespondent	Statistical Significance	Eligible	Respondent	Nonrespondent	Statistical Significance
Sample	7,544	6,708	836	—	6,345	6,009	336	—	5,437	5,264	173	—
	Percent				Percent				Percent
Age
Under 65 Years	13.6	13.6	13.3	—	12.9	12.6	16.8	**	11.6	11.5	14.7	—
65-74 Years	45.5	45.9	41.9	**	43.1	43.1	42.6	—	39.7	39.6	44.3	—
75-84 Years	31.9	31.6	34.5	*	34.6	34.7	33.5	—	38.5	38.7	32.1	*
85 Years or Over	9.0	8.9	10.3	—	9.4	9.5	7.1	—	10.1	10.2	8.9	—
Female	56.1	55.9	57.4	—	56.0	55.8	58.9	—	56.5	56.4	58.9	—
Income Categories
Under $15,000	42.9	43.0	42.3	—	40.9	41.3	33.6	***	41.1	41.5	30.4	***
$15,001-$30,000	25.0	25.5	21.2	***	32.8	32.6	35.1	—	33.9	33.8	36.0	—
$30,001-$50,000	12.9	13.1	11.2	—	16.9	16.8	18.9	—	15.0	14.7	23.1	***
Over $50,000	9.5	9.4	10.6	—	9.5	9.3	12.5	*	10.0	10.0	10.6	—
Race
White	86.0	85.8	87.4	—	86.1	86.2	85.7	—	86.8	86.7	90.3	—
Black	8.7	8.7	8.1	—	8.6	8.6	7.3	—	8.4	8.4	8.0	—
Other	5.4	5.5	4.5	—	5.3	5.2	7.0	—	4.8	4.9	1.7	**
Medicaid Status	12.9	13.5	8.5	—	12.8	13.0	8.2	—	12.8	13.0	7.2	**
Metropolitan Area Status	71.6	70.5	80.4	***	71.2	70.9	77.6	***	69.7	69.4	77.0	**
Mean HCC-DCG Risk Score2	0.93	0.92	0.96	*	0.94	0.93	1.02	**	0.97	0.97	0.97	—
Total Medicare Expenditures	4,019	4,060	3,695	—	3,584	3,520	4,709	**	3,789	3,803	3,369	—
Inpatient Ependitures	1,966	2,009	1,628	—	1,634	1,592	2,380	**	1,708	1,715	1,482	—
Prescription Drug Expenditures	N/A	N/A	N/A	—	872	872	865	—	866	867	859	—
General Health
Excellent	16.0	16.1	14.9	—	14.5	14.6	13.0		14.3	14.2	16.4	—
Very good	25.5	25.5	25.6	—	27.0	27.3	23.0	*	26.3	26.5	22.0	—
Good	29.2	29.3	28.5	—	32.0	32.3	27.5	*	32.9	33.0	32.4	—
Fair	18.8	18.6	20.2	—	18.2	17.8	23.9	***	18.4	18.4	19.5	—
Poor	10.3	10.3	10.9	—	8.1	7.9	12.6	***	7.9	7.9	9.7	—
With ADL difficulties
None	67.8	67.8	67.2	—	72.4	72.4	72.6	—	72.9	72.8	74.9	—
1-2	21.3	21.4	20.6	—	18.8	18.8	17.1	—	18.1	18.2	16.8	—
3-4	7.0	6.9	7.5	—	5.7	5.6	6.2	—	5.6	5.6	6.3	—
5-6	4.0	3.9	4.7	—	3.1	3.0	4.1	—	3.3	3.3	2.0	—
With IADL Difficulties
None	51.7	51.7	52.3	—	54.8	55.1	50.7	*	54.5	54.6	53.0	—
1-2	30.5	30.7	29.3	—	29.7	29.5	33.5	—	29.0	28.9	33.3	—
3-4	11.3	11.1	13.0	—	10.2	10.2	10.6	—	10.3	10.3	11.1	—
5-6	6.4	6.5	5.5	—	5.2	5.2	5.3	—	6.1	6.3	2.6	**

^*p<0.1.

^**p<0.05.

^***p<0.01.

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded.

²Diagnosis-based health status index computed from provider bills (claims). A higher hierarchical condition categories-diagnostic cost group (HCC-DCG) score indicates poorer health.

NOTES: MCBS is Medicare Current Beneficiary Survey. Data for second (third, fourth) year panel attrition analysis sample weighted by first (second, third) year nonresponse adjusted weights. ADLs are activities of daily living. IADLS are instrumental activities of daily living. Statistical testing between respondents and nonrespondents is equivalent to statistical testing between respondents and eligibles. Beneficiary characteristics are measured prior to the year in which response status is determined. NA is not available. A higher HCC-DCG score indicates poorer health.

SOURCE: RTI Analysis of the 1997-1999 MCBS.

Although we find statistically significant differences between respondents and nonrespondents on such sociodemographic characteristics as age, income, and geographic distribution, because of the high MCBS conditional response rates for each attrition sample, the magnitude of the differences between eligibles and respondents is relatively small, and thus unlikely to create a major potential for bias.

Our descriptive findings are supported by multiple logistic regression analyses (Table 7). Metropolitan area residence is found to be a consistent and significant factor affecting the probability of response in all MCBS attrition samples when other factors are held constant. In addition, Medicaid enrollment is a significant factor in two of the three attrition samples.

Table 7. Logistic Regression Models Estimating Likelihood of MCBS Conditional Response for Panel Attrition Analysis Samples¹.

Characteristic	Odds Ratio
	Second Year Attrition Sample	Statistical Significance	Third Year Attrition Sample	Statistical Significance	Fouth Year Attrition Sample	Statistical Significance
Age (65-74 Years Category Omitted)
Under 65 Years	0.83	—	0.71	—	0.69	—
75-84 Years	0.82	**	1.05	—	1.35	—
85 Years or Over	0.77	*	1.29	—	1.25	—
Sex (Male Category Omitted)
Female	0.94	—	0.78	**	0.76	*
Medicaid Status (Non-Medicaid Category Omitted)
Enrolled in Medicaid	1.84	***	1.92	***	1.62	—
Race (White Category Omitted)
Black	1.06	—	1.12	—	0.99	—
Other	0.98	—	0.57	**	3.06	—
Original Reason for Entitlement (Aged Category Omitted)
Disabled	0.71	**	1.15	—	1.12	—
Census Regions (North East Category Omitted)
North Central	1.09	—	1.13	—	1.29	—
South	1.18	—	1.05	—	1.17	—
West	1.24	*	1.46	*	1.68	*
Others2	4.21	***	4.01	**	1.81	—
Metro Area (Non-Metro Category Omitted)
Metropolitan Area Status	0.60	***	0.71	**	0.71	*
HCC-DCG Quintiles (0-20% Quintile Category Omitted)3
20-40%	1.13	—	0.89	—	1.18	—
40-60%	1.26	*	1.04	—	0.92	—
60-80%	1.20	—	0.96	—	0.86	—
80-100%	0.98	—	0.86	—	0.79	—
Marital Status (Non-Married Category Omitted)
Married	0.60	—	0.94	—	0.89	—
Education (Absence of College Degree Category Omitted)
College Degree	1.04	—	1.07	—	1.15	—
Income (Over $50,000 Category Omitted)
Under $15,000	1.32	***	1.65	—	1.32	—
$15,001-$30,000	1.64	***	1.34	—	1.01	—
$30,001-$50,000	1.58	—	1.21	**	0.67	—
Self-Reported General Health Status (Excellent Category Omitted)
Very Good General Health	0.97	—	1.05	—	1.54	*
Good	0.94	—	0.97	—	1.34	—
Fair	0.82	—	0.59	**	1.23	—
Poor	0.84	—	0.50	**	0.97	—
Difficulty with ADLs (0 Category Omitted)
1-2	1.03	—	1.29	—	1.14	—
3-4	0.87	—	1.15	—	0.89	—
5-6	0.76	—	0.83	—	1.08	—
Difficulty with lADLs (0 Category Omitted)
1-2	1.14	—	0.86	—	0.86	—
3-4	0.98	—	1.98	—	0.92	—
5-6	1.50	—	1.05	—	2.42	—

^*p<0.1.

^**p<0.05.

^***p<0.01.

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded.

²Other includes Puerto Rico and other territories.

³Diagnosis-based health status index computed from provider bills (claims). A higher hierarchical condition categories-diagnostic cost group (HCC-DCG) score indicates poorer health.

NOTES: MCBS is Medicare Current Beneficiary Survey. Beneficiary charcteristics are measured prior to the year in which response status is determined. Data for second (third, fourth) year panel attrition analysis sample weighted by first (second, third) year nonresponse adjusted weights. For second, third, and fourth year panel attrition samples, N = 7,540, N = 6,340, and N = 5,433, respectively.

SOURCE: RTI Analysis of the 1997-1999 MCBS.

For all three attrition samples, MCBS adjustments for panel attrition aligned the distribution of respondents across sociodemographic and enrollment characteristics to be far more consistent with the eligible sample (Kautter et al., 2003). In particular, across all attrition samples, the adjustments were effective in correcting the Medicaid and metropolitan area distributions. In every attrition sample there are some analysis variables for which MCBS adjustments for panel attrition were not as effective. However, there is no particular pattern for what types of variables require additional adjustment.

In addition to analyzing conditional nonresponse in the MCBS, an analysis of cumulative nonresponse is conducted. Cumulative response rates in the MCBS are found to be comparable to other large national surveys (U.S. Bureau of the Census, 1998). While conditional response rates increase over the MCBS interview cycle, cumulative response rates decrease. As shown in Table 8, after 4 years of longitudinal data collection, the overall response rate for analysis of the complete longitudinal data set was 67.3 percent. Subpopulations exhibited a range of overall response rates. As an example, the range in cumulative response rates after four survey years is 77.0 percent for non-metropolitan residents and 63.3 percent for metropolitan residents.

Table 8. Cumulative Response Rates Across MCBS Interview Cycle, by Demographic, Eligibility, and Health Status Characteristics¹.

Characteristic	First Year (Initial Round)	Second Year	Third Year	Fourth Year
	N=14,315	N=7,544	N=6,345	N=5,437
	Percent
All Sample	82.6	73.4	69.5	67.3
Age
Under 65 Years	83.3	74.2	69.0	66.2
65-74 Years	82.4	74.1	70.3	67.9
75-84 Years	82.3	72.5	68.8	67.1
85 Years or Over	82.6	72.3	69.4	67.4
Sex
Male	83.9	75.0	71.2	69.0
Female	81.5	72.2	68.2	66.0
Race
White	82.3	73.0	69.2	66.9
Black	85.0	76.2	72.4	69.8
Other	82.2	73.8	69.1	68.4
Original Reason for Entitlement
Aged	82.3	73.3	69.4	67.2
Disabled	86.0	74.8	71.0	69.1
Medicaid status
No Medicaid	82.0	72.3	68.3	66.1
Medicaid	85.7	79.3	76.2	74.3
Current Reason for Entitlement
Aged	82.4	73.2	69.6	67.5
Disabled	83.3	74.2	69.0	66.1
Metropolitan Area Status
Non-Metropolitan	89.3	82.4	78.9	77.0
Metropolitan	79.8	69.7	65.7	63.3
Census Regions
North East	80.1	68.9	64.9	62.5
North Central	81.5	72.2	68.3	66.0
South	84.3	75.8	71.6	69.3
West	82.7	74.4	71.4	69.6
Others2	87.1	83.1	81.1	80.5
Mortality
Died in the Following Year	83.0	71.9	68.0	65.4
Survived the Following Year	82.5	73.5	69.6	67.4
HCC-DCG Risk Score Quintiles3
0-20% (Lowest Score)	79.1	69.9	65.9	63.3
20-40%	80.5	72.0	68.2	66.6
40-60%	82.3	74.3	70.9	68.9
60-80%	85.3	76.2	72.4	70.0
80-100% (Highest Score)	85.6	74.6	70.3	67.8

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded. The cumulative response rate for a panel in their second (third, fourth) MCBS year is approximated by calculating the product of their first year response rate and their conditional response rates through their second (third, fourth) survey year.

²Other includes Puerto Rico and other territories.

³Diagnosis-based health status index computed from provider bills (claims). A higher hierarchical condition categories-diagnostic cost group (HCC-DCG) score indicates poorer health.

NOTES: MCBS is Medicare Current Beneficiary Survey. Data are unweighted.

SOURCE: RTI Analysis of the 1997-1999 MCBS.

As shown in Table 9, there is a slight upward trend in the cumulative nonresponse bias for certain variables, such as Medicare total expenditures (from $85 to $108) and inpatient expenditures (from $47 to $74), indicating that further nonresponse adjustments might be warranted. However, overall, MCBS nonresponse weighting procedures were found to be effective in adjusting for cumulative nonresponse.

Table 9. Cumulative Nonresponse Bias Across MCBS Interview Cycle, Before and After MCBS Nonresponse Adjustment¹.

Characteristic	First Year (Initial Round)		Second Year		Third Year		Fourth Year
	Before	After	Before	After	Before	After	Before	After
Age
Under 65 Years	0.25	0.17	0.29	0.20	0.06	0.15	-0.04	0.18
65-74 Years	-0.05	0.18	0.37	0.32	0.40	0.29	0.23	0.09
75-84 Years	-0.19	-0.29	-0.52	-0.41	-0.45	-0.43	-0.23	-0.29
85 Years or Over	-0.01	-0.06	-0.18	-0.15	-0.05	-0.04	-0.01	-0.03
Female	-0.76	-0.19	-0.93	-0.30	-1.10	-0.51	-1.18	-0.71
Race
White	-0.31	-0.28	-0.49	-0.35	-0.47	-0.30	-0.59	-0.37
Black	0.33	0.18	0.40	0.23	0.47	0.27	0.48	0.28
Other	-0.01	0.09	0.09	0.12	0.00	0.03	0.11	0.08
Originally Disabled	0.23	0.12	0.06	-0.09	0.09	-0.09	0.11	-0.07
Medicaid Status	0.55	0.38	1.12	0.37	1.38	0.48	1.57	0.53
Currently Disabled2	0.25	0.17	0.29	0.20	0.06	0.15	-0.05	0.18
Metropolitan Area Status	-2.27	-0.51	-3.38	-0.64	-3.74	-0.76	-4.00	-0.78
Census Regions
North East	-0.59	-0.10	-1.24	-0.16	-1.42	-0.29	-1.62	-0.45
North Central	-0.28	0.12	-0.39	-0.09	-0.36	-0.05	-0.33	-0.02
South	0.83	0.14	1.27	0.16	1.23	0.01	1.23	-0.01
West	-0.07	-0.15	0.11	-0.03	0.25	0.13	0.39	0.26
Other	0.10	-0.01	0.23	0.12	0.29	0.20	0.32	0.22
Mean HCC-DCG Risk Score	0.02	0.01	0.01	0.01	0.00	0.00	0.00	0.00
HCC-DCG Risk Score Quintiles3
0-20% (Lowest Score)	-0.80	-0.36	-0.88	-0.48	-0.92	-0.44	-1.06	-0.54
20-40%	-0.55	-0.42	-0.42	-0.29	-0.43	-0.33	-0.28	-0.21
40-60%	-0.05	-0.05	0.23	0.19	0.36	0.29	0.40	0.33
60-80%	0.66	0.45	0.74	0.54	0.82	0.56	0.81	0.54
80-100% (Highest Score)	0.74	0.39	0.32	0.05	0.19	-0.05	0.15	-0.08
Total Medicare Expenditures	$137.32	$85.11	$178.89	$138.83	$115.03	$90.74	$129.77	$108.02
Inpatient	$67.35	$47.39	$110.74	$96.66	$68.42	$64.12	$76.33	$74.10
Mortality
Died in the Following Year	0.02	-0.07	0.15	-0.11	0.17	-0.12	0.18	-0.13

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded. The cumulative nonresponse bias at each year is approximated by summing the estimated biases up to and including that year.

²Current reason for Medicare entitlement is disability—equivalent to under 65 age group.

³Diagnosis-based health status index computed from provider bills (claims).

NOTES: MCBS is Medicare Current Beneficiary Survey. HCC-DCG is hierarchical condition categories-diagnostic cost group.

SOURCE: RTI Analysis of the 1997-1999 MCBS.

Item Nonresponse

Methodology

Item nonresponse in the MCBS is also analyzed. The analysis sample for item nonresponse is confined to 1999 MCBS Access to Care file⁸ respondents and, like the analyses of initial round unit nonresponse and panel attrition, are restricted to beneficiaries residing in the community, enrolled in traditional FFS Medicare, and not eligible for Medicare by ESRD. There are 12,524 beneficiaries meeting these criteria.

Item nonresponse rates are derived by calculating the ratio of item nonrespondents to item eligibles. The following response categories are assumed to be item nonresponse: (1) not ascertained, (2) don't know, and (3) refused. In addition, we rank the survey variables by item nonresponse, and examine the distribution of the item nonresponse rates. Finally, survey variables with relatively high item nonresponse rates are selected, and the distribution of nonresponse categories is examined to determine how prevalent the don't know response category is and whether it is a legitimate, meaningful response. For the selected survey variables, item respondents and nonrespondents are compared using available proxy measures.

Findings

As shown in Table 10, item nonresponse is generally low in the 1999 MCBS Access to Care file. The mean item nonresponse rate across survey variables is 1.6 percent, and the majority of the variables have negligible item nonresponse of at most 0.3 percent. However, the distribution of item nonresponse rates across survey variables in the MCBS is skewed, with 10 percent of survey variables having an item nonresponse rate of at least 5.4 percent. For example, the survey question on income has a 6.7-percent item nonresponse rate. It is important to note though that some variables with high item nonresponse rates are only asked of a small subset of survey participants. For example, for the survey question “Need help three months from now with toileting,” the item nonresponse rate is 22.2 percent, but the number of eligibles for this survey question is only 18. With these small sample sizes, the data have limited utility even if all eligible persons responded.

Table 10. Distribution of MCBS Item Nonresponse Rates¹.

	Item Nonresponse Rate2	Item Eligibles	Item Nonrespondents
			All Item Nonrespondents	Don't Know
Mean Item Nonresponse Rate	1.6	—	—	—
Quantiles
100% (Maximum)	28.9	—	—	—
90%	5.4	—	—	—
75%	1.3	—	—	—
50% (Median)	0.3	—	—	—
25%	0.0	—	—	—
Selected Variables
Plan 1 Cover Stay in Nursing Home3	26.6	8,550	2,276	2,266
Need Help 3 Months from Now with Toileting4	22.2	18	4	4
Current Veteran's Administration Disability Rating3	13.9	388	54	53
Does Doctor Make House Calls3	9.8	11,675	1,141	1,141
Income5	6.7	12,524	839	261
High School Grade Completed5	0.7	12,524	86	68

¹MCBS community, fee-for-service sample. Beneficiaries with end stage renal disease are excluded.

²Item nonresponse rates are derived by calculating the ratio of item nonrespondents to item eligibles. The following response categories are assumed to be item nonresponse: not ascertained; don't know; and refused.

³Knowledge question.

⁴Predictive question.

⁵Sensitive question.

NOTES: MCBS is Medicare Current Beneficiary Survey.

SOURCE: RTI Analysis of the 1999 MCBS Access to Care File.

Our analysis of response category distributions among item nonrespondents revealed certain patterns. “Refusals” and “not ascertained” item nonresponse choices are rare. The great majority of item nonrespondents select “don't know” and they often select this choice because there is no other appropriate valid response category available. For example, for the survey question “Current Veteran's Administration disability rating,” there are 54 item nonrespondents to the question, with 53 answering “don't know” (Table 10). In particular, survey participants often select “don't know” if they have trouble recalling a certain health or preventive event, or do not remember details about their military service history. Survey participants also select this answer choice to indicate their lack of knowledge about their health insurance coverage details or when they have difficulty predicting needing help with activities of daily living (ADLs). We suggest that only some of these “don't know” responses, e.g., for variables such as income and education, should be classified as true missing data. For other items, e.g., knowledge or amount of information that survey participants possess, these responses, if retained or reclassified, can provide additional valuable information.

The survey variables we examined indepth may be broadly classified into three groups (Kautter et al., 2003):

• Sensitive questions, such as income and education, that are known in survey research for yielding lower item response rates.

• Questions related to recall of certain past events such as eye exams, prostate cancer tests, and whether the Medicare & You handbook was received.

• Questions assessing beneficiary knowledge of various issues related to health and health insurance coverage.

Comparison of health and demographic characteristics of item respondents and nonrespondents to these variables revealed several patterns. While both income and education variables are considered sensitive items, item nonresponse to education is much lower than for income.⁹ Item nonresponse to the income question is associated with the characteristics generally associated with higher income (e.g., male, white, no Medicaid), whereas item nonresponse to the education question is associated with characteristics generally associated with lower education (older, sicker, Medicaid).

Nonrespondents to questions requiring recall tend to be older, have a higher proportion of minorities and higher rates of Medicaid enrollment, and are likely to be in significantly poorer health than respondents. However, item respondents and nonrespondents to knowledge questions appear to have fewer differences in demographic and health status characteristics. Item nonrespondents to knowledge questions tend to be less educated than respondents. While there are some other variations in demographic characteristics such as race or Medicaid enrollment, we could not detect any other particular patterns that set apart one group from another. There are no consistent differences in health status between the two groups.

Summary and Conclusions

This study had several objectives in evaluating the impact of nonresponse on MCBS estimates: (1) to examine unit nonresponse for beneficiaries in their initial interview round, (2) to evaluate panel attrition, (3) and to measure item nonresponse. For initial round nonresponse, although statistically significant differences occurred between respondents and nonrespondents on such demographic characteristics as sex, race, and geographic distribution, the magnitude of the differences between eligibles and respondents was relatively small and unlikely to cause a major potential for bias. However, current nonresponse adjustments were not as effective for health status, expenditure, and service utilization characteristics. Although initial nonresponse bias was small and further reduced by MCBS nonresponse weights, it was not entirely eliminated.

Beneficiary characteristics affecting the response propensity varied for each panel attrition sample, but because of the high MCBS conditional response rates for each sample, the magnitude of the differences between eligibles and respondents was relatively small, and thus unlikely to create bias. Cumulative response rates were found to be comparable to other large national surveys. While conditional response rates increased over the MCBS interview cycle, cumulative response rates decreased. Finally, item nonresponse was generally low in the MCBS, with the exception of several items pertaining to recall of past events and knowledge of certain health insurance information.¹⁰

Nonresponse in panel surveys can be a serious problem because it is cumulative over all rounds of the survey. Our findings indicate that for most of the measures studied, the bias caused by differences between nonrespondents and respondents in the MCBS was substantially reduced or eliminated by the nonresponse procedures currently employed.