Abstract
Background
The analysis presented in this paper examines the multi-year capacity of the Medicare Current Beneficiary Survey (MCBS).
Methods
We systematically reviewed the literature for methodological approaches in research using multiple years of the MCBS and categorized the studies by study design, use of survey sampling weights, and variance adjustments. We then replicated the approaches in an empirical demonstration using functional status (activities of daily living (ADL) and 2005-2007 MCBS data.
Results
In the systematic review, we identified 22 pooled, 17 repeated cross-sectional, and 17 longitudinal studies. Less than half of these studies explicitly described the weighting approach or variance estimation. In the empirical demonstration, we showed that different study designs and weighting approaches will yield statistically different estimates.
Conclusion
There is a variety of methodological approaches when using multiple years of the MCBS, and some of them provide biased results. Research needs to improve in describing the methods and preferred approaches for using these complex data.
Keywords: Medicare current beneficiary survey, Health Care Surveys, Survey design, Survey sampling weight, Systematic review, SAS, SUDAAN
INTRODUCTION
The Medicare Current Beneficiary Survey (MCBS) is the principal national survey for informing and evaluating health policies for the Medicare program. Since 1991, the Centers for Medicare & Medicaid Services have sponsored this survey, at a cost of $14.8 million a year to administer. To date, nearly 300 published studies have used the MCBS for critical research questions, most recently for assessing the nationwide impact of the Medicare prescription drug program (Part D) (Madden, Graves, Ross-Degnan, Briesacher, & Soumerai, 2009; Madden et al., 2008). However, one aspect of the MCBS that is not well known is its capability for short longitudinal analyses of up to 3 years. Unlike many other national surveys, the MCBS collects follow up data on each individual for up to 3 years of observation. Important questions about health-related changes over time can be answered by combining multiple years of the MCBS.
However, the longitudinal data structure of the MCBS can also complicate analyses and cause confusion about appropriate research approaches. Research that uses multiple years of MCBS data faces the following issues. First, there is a nontrivial overlap in the MCBS sample each year (approximately 60% from one year to the next). This means that the assumption of statistical independence is violated for many standard analytic approaches. Second, the repeated observations introduce an additional level of clustered data beyond the survey design. The MCBS's multi-staged complex sampling design creates data that are clustered in primary sampling units and strata, and that require special statistical approaches for correct estimation and tests. However, the use of multiple years of the MCBS means the data are also clustered at the individual-level. The necessity of correcting for this additional clustering is unclear as long as the primary sampling unit is specified correctly (Sarndal & Swensson, 2003). Thirdly, the choice of weights becomes more complex. The MCBS provides longitudinal weights, but they are not available for the whole sample, and the use of those weights may be unfamiliar to many researchers. Lastly, most statistical software (e.g., SAS, R) now offers survey analysis procedures with a variety of variance estimation methods; this raises questions about the necessity of purchasing a separate license for specialty survey estimators (e.g., WesVAR, SUDAAN, STATA).
The statistical survey literature provides a large body of sophisticated writings on preferred methodological approaches when using complicated data structures like the MCBS (Korn & Graubard, 1999). However, there are only two papers specific to statistical issues in using the MCBS for longitudinal analysis and neither is aimed at a general research audience (Ferraro & Liu, 2005; O'Connell, Chu, & Bailey, 1997). What is still lacking is practical and accessible guidance. As a result, less knowledgeable researchers may be using multiple years of the MCBS in less than optimal ways, or worrying unnecessarily over rather trivial statistical issues. The resolution of these issues is important, since tracking changes in the health status and health care of the Medicare population over time requires a clear understanding of the trade-offs in choosing one approach over another with the MCBS.
The purpose of this study was two-fold. First, we systematically reviewed the methods researchers have used when conducting research using multiple years of MCBS data. Second, we applied the most common approaches identified in the literature review to an empirical demonstration to understand the implications of these methods. For the demonstration, we used the functional status measure of activities of daily living and three years of the Access to Care files (ATC) of the MCBS (2005-2007). Finally, we conclude with several recommendations when using multiple years of the MCBS as drawn from the review and our demonstration.
Brief description of MCBS and Special Data Considerations
The MCBS is an annual population survey of approximately 16,000 Medicare enrollees who provide information about their demographic and household characteristics, as well as health insurance, health status, and medical care encounters(Adler, 1994). The survey is repeated annually for three years with additional thrice-annual interviews for a defined subset of the sample. All interviews are conducted in person with computer-assisted personal interview technology. The MCBS is released each year as two data files, the Access to Care (ATC) and the Cost and Use (CAU), although the samples in both files share substantial overlap. Each year of the MCBS may be used separately for cross-sectional analyses, or linked together over multiple years for conducting longitudinal analyses.
The MCBS sample is selected through a three-stage process that results in a sample of individuals who are more likely to live close to each other than a random draw of individuals. In the first stage, primary sampling units (PSUs) are selected consisting of metropolitan statistical areas or clusters of nonmetropolitan counties. In the second stage, ZIP code clusters are sampled within the PSUs. In the third stage, beneficiaries are sampled within the ZIP code clusters. Furthermore, the MCBS sampling design incorporates roughly equal-sized stratum of PSUs sorted by the percent of Medicare beneficiaries enrolled in HMO plans (and in some cases also by the percentage of minority beneficiaries). The MCBS also oversamples individuals under age 65 (disabled) and age 85 or older (the oldest old) to increase the precision of estimates for these groups.
The MCBS also uses a rotating panel sampling design. In each year of data, there are four overlapping sample panels with staggered entry into the survey. This means one panel is contributing data to the MCBS for the first time, while the other panels are supplying data for the second, third, and fourth time. In each year of data the last panel from the prior year will have been replaced with a new sample panel to ensure that the total MCBS sample provides an accurate reflection of the current Medicare population. Thus, the duration of follow up observation will vary for each sample panel, depending on the years of data used.
Each year of MCBS data contains several types of sampling weights that can be used to produce estimates from the sample that are generalizable to Medicare population. These weights reflect the overall selection probability of each sample person and also include adjustments for survey nonresponse and post-stratification elements related to sample entry, age, sex, race, region, and metropolitan area status. Cross-sectional survey weights are available for the entire MCBS sample. Longitudinal weights are also available for sample persons with full-year observations in the ATC file. These are also called “backward longitudinal” weights, because they apply to only the surviving sample and they are used to “look back” to data collected in previous releases(Ferraro & Liu, 2005). Additionally, the use of the longitudinal weights requires that the study dataset be constructed with two or more annual releases(Ferraro & Liu, 2005). The one-year backward longitudinal weights are used to analyze two years of data; the two-year backward longitudinal weights are used to analyze three years of data; and the three-year backward longitudinal weights are used to analyze four years of data. Omitting the sampling weights, restricting the use of weights to certain portions of the overall sample, or using cross-sectional weights for longitudinal analyses will generally result in biased estimates that cannot be used to make inferences regarding the target population(Centers for Medicare and Medicaid Services, 2007). Analyses that do not account for the MCBS's weighting, clustering, and stratification design will generally get standard errors that are smaller than they should be.
SYSTEMATIC REVIEW
Methods
Search Strategy
We conducted searches of three electronic databases (PUBMed, EconLit, and the National Bureau of Economic Research Working Papers) for English-language using a key term “Medicare Current Beneficiary Survey” and then reviewed titles and abstracts of the retrieved articles for evidence of cohort study, longitudinal study, panel data or multiple year studies. The review covered the years of 1992-2009. Three investigators (BAB, JT, CD) developed the search strategy, and a research assistant (S) retrieved the articles. All candidate papers were randomly divided into three groups and each group was assigned to two investigators for two independent reviews. After the first round review, we conducted a reconciliation process for papers with discordant opinion, as settled by a third review.
Inclusion and exclusion criteria
Articles were selected according to three criteria: (a) reported original study results using the MCBS; (b) used multiple years of data, meaning the study should utilize at least three years of MCBS data; and (c) provided sufficient detail on the use of sampling weight. We excluded studies focused on pure methodological or statistical research.
Data abstraction
As a first step, we developed an online data abstraction form using SurveyMonkey (Ryan Finley, SurveyMonkey.com, Portland, OR). Data were recorded on abstract forms and exported to Microsoft Excel files for further analysis. We captured information on the description of a study, including author, publication year, file type, design, sample, use of weights, statistical software, and outcome variables.
Second, we classified all studies into three study designs (pooled data, repeated cross-sectional, and longitudinal) based on data structure and analytic approach. In pooled data, data from multiple years of the survey are combined to obtain a single estimate without regard to calendar time or repeated measures. Weights used are year-specific cross-sectional weights, which strictly speaking are not appropriate for the subset of respondents in both files. The available MCBS cross-sectional weights are intended to be applied to the total sample in a specific year (including non-overlapping cases). The available cross-sectional weights are used as a crude substitute for longitudinal weights. The pooled estimate using these weights is, thus, an average of the annual (cross-sectional) estimates for the overlapping cases.
In repeated cross-sectional studies, the data were treated as separate yearly files and the analyses were produced as multiple annual estimates. This analysis will involve some respondents who appear in multiple annual data files and some who appear only once. Generally, year-specific cross-sectional weights are used. These studies were further classified by whether the analysis included or excluded repeated measures.
Lastly, in longitudinal studies, the study used multiple years of data to assess within-individual changes over time. All sample persons must be observed at least twice and longitudinal weights should be used. The difficulty with this design is that longitudinal weights are available for only surviving sample persons, so appropriately weighted transitions that include death endpoints cannot be conducted.
Results
Exhibit 1 shows the results from the literature search strategy. We identified 274 articles through the initial key term search, and excluded 152 irrelevant articles through a title and abstract review. Several rounds of 122 full text reviews resulted in a final selection of 55 articles:
22 used pooled analysis designs (Crystal, Sambamoorthi, Walkup, & Akincigil, 2003; Ettner, Hermann, & Tang, 1999; Fiscella & Holt, 2007; Fiscella, Holt, Meldrum, & Franks, 2006; Foote & Hogan, 2001; Hancox et al., 2005; Hill, Fillit, Thomas, & Chang, 2006; Holt, Franks, Meldrum, & Fiscella, 2006; Hoover, Crystal, Kumar, Sambamoorthi, & Cantor, 2002; Housman et al., 2003; Kemper, Weaver, Short, Shea, & Kang, 2008; Noyes, Liu, Holloway, & Dick, 2007; Noyes, Liu, & Holloway, 2006; Noyes, Liu, Li, Holloway, & Dick, 2006; O'Malley & Forrest, 2006; Patel & Davis, 2006; Rice, Snyder, Kominski, & Pourat, 2002; Riley, Lubitz, & Zhang, 2003; Simoni-Wastila, Zuckerman, Shaffer, Blanchette, & Stuart, 2008; Stuart, Simoni-Wastila, & Chauncey, 2005; Waidmann & Liu, 2000; Wei, Sambamoorthi, Olfson, Walkup, & Crystal, 2005)
17 conducted repeat cross-sectional studies (Centers for Disease Control and Prevention, 2004; Chen et al., 2001; Craig, Kreling, & Mott, 2003; Doshi, Polsky, & Chang, 2007; Kamal-Bahl, Stuart, & Beers, 2005; Lakdawalla et al., 2003; Laschober, Kitchman, Neuman, & Strabic, 2002; Lundy & Craig, 2006; McCormack et al., 2002; McKnight, 2006; Murray & Eppig, 2002; Riley & Zarabozo, 2006; Rosenbach, Acamache, & Khandker, 1995; Sambamoorthi, Olfson, Walkup, & Crystal, 2003; Stein, Sloan, & Lee, 2007; Stuart et al., 2003; Yang, Norton, & Stearns, 2003), and
17 were longitudinal studies (Anderson, Norton, & Dow, 2003; Anderson, Norton, & Kenney, 2003; Balsa, Cao, & McGuire, 2007; Briesacher, Stuart, Ren, Doshi, & Wrobel, 2005; Cai & Lubitz, 2007; Chin, Zhang, & Rathouz, 2003; Fisher et al., 2003; Iezzoni, Davis, Soukup, & O'Day, 2004; Lakdawalla et al., 2003; Lubitz, Cai, Kramarow, & Lentzner, 2003; Mello, Stearns, & Norton, 2002; Mello, Stearns, Norton, & Ricketts, 2003; Porell & Miltiades, 2001a; Porell & Miltiades, 2001b; Shang & Goldman, 2007; Tosteson, Gottlieb, Radley, Fisher, & Melton, 2007; Zhou Yang, Gilleskie, & Norton, 2004).
Some studies used multiple designs, especially in sensitivity analyses, so the total exceeds 100%. Exhibit 2 summarizes the main study approaches used in research with multiple years of the MCBS. Sampling weights were inconsistently described or not applied in most cases. In 35 of the 55 studies, the papers did not describe the weighting approach or described the use of “survey weights,” but did not specify which weights. Four studies explicitly described conducting an unweighted analysis, 10 applied the cross-sectional weights, and only 4 described using the longitudinal weights. Ten studies also described adjusting the survey weights. For instance, Lakdawalla and colleagues, conducting a repeated cross-sectional analysis of only respondents in their first survey year, normalized the cross-sectional weight of each year by dividing the weight of each observation by the total sample weight from the relevant survey year.
Exhibit 2. Summary of Study Designs and Approaches Identified in Literature Review.
Studies | Totala |
---|---|
All | 55 |
Design* | |
Pooled | 22 |
Repeated Cross-sectional | 17 |
Longitudinal/Panel | 17 |
Use of Weights* | |
Unweighted | 4 |
Cross-sectional weights | 10 |
Longitudinal weights | 4 |
Averaged or normalized weights | 10 |
Not described/unclear | 35 |
Approach for Survey Design Effects* | |
Used survey data software | 33 |
Used other method (e.g., bootstrapping with PSUs) | 3 |
Explicitly did not adjust for survey design | 2 |
Not described/unclear | 22 |
Approach for Repeat Observations* | |
Used longitudinal/panel data estimators | 22 |
Used other methods (e.g., removed duplicate observations) | 25 |
Explicitly treated observations as independent | 2 |
Did not conduct statistics over time | 4 |
Not described/unclear | 15 |
Total exceeds 100% due to studies using multiple designs and approaches
SOURCE: Data collected by author
To address the survey design effects, 60% (Appendix: Exhibit A1) described using survey data software (e.g., SUDAAN, or the survey estimators of SAS or STATA). Three studies described using bootstrapping methods with the PSUs, and 2 studies explicitly described ignoring the MCBS's survey design. Forty percent of the studies did not clearly describe their approach for addressing the survey design effects.
As for the repeated observations (Appendix: Exhibit A1), 40% of the studies used longitudinal or panel data estimators to explicitly examine change within individuals. Over 45% of the studies described using other methods (e.g., removed duplicate observations) and 2 studies treated the observations as independent. Four studies provided only descriptive information (level trends) and did not conduct statistical tests over time. Fifteen studies did not describe the approach for addressing the repeated observations in the data.
EMPIRICAL DEMONSTRATION
Methods
Data source
We used MCBS data from the Access to Care (ATC) for the years 2005 to 2007. Exhibit 3 illustrates our sample selection procedures. We specified four designs and included only the community dwelling sample. In addition to conducting an unweighted analysis, we also applied a variety of weighting techniques as described below.
(a) Pooled Study Design (Exhibit 3a):
Sample: We pooled all observations (n=44,237) from 2005 to 2007, which were used to calculate a pooled mean activity of daily living (ADL) score. These observations represented 24,423 unique individuals.
Weights: We summed the cross-sectional weights and divided by the number of years that an individual has been in the MCBS. For example, a person with 3 years of data would have a weight = (cross-sectional weight from year 1 + cross-sectional weight from year 2 + cross-sectional weight from year 3)/3, but someone with only 1 year of data would have a weight equal to the cross-sectional weight for the year the person was interviewed.
(b) Repeated Cross-sectional Design (Exhibit 3b):
Sample: We included individuals observed in 2005, 2006, and 2007. Individuals in each year (2005: n=14,701; 2006: n=14,732 and 2007: n=14,804), instead of all observations from three years, were used to calculate mean ADL scores from 2005 to 2007.
Weights: We applied the cross-sectional weights applicable to each year.
We also considered two types of longitudinal designs.
(c) Longitudinal Design I – repeated measures of ADL (Exhibit 3c):
Sample: We included all individuals with repeated ADL measures from 2005 to 2007. The sample included 9,799 observations in 2005, 14,732 in 2006 and 9,969 in 2007.
Weights: We used the cross-sectional weights from the middle year, 2006.
(d) Longitudinal Design II – individuals who were interviewed all three years (Exhibit 3d):
Sample: We only included individuals who have survived for three years—having an ADL score in each year (n=6,130).
Weights: We used three-year backward longitudinal weights from the 2007 file.
Outcomes Measure: We selected functional status for analysis using a 6-item assessment of limitations in activities of daily living (ADL) including bathing, dressing, toileting, transferring, continence, and feeding. This outcome is frequently cited in the literature.
Statistical Plan: We calculated unweighted and weighted means and standard errors according to the four designs (pooled, repeat cross-sectional and two longitudinal designs). For repeat cross-sectional and two longitudinal designs, we also calculated unweighted and weighted changes in average ADL from 2005 to 2006 and from 2006 to 2007 using PROC SURVEYREG (SAS) and PROC DESCRIPTS (SUDAAN). All weighted analyses were adjusted for the survey's complex sampling structure. We used PSU level variance estimations with a Taylor Linearization approach (SAS and SUDAAN code can be found in the Appendix). All statistical analyses were carried out with SAS 9.2 (SAS, Cary, NC) and SUDAAN 10.0 version (RTI, Research Triangle Park, NC). Additionally, this study received an exemption from the University of Massachusetts' Institutional Review Board for the use of previously collected and de-identified data.
Results
Exhibit 4 compares weighted and unweighted estimates of mean ADL scores in ATC files. Samples sizes vary from 6,130 to 44,237. The weighted mean ADL scores were always smaller than the unweighted mean ADL scores. The mean ADL scores varied by study design and ranged from 0.5767 to 0.6872 in unweighted analyses, while they ranged from 0.5616 to 0.6630 in the weighted analyses. In contrast, standard errors (SEs) estimated from unweighted analyses (range: from 0.0061 to 0.0155) were smaller than those from weighted analyses (range: from 0.0160 to 0.0234). Adjustments for the overlap in samples computed manually resulted in nearly identical SEs (data not shown).
Exhibit 4. Mean of Activity Daily Living Calculated for Different Study Designs Using MCBS Data from Multiple.
Year | Sample size | Un-weighted | Weighted* | ||
---|---|---|---|---|---|
Mean | Std error | Mean | Std error | ||
Pooled (design a in exhibit 3) | |||||
2005-2007 | 44,237 (24,423 unique individuals) | 0.6729 | 0.0061 | 0.6389 | 0.0163 |
Repeat cross-sectional (design b in exhibit 3) § | |||||
2005 | 14,701 | 0.6804 | 0.0105 | 0.6447 | 0.0160 |
2006 | 14,732 | 0.6872 | 0.0107 | 0.6630 | 0.0215 |
2007 | 14,804 | 0.6511 | 0.0103 | 0.6161 | 0.0178 |
Longitudinal using cross-sectional weights (design c in exhibit 3) † | |||||
2005 | 9,799 | 0.6185 | 0.0122 | 0.6018 | 0.0180 |
2006 | 14,732 | 0.6872 | 0.0107 | 0.6630 | 0.0215 |
2007 | 9,969 | 0.6212 | 0.0123 | 0.5834 | 0.0209 |
Longitudinal using 3 year backward longitudinal weights (design d in exhibit 3)‡ | |||||
2005 | 6,130 | 0.5883 | 0.0149 | 0.5743 | 0.0197 |
2006 | 6,130 | 0.5767 | 0.0149 | 0.5616 | 0.0234 |
2007 | 6,130 | 0.6026 | 0.0155 | 0.5772 | 0.0215 |
Calculations of weighed mean and standard error are account complex survey design. Weight calculation:
Sum of cross-sectional weights/ number of years in the survey. People in 05 only=05 cross-sectional weight/1. People in 05 and 06 = (05 cross-sectional weight +06 cross-sectional weight)/2. People in 05, 06 and 07 = (06 cross-sectional weight +06 cross-sectional weight + 07 cross-sectional weight)/3. People in 06 and 07 = (06 cross-sectional weight +07 cross-sectional weight)/2. People in 07 only= 07cross-sectional weight/1.
- People in 2005: 05 cross-sectional weights.
- People in 2006: 06 cross-sectional weights.
- People in 2007: 07 cross-sectional weights.
Use cross-sectional weights from the middle year (2006)
Use longitudinal weights from 2007 file (rix3)
SOURCE: MCBS access to care files: 2005-2007
Exhibit 5 shows differences in mean ADL scores between 2005 and 2006 and between 2006 and 2007 according to three study designs (repeat cross-sectional and two longitudinal designs). The differences in unweighted mean ADL scores between 2005 and 2006 and between 2006 and 2007 ranged from -0.0687 to 0.0660 and SEs ranged from 0.0148 to 0.2133. The differences in weighted mean ADL scores ranged from -0.0612 to 0.0797 and SEs ranged from 0.0144 to 0.0171. SAS and SUDAAN produced similar results.
Exhibit 5. Change of Activity Daily Living from 2005 to 2006 and from 2006 to 2007 Calculated for Different Study Designs Using MCBS Data From Multiple Years.
Year | Un-weighted | Weighted | ||
---|---|---|---|---|
Diff. in means | Std. error | Diff. in means | Std. error | |
Repeat cross-sectional (design b in exhibit 3), n=44237 person years§ | ||||
2005-2006 | -0.0068 | 0.0149 | -0.0183 | 0.0159 |
2006-2007 | 0.0361 | 0.0148 | 0.0469 | 0.0144 |
Longitudinal using cross-sectional weights (design c in exhibit 3), n=34500 person years† | ||||
2005-2006 | -0.0687 | 0.0163 | -0.0612 | 0.0171 |
2006-2007 | 0.0660 | 0.0162 | 0.0797 | 0.0160 |
Longitudinal using 3 year backward longitudinal weights (design d in exhibit 3), n=18390 person years‡ | ||||
2005-2006 | 0.0116 | 0.0213 | 0.0127 | 0.0159 |
2006-2007 | -0.0259 | 0.0213 | -0.0157 | 0.0152 |
calculations of weighed mean and standard error are account complex survey design
Weight calculation:
Use weights from each year's data
- People in 2005: 05 cross-sectional weights
- People in 2006: 06 cross-sectional weights
- People in 2007: 07 cross-sectional weights
Use cross-sectional weights from the middle year (2006)
Use longitudinal weights from 2007 file (rix3)
SOURCE: MCBS access to care files: 2005-2007
DISCUSSION
Our systematic review and empirical demonstration illustrate that methods matter in studies using multiple years of the MCBS and that it is especially important that the methods are clearly described. Not all methods can be used interchangeably. Pooling the data from different years aids in obtaining an overall summary, but this study design cannot be used to look at trends over time. A repeated cross-sectional design that excludes duplicates is not an appropriate design to use since it is impossible to identify a target population. A repeated cross-sectional design with duplicates can be used for computing estimates for individual years and their comparison across the years. As discussed in the Methods section, each of these designs requires different weighting schemes and all require the incorporation of the complex survey design in the analysis.
These findings support the following recommendations:
First, researchers need to be much more careful in clearly describing the approaches used in weighting and addressing design effects and repeat observations. This would help readers to interpret the results correctly and also to replicate the analysis. Second, unweighted estimates may be biased and should be interpreted only in terms of the sample, but not the Medicare population. This recommendation is based on the wide and statistically significant differences between weighted and unweighted estimates from any of the four study designs examined in our research. Third, as mentioned above, repeated cross-sectional study design without duplicates (entirely independent observations) is a study design that should generally be avoided. The unduplicated sample in subsequent years is quite different from the baseline sample and the reference population. In fact, any deviations from the total weighting design may bias the results unless appropriate adjustments are made to the weights. This is very difficult to do when the weights incorporate not just the response probability but also include non-response and post-stratification adjustments. Estimation procedures using domain (in SAS) or subpopulation (SUBPOPN in SUDAAN) command are preferred whenever possible. Fourth, the statistical plan for both pooled and repeated cross-sectional study design with duplicates need to address the substantial overlap in samples, otherwise the variance is underestimated since the correlation among the observations is not taken into account. Fifth, the longitudinal weights need to be more fully utilized when appropriate. These weights take into account the target population for the number of years of data being used and will help generalize the results to the appropriate population. We found few examples of studies using these weights. Sixth, there are several software packages (SAS, SUDAAN, R, STATA) that can easily accommodate the multiple levels of data clusters in three years of MCBS data, making it easy for less knowledgeable investigators to apply standard methods that incorporate the complex survey design aspects. Lastly, it is clear that we need a standardized approach for reporting the statistical approach in multi-year studies with the MCBS so we can compare results across different studies. We would like to re-iterate that it is critical that the analysis plan explicitly describe the methods used for addressing the weights and repeated measures.
In summary, all methods have their advantages and limitations as discussed above and researchers need to be aware of these when choosing the appropriate method to answer their questions of interest. We have only demonstrated the different designs to obtain a single summary statistic but these methods can be extended to accommodate more complicated analysis (e.g., regressions).
The implication of these methods and the recommendations will take on greater importance as more evaluations are undertaken to assess the impact of Medicare policies and medical advances over time. Researchers can best aid policymakers by applying consistent and clearly defined approaches. These methodological decisions may have a major bearing on how successful the Medicare program is in serving the health needs of the Medicare population as a whole.
Appendix
Acknowledgements
The authors would like to acknowledge the assistance of Sarah Velten.
Footnotes
Financial Disclosure: Dr. Briesacher was supported by a Research Scientist Development Award from the National Institute on Aging (K01AG031836) and Dr. Chen was supported for an administrative supplement to the same grant. Dr. Doubeni was supported by the National Cancer Institute (5 K01 CA127118 and R01CA151736).
REFERENCES
- Adler GS. A profile of the Medicare Current Beneficiary Survey. Health Care Financing Review. 1994;15(4):153–163. [PMC free article] [PubMed] [Google Scholar]
- Anderson WL, Norton EC, Dow WH. Medicare maximization by state Medicaid programs: effects on Medicare home care utilization. Medical Care Research and Review. 2003;60(2):201–222. doi: 10.1177/1077558703060002004. [DOI] [PubMed] [Google Scholar]
- Anderson WL, Norton EC, Kenney GS. Effects of state Medicaid home care Medicare maximization programs on Medicare expenditures. Home Health Care Services Quarterly. 2003;22(3):19–40. doi: 10.1300/J027v22n03_02. [DOI] [PubMed] [Google Scholar]
- Balsa AI, Cao Z, McGuire TG. Does managed health care reduce health care disparities between minorities and Whites? Journal of Health Economics. 2007;26(1):101–121. doi: 10.1016/j.jhealeco.2006.06.001. [DOI] [PubMed] [Google Scholar]
- Briesacher BA, Stuart B, Ren X, Doshi JA, Wrobel MV. Medicare beneficiaries and the impact of gaining prescription drug coverage on inpatient and physician spending. Health Services Research. 2005;40(5, Pt. 1):1279–1296. doi: 10.1111/j.1475-6773.2005.00432.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cai L, Lubitz J. Was There Compression of Disability for Older Americans from 1992 to 2003? Demography. 2007;44(3):479–495. doi: 10.1353/dem.2007.0022. [DOI] [PubMed] [Google Scholar]
- Centers for Disease Control and Prevention. Influenza vaccination and self-reported reasons for not receiving influenza vaccination among Medicare beneficiaries aged >= years - United States, 1991-2002. MMWR Weekly. 2004 Retrieved May 18, 2010, from http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5343a3.htm. [PubMed]
- Centers for Medicare and Medicaid Services O. o. t. A. Section 5: Sample Design and Guidelines for Preparing Statistics. 2007.
- Chen JG, Fleischer AB, Jr., Smith ED, Kancler C, Goldman ND, Williford PM, Feldman SR. Cost of nonmelanoma skin cancer treatment in the United States. Dermatologic Surgery. 2001;27(12):1035–1038. doi: 10.1046/j.1524-4725.2001.01004.x. [DOI] [PubMed] [Google Scholar]
- Chin MH, Zhang JX, Rathouz PJ. Transitions in Health Status in Older Patients with Heart Failure. Southern Medical Journal. 2003;96(11):1096–1106. doi: 10.1097/01.SMJ.0000078680.44187.3D. [DOI] [PubMed] [Google Scholar]
- Craig BM, Kreling DH, Mott DA. Do seniors get the medicines prescribed for them? Evidence from the 1996-1999 Medicare Current Beneficiary Survey. Health Affairs (Project Hope) 2003;22(3):175–182. doi: 10.1377/hlthaff.22.3.175. [DOI] [PubMed] [Google Scholar]
- Crystal S, Sambamoorthi U, Walkup JT, Akincigil A. Diagnosis and treatment of depression in the elderly medicare population: predictors, disparities, and trends. Journal of the American Geriatrics Society. 2003;51(12):1718–1728. doi: 10.1046/j.1532-5415.2003.51555.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Doshi JA, Polsky D, Chang VW. Prevalence and trends in obesity among aged and disabled U.S. Medicare beneficiaries, 1997-2002. Health Affairs (Project Hope)s. 2007;26:1111–1117. doi: 10.1377/hlthaff.26.4.1111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ettner SL, Hermann RC, Tang H. Differences between generalists and mental health specialists in the psychiatric treatment of medicare beneficiaries. Health Services Research. 1999;34(3):737–760. [PMC free article] [PubMed] [Google Scholar]
- Ferraro D, Liu H. Uses of medicare current beneficiary survey for analysis across time. 2005.
- Fiscella K, Holt K. Impact of primary care patient visits on racial and ethnic disparities in preventive care in the United States. Journal of the American Board of Family Medicine. 2007;20(6):587–597. doi: 10.3122/jabfm.2007.06.070053. [DOI] [PubMed] [Google Scholar]
- Fiscella K, Holt K, Meldrum S, Franks P. Disparities in preventive procedures: comparisons of self-report and Medicare claims data. BMC Health Services Research. 2006;6(122) doi: 10.1186/1472-6963-6-122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. The implications of regional variations in Medicare spending. Part 2: health outcomes and satisfaction with care. Annals of Internal Medicine. 2003;138(4):288–298. doi: 10.7326/0003-4819-138-4-200302180-00007. [DOI] [PubMed] [Google Scholar]
- Foote SM, Hogan C. Disability profile and health care costs of medicare beneficiaries under age sixty-five. Health Affairs. 2001;20(6):242–253. doi: 10.1377/hlthaff.20.6.242. [DOI] [PubMed] [Google Scholar]
- Hancox JG, Neville JA, Chen J, Coldiron B, Williford PM, Feldman SR. Interpretation of dermatopathology specimens is within the standard of care of dermatology practice. Dermatologic Surgery. 2005;31(3):306–309. doi: 10.1111/j.1524-4725.2005.31079. [DOI] [PubMed] [Google Scholar]
- Hill J, Fillit H, Thomas SK, Chang SB. Functional impairment, healthcare costs and the prevalence of institutionalisation in patients with Alzheimer's disease and other dementias. PharmacoEconomics. 2006;24(3):265–280. doi: 10.2165/00019053-200624030-00006. [DOI] [PubMed] [Google Scholar]
- Holt K, Franks P, Meldrum S, Fiscella K. Mammography self-report and mammography claims - Racial, ethnic, and socioeconomic discrepancies among elderly women. Medical Care. 2006;44(6):513–518. doi: 10.1097/01.mlr.0000215884.81143.da. [DOI] [PubMed] [Google Scholar]
- Hoover DR, Crystal S, Kumar R, Sambamoorthi U, Cantor JC. Medical Expenditures during the Last Year of Life: Findings from the 1992-1996 Medicare Current Beneficiary Survey. Health Services Research. 2002;37(6):1625–1642. doi: 10.1111/1475-6773.01113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Housman TS, Williford PM, Feldman SR, Teuschler HV, Fleischer AB, Goldman ND, et al. Nonmelanoma skin cancer: An episode of care management approach. Dermatologic Surgery. 2003;29(7):700–711. doi: 10.1046/j.1524-4725.2003.29185.x. [DOI] [PubMed] [Google Scholar]
- Iezzoni LI, Davis RB, Soukup J, O'Day B. Physical and sensory functioning over time and satisfaction with care: the implications of getting better or getting worse. Health Services Research. 2004;39(6, Pt. 1):1635–1651. doi: 10.1111/j.1475-6773.2004.00310.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kamal-Bahl S, Stuart BC, Beers MH. National trends in and predictors of propoxyphene use in community-dwelling older adults. The American Journal of Geriatric Pharmacotherapy. 2005;3(3):186–195. doi: 10.1016/S1543-5946(05)80025-X. [DOI] [PubMed] [Google Scholar]
- Kemper P, Weaver F, Short PF, Shea D, Kang H. Meeting the need for personal care among the elderly: does Medicaid home care spending matter? Health Services Research. 2008;43(1, Pt. 2):344–362. doi: 10.1111/j.1475-6773.2007.00762.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Korn E, Graubard B. Analysis of Health Survey. New York: John Wiley &Sons, Inc.; 1999. [Google Scholar]
- Lakdawalla D, Goldman DP, Bhattacharya J, Hurd MD, Joyce GF, Panis CW. Forecasting the nursing home population. Medical Care. 2003;41(1):8–20. doi: 10.1097/00005650-200301000-00003. [DOI] [PubMed] [Google Scholar]
- Laschober MA, Kitchman M, Neuman P, Strabic AA. Trends in Medicare supplemental insurance and prescription drug coverage, 1996-1999. Health Affairs (Project Hope)(Suppl Web Exclusives) 2002:W127–138. doi: 10.1377/hlthaff.w2.127. [DOI] [PubMed] [Google Scholar]
- Lubitz J, Cai L, Kramarow E, Lentzner H. Health, life expectancy, and health care spending among the elderly. The New England Journal of Medicine. 2003;349(11):1048–1055. doi: 10.1056/NEJMsa020614. [DOI] [PubMed] [Google Scholar]
- Lundy J, Craig BM. The use of disease-modifying agents among multiple sclerosis patients enrolled in medicare from 1995 to 2002 and the impact of medicare part D: analysis of claims data from the medicare current beneficiary survey. Clinical Therapeutics. 2006;28(1):140–145. doi: 10.1016/j.clinthera.2006.01.003. [DOI] [PubMed] [Google Scholar]
- Madden JM, Graves AJ, Ross-Degnan D, Briesacher BA, Soumerai SB. Cost-related medication nonadherence after implementation of Medicare Part D, 2006-2007. Journal of the American Medical Association. 2009;302(16):1755–1756. doi: 10.1001/jama.2009.1516. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Madden JM, Graves AJ, Zhang F, Adams AS, Briesacher BA, Ross-Degnan D, et al. Cost-related medication nonadherence and spending on basic needs following implementation of Medicare Part D. Journal of the American Medical Association. 2008;299(16):1922–1928. doi: 10.1001/jama.299.16.1922. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McCormack LA, Gabel JR, Berkman ND, Whitmore H, Hutchison K, Anderson WL, et al. Retiree health insurance: recent trends and tomorrow's prospects. Health Care Financing Review. 2002;23(3):17–34. [PMC free article] [PubMed] [Google Scholar]
- McKnight R. Home care reimbursement, long-term care utilization, and health outcomes. Journal of Public Economics. 2006;90(1-2):293–323. doi: 10.1016/j.jpubeco.2004.11.006. [DOI] [Google Scholar]
- Mello MM, Stearns SC, Norton EC. Do Medicare HMOs still reduce health services use after controlling for selection bias? Health Economics. 2002;11(4):323–340. doi: 10.1002/hec.664. [DOI] [PubMed] [Google Scholar]
- Mello MM, Stearns SC, Norton EC, Ricketts TC., III Understanding biased selection in Medicare HMOs. Health Services Research. 2003;38(3):961–992. doi: 10.1111/1475-6773.00156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Murray LA, Eppig FJ. Insurance trends for the Medicare population, 1991-1999. Health Care Financing Review. 2002;23(4):9–15. [PMC free article] [PubMed] [Google Scholar]
- Noyes K, Liu H, Holloway R, Dick AW. Accuracy of Medicare claims data in identifying parkinsonism cases: Comparison with the Medicare Current Beneficiary Survey. Movement Disorders. 2007;22(4):509–514. doi: 10.1002/mds.21299. [DOI] [PubMed] [Google Scholar]
- Noyes K, Liu HS, Holloway RG. What is the risk of developing parkinsonism following neuroleptic use? Neurology. 2006;66(6):941–943. doi: 10.1212/01.wnl.0000203348.13337.89. [DOI] [PubMed] [Google Scholar]
- Noyes K, Liu HS, Li Y, Holloway R, Dick AW. Economic burden associated with Parkinson's disease on elderly medicare beneficiaries. Movement Disorders. 2006;21(3):362–372. doi: 10.1002/mds.20727. [DOI] [PubMed] [Google Scholar]
- O'Connell J, Chu A, Bailey R. Considerations for analysis of the Medicare Current Beneficiary Survey (MCBS) across time. Proceedings of the Survey Research Methods Section of the American Statistical Association. 1997:645–650. [Google Scholar]
- O'Malley AS, Forrest CB. Immunization disparities in older Americans - Determinants and future research needs. American Journal of Preventive Medicine. 2006;31(2):150–158. doi: 10.1016/j.amepre.2006.03.021. [DOI] [PubMed] [Google Scholar]
- Patel UD, Davis MM. Falling into the doughnut hole: drug spending among beneficiaries with end-stage renal disease under Medicare Part D plans. Journal of the American Society of Nephrology. 2006;17(9):2546–2553. doi: 10.1681/ASN.2005121385. [DOI] [PubMed] [Google Scholar]
- Porell FW, Miltiades HB. Access to care and functional status change among aged Medicare beneficiaries. The Journals of Gerontology Series B, Psychological Sciences and Social Sciences. 2001a;56(2):S69–83. doi: 10.1093/geronb/56.2.S69. [DOI] [PubMed] [Google Scholar]
- Porell FW, Miltiades HB. Disability outcomes of older Medicare HMO enrollees and fee-for-service Medicare beneficiaries. Journal of the American Geriatrics Society. 2001b;49(5):615–631. doi: 10.1046/j.1532-5415.2001.49123.x. [DOI] [PubMed] [Google Scholar]
- Rice T, Snyder RE, Kominski G, Pourat N. Who switches from Medigap to Medicare HMOs? Health Services Research. 2002;37(2):273–290. doi: 10.1111/1475-6773.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Riley G, Zarabozo C. Trends in the health status of medicare risk contract enrollees. Health Care Financing Review. 2006;28(2):81–95. [PMC free article] [PubMed] [Google Scholar]
- Riley GF, Lubitz JD, Zhang N. Patterns of health care and disability for Medicare beneficiaries under 65. Inquiry: A Journal of Medicalcare Organization, Provision and Financing. 2003;40(1):71–83. doi: 10.5034/inquiryjrnl_40.1.71. [DOI] [PubMed] [Google Scholar]
- Rosenbach ML, Acamache KW, Khandker RK. Variations in Medicare access and satisfaction by health status: 1991-93. Health Care Financing Review. 1995;17(2):29–49. [PMC free article] [PubMed] [Google Scholar]
- Sambamoorthi U, Olfson M, Walkup JT, Crystal S. Diffusion of new generation antidepressant treatment among elderly diagnosed with depression. Medical Care. 2003;41(1):180–194. doi: 10.1097/00005650-200301000-00019. [DOI] [PubMed] [Google Scholar]
- Sarndal C, Swensson BJW. Model Assisted Survey Sampling. New York: Springer Verlag; 2003. [Google Scholar]
- Shang B, Goldman DP. Prescription Drug Coverage and Elderly Medicare Spending. National Bureau of Economic Research Working Paper Series, No. 13358. 2007.
- Simoni-Wastila L, Zuckerman IH, Shaffer T, Blanchette CM, Stuart B. Drug use patterns in severely mentally ill medicare beneficiaries: Impact of discontinuities in drug coverage. Health Services Research. 2008;43(2):496–514. doi: 10.1111/j.1475-6773.2007.00779.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stein JD, Sloan FA, Lee PP. Rates of glaucoma medication utilization among older adults with suspected glaucoma, 1992 to 2002. American Journal of Ophthalmology. 2007;143(5):870–872. doi: 10.1016/j.ajo.2006.12.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stuart B, Kamal-Bahl S, Briesacher B, Lee E, Doshi J, Zuckerman IH, et al. Trends in the prescription of inappropriate drugs for the elderly between 1995 and 1999. The American Journal of Geriatric Pharmacotherapy. 2003;1(2):61–74. doi: 10.1016/S1543-5946(03)90002-X. [DOI] [PubMed] [Google Scholar]
- Stuart B, Simoni-Wastila L, Chauncey D. Assessing the impact of coverage gaps in the Medicare Part D drug benefit. Health Affairs. 2005;24(3):W5-167–W5-179. doi: 10.1377/hlthaff.w5.167. [DOI] [PubMed] [Google Scholar]
- Tosteson AN, Gottlieb DJ, Radley DC, Fisher ES, Melton LJ., III Excess mortality following hip fracture: the role of underlying health status. Osteoporosis International. 2007;18(11):1463–1472. doi: 10.1007/s00198-007-0429-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Waidmann TA, Liu KB. Disability trends among elderly persons and implications for the future. The Journals of Gerontology Series B, Psychological Sciences and Social Sciences. 2000;55(5):S298–S307. doi: 10.1093/geronb/55.5.S298. [DOI] [PubMed] [Google Scholar]
- Wei W, Sambamoorthi U, Olfson M, Walkup JT, Crystal S. Use of psychotherapy for depression in older adults. The American Journal of Psychiatry. 2005;162(4):711–717. doi: 10.1176/appi.ajp.162.4.711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yang Z, Gilleskie DB, Norton EC. Prescription Drugs, Medical Care, and Health Outcomes: A Model of Elderly Health Dynamics. National Bureau of Economic Research Working Paper Series, No. 10964. 2004.
- Yang Z, Norton EC, Stearns SC. Longevity and health care expenditures: The real reasons older people spend more. The Journals of Gerontology Series B, Psychological Sciences and Social Sciences. 2003;58(1):S2–S10. doi: 10.1093/geronb/58.1.S2. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.