Losses to Follow-up in a Nationally Representative Study of Community-Living Older Americans: Two Approaches to Censoring and Implications for Mortality Outcomes

Abstract

Background:

In the National Health and Aging Trends Study (NHATS), use of its Sensitive files leads to incomplete ascertainment of mortality, largely because of losses to follow-up. To account for these losses, we compared two censoring approaches for evaluating mortality.

Methods:

In a hybrid approach, most participants were censored at the time of last contact, while the remainder were censored at the time of last completed interview. In the other approach, all participants were censored at the time of last completed interview (LCI). All-cause mortality was evaluated among 7,608 and 7,498 community-living members of the 2011 (over 10 years) and 2015 (over 5 years) cohorts based on the Sensitive files, with linked Medicare data serving as the reference standard.

Results:

Using the hybrid and LCI approaches, the median (IQR) follow-up times among nondecedents were 7.7 (5.9–8.7) and 8.8 (6.3–9.9) years shorter for the Sensitive files than Medicare data in the 2011 cohort. The corresponding values in the 2015 cohort were 2.9 (1.8–3.8) and 4.0 (2.2–4.9) years. For both cohorts, cumulative mortality based on the Sensitive files relative to the Medicare data was modestly lower for the hybrid approach but comparable for the LCI approach. The incidence rate ratios (95% CI) for the Sensitive files relative to the Medicare data in the 2011 and 2015 cohorts, respectively, were 0.89 (0.87, 0.91) and 0.89 (0.86, 0.93) for the hybrid approach and 0.98 (0.95, 1.00) and 0.97 (0.94, 1.00) for the LCI approach. However, the numbers of participants at risk for death over time were considerably smaller for the latter than former approach.

Conclusions:

When evaluating mortality using the NHATS Sensitive files, investigators should consider censoring all participants who are lost to follow-up at the time of last completed interview, recognizing potential trade-offs in terms of reductions in power and precision.

INTRODUCTION

Complete and accurate ascertainment of mortality is needed to minimize potential biases, especially in longitudinal studies of older persons who generally have a much higher risk of death than their younger counterparts. In a recently published report,¹ we found that use of the Sensitive Demographic files in the National Health and Aging Trends Study (NHATS) leads to incomplete ascertainment and, to a lesser degree, misclassification of mortality. In our analysis, most participants who were lost to follow-up were censored at time of last contact based on the NHATS tracker file, which provides disposition codes for each annual interview. We subsequently discovered that some participants who were censored using this approach had died before the last contact based on linked Medicare data. An alternative approach, which should not be susceptible to this type of error, is to censor all such participants at time of last completed interview. In the current report, we compared these two censoring approaches with the goal of providing transparent, evidence-based recommendations for evaluating mortality using the NHATS Sensitive files.

METHODS

Study Populations

NHATS is a nationally-representative longitudinal study of Medicare beneficiaries that includes annual interviews.² In September 2010, NHATS drew a random sample of persons 65 or older living in the contiguous US from the Medicare enrollment file, with oversampling of those 90 or older and non-Hispanic Blacks.² Round 1 assessments, completed in 2011, yielded a sample of 8,245 persons. In Round 5 (2015), a new sample of beneficiaries, 65 or older as of September 2014, was recruited using similar procedures to replenish the cohort.³ These participants, together with survivors from the 2011 cohort, form the 2015 cohort (n=8,334). The two cohorts have been linked to Medicare data.²

The current analysis was restricted to participants who were living in the community. This included those in assisted living facilities and other residential care settings that were not nursing homes, as in prior studies.^4–7 Of these, one participant had no Medicare data after October 2011, leaving 7,608 (2011) and 7,498 (2015) participants. For each of the two cohorts, baseline survey weights and design variables can be used to generate population-based estimates of Medicare beneficiaries 65 or older. As previously described,¹ age, sex, and self-reported race and ethnicity were comparable between the two cohorts.

The Johns Hopkins IRB approved the NHATS protocol, and all participants provided written informed consent. Use of NHATS data was approved by the Yale IRB.

Mortality and Censoring Approaches

All-cause mortality was ascertained through the NHATS Sensitive Demographic files (i.e., Sensitive files), and linked Fee-for-Service and Managed Medicare data. Since they are essentially 100% complete, Medicare data served as the reference standard. When an NHATS participant is confirmed as deceased, a last month of life interview is completed with a proxy,² and year and month of death are ascertained. In 49 cases, year and/or month were imputed as described in Supplementary Text S1. Because complete details about these procedures have been previously described,¹ we focus herein on the two approaches for censoring participants who were lost to follow-up. In a hybrid approach, which was used in our earlier report,¹ time of last contact was used for disposition codes that suggest that participants were still alive; otherwise, time of last completed interview was used. In the second approach, all participants were censored at time of last completed interview. The last contact and last completed interview were determined, respectively, from the NHATS tracker files, which provide disposition codes for each annual interview, as shown in Supplementary Table S1.

The NHATS tracker files are available for public use, while access to the Sensitive files requires an approved application and Sensitive Data Agreement⁸ and to the restricted Medicare data requires an approved application, Data Use Agreement, and data protection plan.⁹

Statistical Analysis

Unless otherwise stated, all results are reported using the baseline survey weights for each cohort.¹⁰ The analyses for the two cohorts were similar, although follow-up was 10 years for the 2011 cohort and 5 years for the 2015 cohort. We identified participants who were classified as deceased based on Medicare data and as lost to follow-up based on the Sensitive and tracker files. Among decedents who were censored at time of last contact, we determined the unweighted number (%) who had already died based on Medicare data.

Among censored participants who were classified as alive by both Medicare data and Sensitive files, weighted differences in follow-up times were calculated. The duration of follow-up time was determined from time of enrollment to loss to follow-up for Sensitive files or end of follow-up period for Medicare data. These differences were calculated for each censoring approach.

Cumulative mortality curves were generated over 10 years for the 2011 cohort and 5 years for the 2015 cohort based on data from the Medicare and Sensitive files.^11,12 For the Sensitive files, the annual analytic weights were used instead of baseline weights to account for losses to follow-up, which affect the number of participants at risk over time.¹³ We calculated mortality incidence rates using Poisson models and incidence rate ratios, denoting the incidence rate based on the Sensitive files divided by that based on Medicare data. Participants who were lost to follow-up using the Sensitive files were censored using each of the two approaches.

All analyses were performed using SAS version 9.4 and R version 4.4.1.

RESULTS

In the 2011 cohort, 1479 decedents were lost to follow-up over 10 years. Of the 1306 (88.3%) decedents who were censored at time of last contact, 17 (1.3%) had already died an average (SD) of 6.7 (4.1) months earlier. Of these, 15 had disposition codes of 77 (refused by participant) or 87 (refused by proxy). In the 2015 cohort, 431 decedents were lost to follow-up over 5 years. Of the 328 (76.1%) decedents who were censored at time of last contact, 16 (4.9%) had already died an average (SD) of 7.6 (4.6) months earlier. Of these, 15 had disposition codes of 77, 87, or 85 (refused by facility).

Figure 1 provides the weighted differences in follow-up times between the Sensitive files and Medicare data for nondecedents according to whether participants were censored based on the hybrid or last completed interview approach. For both approaches, large differences in follow-up time were observed, but the differences were greater for last completed interview. For the 2011 cohort, the median (IQR) differences were 7.7 (5.9–8.7) and 8.8 (6.3–9.9) years shorter for the Sensitive files than Medicare data for the hybrid and last completed interview approaches. The corresponding values for the 2015 cohort were 2.9 (1.8–3.8) and 4.0 (2.2–4.9) years.

Figure 1.

Weighted Differences in Follow-up Times Between the Sensitive Files and Medicare Data for Nondecedents According to Censoring Approach. Negative values indicate that follow-up times were shorter for the Sensitive Files than Medicare Data. As described in the Methods, participants were censored using two approaches: hybrid and last completed interview. The numbers of nondecedents for these approaches were 1702 and 1803 for the 2011 cohort and 1275 and 1511 for the 2015 cohort. For each box plot, the left border, midline, and right border represent the 75th percentile, median, and 25th percentile values, based on the magnitude of the negative values, while the red diamond represents the mean value. The lines from the boxes extend to the 95th and 5^th percentiles, respectively. Follow-up time was determined from time of enrollment to loss to follow-up, i.e. censored, for Sensitive files or to end of the follow-up period (10 years for 2011 cohort and 5 years for 2015 cohort) for Medicare data.

Cumulative mortality curves for the 2011 and 2015 cohorts are provided in Figure 2. For each cohort, the curve based on the hybrid approach differed modestly from that based on the Medicare data, diverging lower around year 5 for the 2011 cohort and year 3 for the 2015 cohort. When the last completed interview approach was evaluated, these differences were delayed and less striking, occurring after year 7 for the 2011 cohort and shortly before year 4 for the 2015 cohort. For the 2011 cohort, the values for weighted cumulative mortality (95% CI) at 10 years were 44.3% (43.2%−45.4%) based on Medicare data and 41.3% (39.9%−42.6%) and 43.0% (41.5%−44.3%) for the Sensitive files, using the hybrid and last completed interview approaches. For the 2015 cohort, the corresponding values at 5 years were 20.7% (19.8%−21.6%), 19.3% (18.2%−20.4%), and 20.4% (19.3%−21.5%). For both cohorts, the unweighted number of at-risk participants decreased more rapidly for the Sensitive files than Medicare data, reflecting losses to follow-up. These reductions were more pronounced for the last completed interview than hybrid approach.

Figure 2.

Cumulative Mortality over 10 Years for the 2011 Cohort and 5 Years for the 2015 Cohort. Standard error bands accompany each of the curves, which were generated using the Aalen (hazard-based) estimator implemented in the R survey package. For the Sensitive files, participants who were lost to follow-up were censored using two approaches—hybrid and last completed interview, as described in the Methods. Values are based on the baseline survey weights for the Medicare data and annual analytic weights for the Sensitive files. Unweighted values are provided for the number of participants at risk.

Table 1 provides the mortality incidence rates, based on the number of deaths and person-years of follow-up, for the two cohorts. For each, rates for the Sensitive files were modestly lower when participants were censored using the hybrid than last completed interview approach; and the latter values were comparable to those for the Medicare data, despite large differences in the number of deaths and person-years of follow-up. For the 2011 and 2015 cohorts, the incidence rate ratios (95% CI) for the Sensitive files relative to Medicare data were 0.89 (0.87–0.91) and 0.89 (0.86–0.93) for the hybrid approach and 0.98 (0.95–1.00) and 0.97 (0.94–1.00) for the last completed interview approach.

Table 1.

Mortality Incidence Rates for Community-living Participants in the 2011 and 2015 Cohorts According to Data Source and Censoring Approach ^a

Data Source	2011 Cohort			2015 Cohort
	Number of deaths	Person-years of follow-up	Rates per 100-Person Years (95% CI)	Number of deaths	Person-years of follow-up	Rates per 100-Person Years (95% CI)
Medicare data	15,670,118	277,748,259	5.64 (5.45–5.84)	8,447,402	18,387,2185	4.59 (4.38–4.82)
Sensitive files, censoring approach
Hybrid b	9,682,036	193,051,231	5.02 (4.82–5.22)	6,426,255	156,26,1842	4.11 (3.88–4.36)
Last completed interview	9,682,036	175,552,164	5.52 (5.30–5.74)	6,424,255	144,180,156	4.46 (4.20–4.73)

Abbreviations: CI, confidence interval

^aValues are based on the baseline survey weights for the Medicare data and Sensitive files. The duration of follow-up was 10 years for the 2010 cohort and 5 years for the 2015 cohort. For the Sensitive files, participants who were lost to follow-up for reasons other than death were censored using two approaches described in the Methods. The mortality rates and 95% CIs were estimated using Poisson models including the logarithm of follow-up time as an offset, and survey weights and design were then incorporated via the R survey package.

^bMost participants were censored at time of last contact, while the remainder were censored at time of last completed interview, based on disposition codes of the tracker file for each annual interview, as shown in Supplementary Table S1.

DISCUSSION

Using Medicare data as the reference standard for mortality, we compared two approaches for censoring participants who were lost to follow-up in NHATS with the goal of informing decisions about how best to evaluate mortality using the Sensitive files. Four major findings warrant comment. First, a sizeable number, albeit small minority of decedents who were censored at time of last contact had already died about 7 months earlier. Most were coded as refusals by the participant, proxy or facility. Second, among nondecedents who were censored, differences in follow-up times between the Sensitive files and Medicare data were greater for the last completed interview than hybrid approach. Third, cumulative mortality and mortality incidence rates based on the Sensitive files, relative to Medicare data, were modestly lower for the hybrid approach but comparable for the last completed interview approach. Fourth, the numbers of participants at risk for death were smaller when participants were censored using the time of last completed interview than hybrid approach. Collectively, these findings, which were observed in both the 2011 and 2015 cohorts, suggest that censoring all participants who were lost to follow-up at time of last completed interview should be the preferred approach for evaluating mortality when the NHATS Sensitive files are used, albeit with possible reductions in power and precision.

In longitudinal studies of older persons, valid approaches for evaluating mortality are needed. The Sensitive files are the most commonly used data source for ascertaining mortality in NHATS,¹ but no guidance has been provided about the preferred approach to censoring participants who were lost to follow-up, despite attrition rates that are not insubstantial.² Based on our review of the literature, moreover, censoring approaches have not been described in relevant NHATS publications,^4,14–29 with only one possible exception.³⁰ In an earlier study,¹ we found that use of the Sensitive files, relative to Medicare data, leads to incomplete ascertainment of mortality. To account for losses to follow-up, we used a hybrid approach that censored participants primarily at time of last contact based on disposition codes from the NHATS tracker files under the supposition that these codes were accurate. As demonstrated in the current study, however, some participants who were censored using this approach had died before the last contact based on Medicare data.

In addition to these misclassified codes, we found that mortality rates were underestimated when participants were censored based on the hybrid approach. The primary reason for these underestimates is that participants who were censored at time of last contact had an additional year of follow-up than those who were censored at time of last completed interview, thereby increasing the denominator for calculation of mortality rates, but were not included in the numerator since they did not appear in the Sensitive files as decedents. In the current analysis, 87.5% and 74.9% of the decedents in the 2011 and 2015 cohorts who were lost to follow-up were censored at time of last contact versus time of last completed interview using the hybrid approach. Another reason is that some participants who were censored at time of last contact had already died, meaning that they were included in the numerator when calculating mortality rates using Medicare data but not Sensitive files.

Because censoring all participants at time of last completed interview does not lead to misclassification or to spurious reductions in mortality rates, this approach is preferable to the hybrid approach in terms of validity, but it could lead to reductions in power and precision. This is due to the shorter duration of follow-up, as evidenced by larger differences in follow-up times between the Sensitive files and Medicare data, smaller numbers of participants at risk for death over time, and lower numbers for person-years of follow-up. In our earlier report,¹ we raised concerns that use of the Sensitive files to ascertain mortality could lead to reductions in power and precision due to non-mortality attrition, a problem that might be worsened if participants are censored at time of last completed interview. Nonetheless, a potential advantage of the latter approach is that the annual analytic weights, which account for losses to follow-up, can be used until an observation is censored, since they are available only for completed interviews.¹³ We have shown, however, that cumulative mortality curves for the Sensitive files are comparable regardless of whether the baseline or annual analytic weights are used.1

Underestimation and misclassification of mortality can have important public health consequences. They could mask true mortality burdens, obscure health disparities, and lead to under-allocation of resources to high-risk groups. Older persons are especially affected, as mortality data guide aging-related policies, service provision, and program design. Because mortality statistics inform planning for health care capacity, systematic bias in ascertainment can compromise policy and programmatic decisions. Our findings highlight that the censoring approach can affect mortality estimates, underscoring the need for methods that optimize completeness and accuracy of mortality ascertainment.

Given the limitations of the Sensitive files, we have previously suggested that mortality could potentially be ascertained centrally by NHATS through linkages with the National Death Index (NDI),¹ as in the Health and Retirement Study (HRS),³¹ another study sponsored by the National Institute on Aging. These more complete mortality data, which include date and cause of death, could be made available to approved investigators via the NHATS Restricted Data Repository.³² If feasible, this improved approach would permit mortality to be more completely and accurately ascertained and, in turn, increase the value of NHATS to the scientific community.

An important strength of the current study is the consistency of results in two cohorts over two different follow-up periods. Given the unique design of NHATS, however, our findings may not be directly applicable to other large longitudinal studies.

The current study and our prior report¹ provide transparent and evidence-based operational details on how mortality can best be ascertained and evaluated using the Sensitive files and tracker files, information that is not available in prior NHATS publications or documentation. Given the importance of mortality as an outcome in longitudinal studies of older persons and the nearly exponential increase in number of NHATS publications over the past 12 years,³³ these details should help to guide investigators in future studies.

In summary, when evaluating mortality using the NHATS Sensitive files, investigators should consider censoring all participants who are lost to follow-up at time of last completed interview, recognizing potential trade-offs in terms of reductions in power and precision.

Supplementary Material

Supplementary Material

Supplementary Text S1. Imputation of Missing Month and Year of Death in Sensitive Files

Supplementary Table S1. Disposition Codes in Tracker File for National Health and Aging Trends Study

Key Points.

• Among nondecedents who were censored, differences in follow-up times between the Sensitive files and Medicare data were greater for the last completed interview than hybrid approach.

• Cumulative mortality and mortality incidence rates based on the Sensitive files, relative to the Medicare data, were lower for the hybrid approach but comparable for the last completed interview approach.

• When evaluating mortality using the NHATS Sensitive files, investigators should consider censoring all participants who are lost to follow-up at the time of the last completed interview.

Why does this matter.

In longitudinal studies of older persons, valid approaches for evaluating mortality are needed.

The current study provides transparent and evidence-based operational details on how mortality can best be evaluated using the NHATS Sensitive files.