Assessment of the Social Security Administratino Death Master File for Comparative Analysis Studies of Peripheral Vascular Devices

Abstract

Objectives:

The objective of this study was to assess the reliability the Social Security Administration Death Master File (SSADMF) for evaluating mortality in comparative peripheral vascular device studies.

Methods:

We leveraged two versions of an administrative claims data set that were identical except for the source of mortality data. The SSADMF was the primary source of mortality records in one version. The SSDMF was combined with mortality from Medicare beneficiary records in the other. Our study was set in the context of a comparative effectiveness analysis of recent FDA interest involving peripheral paclitaxel-coated devices. Mortality of patients with Medicare Advantage insurance coverage from 2015 through 2018 who underwent femoropopliteal artery revascularization with a drug-coated device (DCD) or non-drug-coated device (NDCD) was assessed through 2019. Covariate differences between treatment groups were adjusted by inverse propensity treatment weighting. The hazard ratio (HR) of DCD to NDCD mortality was estimated using Cox regressions.

Results:

The cumulative incidences of mortality differed substantially between versions of the data. However, we could not reject the null hypothesis that the HRs of the SSADMF, 1.05 (95% CI .95 – 1.17), and the MBSF/SSADMF 1.03 (95% CI .96 – 1.11), were the same (P=.63).

Conclusions:

The SSADMF is a common source of mortality records in the U.S. that can be linked to real-world data (RWD) sources but is known to under report mortality rates. We find that the SSADMF provides a reliable source of all-cause mortality for a comparative study of peripheral vascular device all-cause mortality.

Introduction

Mortality is a commonly reported endpoint for comparative analyses of therapies in health care research, including observational research employing real-world data (RWD). Multiple sources are available to ascertain death data in the US, yet the Social Security Administration Death Masterfile (SSADMF) is commonly used due to its accessibility and affordability.¹ Although the SSA uses multiple sources to identify deaths including family members, funeral homes, financial institutions, postal authorities, States and other Federal agencies, the data are not a comprehensive source of deaths.^2–4 Under reporting in the SSADMF became more prominent in 2011 when the SSA stopped including data provided by state agencies in the SSADMF.^5,6 Furthermore, the underestimation of mortality rates by the SSADMF has been shown to vary geographically.⁵ Even with these substantial limitations, the SSADMF continues to be used to measure mortality in a variety of comparative effectiveness studies, either alone or supplementing other sources.^7–14

Given the critical need to evaluate mortality as an endpoint for many health care interventions, it is important to maximize the potential of all available data sources. Although it is known the SSAMDF is not a suitable source for calculating mortality rates, it is unknown whether and to what extent analyses designed to evaluate for differences in mortality risk between treatment groups are impacted by the incompleteness of the SSADMF. If the reporting (or non-reporting) of death is uncorrelated with treatment exposure across states and overtime, the SSAMDF may provide a useful source of mortality data for comparative research.

The objective of this study was to assess the reliability the SSADMF for evaluating mortality in comparative treatment studies of peripheral vascular devices. We leveraged two versions of the Optum CDM® administrative claims data that differed in their mortality data sources but were otherwise identical to empirically to test the utility of the SSAMDF for comparative research. Moreover, we performed our assessment in the context of a contemporary comparative effectiveness analysis of high public health importance to demonstrate the relevance of this topic to policy and practice. In 2019 the US Food and Drug Administration (FDA) issued a warning related to the use of drug-coated devices (DCDs) – stents and balloons – used for peripheral endovascular intervention (PVI) because of a possible association with increased mortality.¹⁵ The FDA has deemed a randomized trial to be infeasible, necessitating the need for RWD based approaches to long term surveillance.^16,17 The Vascular Quality Initiative (VQI) maintains registries that are a valuable source of RWD but they require linking to a mortality data source to assess long term mortality.¹⁸ VQI has published the results of one study that linked to the SSADMF and found no mortality effect.¹⁹ However, the significance of a potential mortality signal necessitates the ability to evaluate the safety of DCDs over the long term using RWD data sources like the VQI Registry. As the impact of mortality underreporting in the SSADMF in comparative evaluations is unknown, these results have implications for continued safety monitoring for a wide range of health care interventions.

Methods

Our analysis uses 2 versions of a dataset that only differed in their mortality source data. The Optum CDM®, a nationwide de-identified sample of administrative health insurance data that includes medical and prescription claims and enrolment records for individuals with employer-sponsored (“commercial”) and Medicare Advantage (MA) insurance plans that is updated quarterly. The first quarter 2019 Optum CDM® included mortality records originating from both the Centers for Medicare and Medicaid Services (CMS) Master Beneficiary Summary File (MBSF), which has been shown to be nearly complete, as well as the SSADMF.²⁰ Thus, there was a substantial improvement in mortality record capture among Medicare Advantage plan enrollees compared to the fourth quarter 2018 Optum CDM®, which used the SSADMF but not the MBSF.

Our analysis sample included all patients in the Optum CDM® data insured by an MA plan who underwent femoropopliteal artery PVI with either a DCD or non-drug-coated device (NDCD) from 4/1/2015 through 12/31/2017 with at least 90 days enrollment prior to the procedure. Additional details of the dataset and determination of treatment assignments have been previously published.²¹ In brief, the Optum CDM® is a claims database that includes demographics, diagnosis codes, procedure codes, place of service codes, and prescription drug claims codes, as well as dates of service. Baseline comorbidities and medication usage were identified using diagnosis and NDC codes that were present anytime in the 90 days prior to an individual’s index procedure. The outcome of interest was all-cause mortality through 12/31/2018. Mortality was ascertained from two versions of the Optum CDM®. For the purposes of our analysis, only the mortality data differed; the observation period, analysis population, and patient characteristics were identical between data versions.

Logistic regression was used to calculate the propensity of receiving a DCD, the treatment of interest, using all patient, procedural, and hospital characteristics. The comparator was NDCDs. Inverse probability of treatment weighting (IPTW) was used to balance differences in characteristics between treatment groups. Propensity score estimation and IPTW adjustment methods are independent of outcomes. A single IPTW was calculated for the analysis population and applied to both versions of the data. The covariates included in the propensity score analysis and the covariate balance after adjustment is shown in Appendix Table A1. A standardized mean difference of <0.10 for between group comparisons was considered well-balanced.

Cox regression was used to estimate the all-cause mortality hazard ratio (HR) of DCD versus NDCD. We used two approaches to test for differences in HR estimates between versions of the data. First, we compared, HR estimates from separate Cox regression models for each mortality source. Specifically, we tested the null hypothesis that the HR estimates are equal. These calculations were performed using the seemingly unrelated estimation analytics functionality in Stata 15.1, which is commonly applied to testing intra-model hypotheses. Cluster robust standard errors were calculated by patient in the test statistic calculation.

In our second approach, the two versions of the data were combined by appending one to the other, resulting in a data file containing two observations per patient with different sources of mortality data. An indicator variable was also added to the combined data that identified observations with the MBSF/SSADMF mortality source. We estimated a Cox regression, which included a treatment indicator, a mortality data source indicator, and an interaction of treatment and data source. Robust standard errors were clustered by patient. The coefficient estimate of the interaction term is the parameter of interest. This is the HR of DCD treated patients from the complete mortality data versus all NDCD patient observations. The treatment indicator provides the HR estimate of DCD vs NDCD using all observations analogous to a standard Cox model. We then evaluated the null hypothesis that the treatment HR and the interaction HR were equal.

In addition to an alternate specification of the main analysis, we evaluated the robustness of our results given the documented geographic and temporal variation in mortality in the SSADMF. We estimated a series of multivariate Cox models for each version of the mortality data by US Census Region. Because geography was included in the propensity score model, the IPTW adjusted sample could not be analyzed by subgroups. However, because the covariates were overwhelmingly balanced between groups prior to IPTW, a multivariate Cox regression was sufficient for controlling for differences between the NDCD and DCD populations within Census Divisions. We also estimated multivariate Cox models for subgroups based on index year allowing for a maximum of 2 years follow-up from the start of the index year. This allowed for testing 3 subgroups with comparable follow-up times. We employed the same intra-model statistical methods as in our main analysis to test the null hypothesis that the HR estimates are equal between mortality data sources.

Results

In total, 16,796 MA patients were included in the analysis – 12,369 (73.6%) were treated with a NDCD and 4,427 (26.4%) were treated with a DCD. Median follow-up was 2.0 years (IQR: 1.4-2.8 years) in the SSADMF data and 1.8 years (IQR: 1.3-2.6 years) in the MBSF/SSADMF data. The average age was 73.3 years, 44.4% were female, 50.8% had critical limb ischemia, and 63.1% of procedures were performed in an outpatient location. Prior to IPTW adjustment, the population treated with DCDs were more often diagnosed with intermittent claudication, treated with a P2Y12 inhibitor prior to index and adjunctive atherectomy, and were more likely to undergo revascularization in the East North Central, Mid Atlantic, and South Atlantic U.S. Census Regions. After weighting, the data was well balanced with SMD of less than 0.1 across all covariates. Appendix Table A1 presents the covariate distribution pre and post IPTW adjustment.

Consistent with previous assessments of the SSADMF, overall mortality was substantially lower using the SSADMF data compared to mortality rates calculated using MBSF/SSADMF. Appendix Table A2 displays death rates overall and by U.S. Census Divisions. Among those treated with NDCDs and DCDs, the overall mortality rates were 12.9% and 12.5%, respectively using the SSADMF linked data, whereas they were 26.4% and 24.5% when linked to the MBSF/SSADMF. Thus, 50.7% of total deaths in our analysis were not captured in the data linked to SSADMF.

There were also notable differences in mortality captured by Census Division between data sources. The largest proportion of deaths occurred in the Pacific Division for both NDCD and DCD (32.6% and 31.5%, respectively based on MBSF/SSADMF data) and more than half of total deaths were not accounted for when using only the SSADMF (60.5% and 52.1% of deaths were missing, respectively). Conversely, the East South Central Division had the lowest NDCD and DCD mortality (22.3% and 21.0%, respectively) based on the MBSF/SSADMF with comparatively low rates of missing deaths relative to the SSADMF (32.8% and 36.2%, respectively). However, there was not an association between the use of DCDs and unreported deaths across Census Divisions. For example, the lowest rates of DCD use were in the Mountain (18.0%) and West North Central (17.9%) but these areas did not consistently have the highest or lowest rates of under reported deaths. The Mountain division rate of under reporting, 60.8%, was highest, but the West North Central rate, 55.2%, was only fourth highest.

Despite variation in and geography and under reporting of deaths in the SSADMF, IPTW adjusted HR estimates of mortality did not differ by mortality data source. Table 1 shows the results of the two specifications of our main analysis. Using the first approach, the HRs from the SSADMF and MBSF/SSADMF versions of the data are 1.05 (95% CI 0.95 – 1.17, P=.33) and 1.03 (95% CI 0.96 – 1.11, P=.40), respectively. The inter-model test of equivalent HRs produced a chi-square statistic (df 1) of 0.23 (P=.63). A value <0.05 was considered significant and therefore we could not reject the null hypothesis that the HRs estimates from the two models are the same. IPTW adjusted cumulative mortality by treatment and mortality data source are displayed in Appendix Figure A1. Using the second approach, the overall effect estimate, the HR of the death for DCDs versus NDCDs, was 1.05 (95% CI 0.95 – 1.17, P=0.27) and the interaction term estimate was 0.97 (95% CI 0.89-1.05, P=0.47). A test of the hypothesis that the treatment effect and interaction term were equal had a chi-square statistic (df 1) of 1.01 with Pr>Chi-2 = 0.314. Again, the null hypothesis could not be rejected.

Table 1.

Cox regression results by regression specification

	Intra-model test specification*		Pooled data within model test specification**
	SSADMF	MBSF/SSADMF
Hazard RatioDCD	1.05	1.03	1.06
95% CI	0.95 – 1.17	0.96 – 1.11	0.96 – 1.17
P=	.33	.40	.27
INDICATORMBSF/SSADMF	-	-	2.20
95% CI	-	-	2.12 – 2.29
P=	-	-	.000
Hazard RatioDCD
x	-	-	.97
INDICATORMBSF/SSADMF	-	-	.89-1.05
95% CI	-	-	.47
P=
Hypothesis test	HRSSADMF – HRMBSF/SSADMF = 0		HRDCD – (HRDCD x INDICATORMBSF/SSADMF) = 0
	Chi-2(1) = .23		Chi-2(1) = 1.01
	P>Chi-2 = .63		P>Chi-2 = .31

CI: confidence interval; DCD: Drug Coated Device; HR: hazard ratio; MBSF: Medicare Beneficiary Summary File; SSADMF: Social Security Administration Death Master File.

^*Intra-model hypothesis test calculations were performed using seemingly unrelated regression method with Robust standard error calculations were clustered by patient

^**Robust standard error calculations were clustered by patient

The multivariate Cox regression HR estimates and the 95% confidence intervals for the association between DCDs and mortality by Census Division and mortality data source are shown in Figure 1. (Appendix Table A3 shows the IPTW adjustment and a multivariate Cox regressions HRs from the full sample for both mortality data sources, which demonstrates the similarity in results between adjustment methods.) When evaluated by geographic region, the HRs of death for DCDs versus NDCDs did not differ substantially by mortality data source, except in the West North Central division. There the HR for mortality associated with DCDs was statistically significant with the SSADMF, but not with the SSADMF/MBSF. Numerous publications have demonstrated there is not a mortality signal related to DCD use.^{19, 21–23} Thus, the result may be due to a random sampling error or systemic differences in the use of DCD in these states. This division has the smallest sample size and the one of the largest discrepancies in NDCD and DCD mortality capture as shown in Appendix Table A2.

Figure 1.

Comparison of multivariate Cox regression hazard ratio estimates of mortality by U.S. Census Division

The results of the temporal variation robustness analysis were also consistent with our main analysis. The HRs and intra-model test statistics are shown in Appendix Table A4. For each index year-subgroup we were not able to reject the null hypothesis that the HRs from the SSADMF and SSADMF/MBSF were equivalent. There is an increase in sample size over time and a decreasing trend in the HRs using the SSADMF. This suggests that for comparative effectiveness larger samples and longer observations periods may reduce the impact of temporal variation.

Discussion

By leveraging updates to the Optum CDM® mortality data sources, we were able to evaluate whether under reporting of deaths prohibits use of the SSADMF for comparative evaluations. We demonstrated that despite substantial under-ascertainment of mortality with linkage to the SSADMF, the estimated relative risks for mortality were similar and not statistically different from the relative risks estimated from linkage to a comprehensive mortality data source, the MBSF/SSADMF. In addition, although completeness of mortality data differed geographically by mortality data source, risk estimates were generally consistent across Census Divisions.

In this analysis, we find that the SSADMF provides valid estimates of the relative risk of mortality between treatments (e.g., DCD vs NDCD). Thus, the SSADMF remains a viable potential source of mortality data for other RWD studies comparing treatments and mortality risks. Our analysis also confirmed that the SSADMF is not reliable for estimates of the cumulative frequency of death. These are critically important findings for health services research focused on evaluating survival after an intervention. The SSADMF is a commonly used source of mortality data due to its availability and affordability.

In the context of our study related to DCDs, the VQI registry data, claims, or electronic health record data linked to the SSADMF are all possible sources for long term evaluation of safety. Our results should provide more confidence in the validity of safety signals or rather the lack of a differential mortality effect for DCDs. Our findings may also provide confidence to regulatory agencies seeking validation of various RWD sources. More generally, our results suggest there is value of SSADMF for outcomes research. Following the change in the data sources contributing to the SSADMF in 2011 some believed that the change would severely limit comparative effectiveness research.²⁴ The continued use of the SSADMF demonstrates comparative effectiveness research was not completely restrained. However, to the extent that comparative effectiveness studies of mortality have been avoided due to lack of data, our results should encourage additional research. Under the right conditions – with strong but not untenable assumptions – the SSADMF is a resource worth consideration for comparing mortality effects. If nothing else, researchers should be confident in the results of past and additional peripheral vascular treatment studies.

Although we did find that the SSADMF remains a potentially valuable data source for comparative effectiveness analyses, it is not without limitations. The appropriate application of the SSADMF requires that the treatment and comparator being evaluated are in no way correlated with mortality reporting. In the context of our study, we were able to directly compare rates of DCD use with the degree of under reporting, overall and by Census Divisions. Moreover, we had access to a version of the data with comprehensive mortality reporting to compare with the SSADMF results. Access to an additional mortality data source is extremely unlikely in most other studies employing the SSADMF.

Knowledge of the clinical and institutional details of a procedure, as well as the capability to perform additional sensitivity testing, can be employed to support the assumptions needed to use SSADMF mortality data. For example, DCDs were recommended for specific patient populations per medical society guidelines during our study period.²⁵ Although there may be practice pattern variation in adherence to guidelines, that variation is not likely to be related to state-level mortality record privacy rules and regulations that affect SSADMF mortality reporting. In this study we were able to directly test this assumption. In other applications, researchers will have to support the use of the SSADMF as they plan their analysis. The relevant details and suitable sensitivity tests will need to be study specific, but the assumption that mortality reporting is independent of treatment assignment is not necessarily a strong assumption.

There are also limitations to the clinical results of our study. Although this example of DCD versus NDCD treatment failed to demonstrate a difference in the risk of mortality between death data sources, it is possible that the same finding might not apply for other services or interventions. In addition, the finding that missing outcome data did not influence treatment effect estimates provides reassurance that such estimates in another population with only SSADMF mortality data are unlikely to be biased by missing outcome data. However, we did not have access to an alternative death data source for other patients to confirm this to be true.

Conclusion

The need for timely and accessible mortality data is important for a variety of outcomes research activities, including safety surveillance. Our results confirm that evaluations of DCD mortality using the SSADMF are consistent with results using other mortality data sources. Although the SSADMF alone is not a useful source for mortality rates, it may be fit for purpose for comparative effective analyses assessing mortality outcomes.