This comparative effectiveness research study compares percutaneous microaxial left ventricular assist device vs alternative treatments among patients presenting with acute myocardial infarction with cardiogenic shock.
Key Points
Question
Can the effectiveness of the percutaneous microaxial left ventricular assist device (LVAD) compared with alternative treatments for acute myocardial infarction with cardiogenic shock (AMICS) be evaluated in an observational claims database?
Findings
In this comparative effectiveness study of 23 478 patients with AMICS identified in Medicare claims, percutaneous microaxial LVAD was associated with worse outcomes in some analyses, while in other analyses, the association was too imprecise to draw meaningful conclusions. The distribution of patient and institutional characteristics between groups, combined with clinical knowledge of illness severity factors not accounted for, suggested violations of key assumptions that are needed for valid causal inference.
Meaning
Currently available administrative data do not support a causal interpretation for the results of our observational analyses of the percutaneous microaxial LVAD for AMICS; randomized clinical trials will help resolve ongoing controversies.
Abstract
Importance
Recent studies have produced inconsistent findings regarding the outcomes of the percutaneous microaxial left ventricular assist device (LVAD) during acute myocardial infarction with cardiogenic shock (AMICS).
Objective
To compare the percutaneous microaxial LVAD vs alternative treatments among patients presenting with AMICS using observational analyses of administrative data.
Design, Setting, and Participants
This comparative effectiveness research study used Medicare fee-for-service claims of patients admitted with AMICS undergoing percutaneous coronary intervention from October 1, 2015, through December 31, 2019. Treatment strategies were compared using (1) inverse probability of treatment weighting to estimate the effect of different baseline treatments in the overall population; (2) instrumental variable analysis to determine the effectiveness of the percutaneous microaxial LVAD among patients whose treatment was influenced by cross-sectional institutional practice patterns; (3) an instrumented difference-in-differences analysis to determine the effectiveness of treatment among patients whose treatment was influenced by longitudinal changes in institutional practice patterns; and (4) a grace period approach to determine the effectiveness of initiating the percutaneous microaxial LVAD within 2 days of percutaneous coronary intervention. Analysis took place between March 2021 and December 2022.
Interventions
Percutaneous microaxial LVAD vs alternative treatments (including medical therapy and intra-aortic balloon pump).
Main Outcomes and Measures
Thirty-day all-cause mortality and readmissions.
Results
Of 23 478 patients, 14 264 (60.8%) were male and the mean (SD) age was 73.9 (9.8) years. In the inverse probability of treatment weighting analysis and grace period approaches, treatment with percutaneous microaxial LVAD was associated with a higher risk-adjusted 30-day mortality (risk difference, 14.9%; 95% CI, 12.9%-17.0%). However, patients receiving the percutaneous microaxial LVAD had a higher frequency of factors associated with severe illness, suggesting possible confounding by measures of illness severity not available in the data. In the instrumental variable analysis, 30-day mortality was also higher with percutaneous microaxial LVAD, but patient and hospital characteristics differed across levels of the instrumental variable, suggesting possible confounding by unmeasured variables (risk difference, 13.5%; 95% CI, 3.9%-23.2%). In the instrumented difference-in-differences analysis, the association between the percutaneous microaxial LVAD and mortality was imprecise, and differences in trends in characteristics between hospitals with different percutaneous microaxial LVAD use suggested potential assumption violations.
Conclusions
In observational analyses comparing the percutaneous microaxial LVAD to alternative treatments among patients with AMICS, the percutaneous microaxial LVAD was associated with worse outcomes in some analyses, while in other analyses, the association was too imprecise to draw meaningful conclusions. However, the distribution of patient and institutional characteristics between treatment groups or groups defined by institutional differences in treatment use, including changes in use over time, combined with clinical knowledge of illness severity factors not captured in the data, suggested violations of key assumptions that are needed for valid causal inference with different observational analyses. Randomized clinical trials of mechanical support devices will allow valid comparisons across candidate treatment strategies and help resolve ongoing controversies.
Introduction
Acute myocardial infarction with cardiogenic shock (AMICS) is associated with significant morbidity and mortality.1 Guideline-based management of AMICS involves immediate percutaneous coronary intervention (PCI), with consideration of mechanical circulatory support (MCS) among select patients with refractory shock.2,3 MCS devices can produce potentially beneficial hemodynamic effects in these critically ill patients.4 There has been significant growth in MCS use, particularly the percutaneous microaxial left ventricular assist device (LVAD) (Impella; Abiomed), over the past decade among patients with AMICS undergoing urgent revascularization with PCI.5,6,7 Large randomized clinical trials evaluating the safety and efficacy of percutaneous microaxial LVADs in AMICS have not been completed, although some are ongoing or planned (NCT04763200, NCT01633502, and NCT05003817).
Observational analyses are frequently performed to assess the comparative effectiveness and safety of medical devices, particularly in the absence of randomized clinical trial data. Recent observational analyses of claims and registry data have raised concerns regarding increases in the risk of death and bleeding associated with percutaneous microaxial LVAD use in patients undergoing PCI.5,6,8 Other observational analyses have suggested improved outcomes with use of percutaneous microaxial LVAD prior to PCI.9 Critiques of these analyses have highlighted methodological concerns, particularly the potential for confounding by unmeasured variables.10,11
Following the enactment of the 21st Century Cures Act, the US Food and Drug Administration (FDA) has had a strong interest in understanding different approaches for evaluating the effectiveness and safety of medical devices using real-world data. The primary objective of this study was to explore various approaches to assess the potential effect of the percutaneous microaxial LVAD vs alternative treatments, including intra-aortic balloon pump (IABP) or no MCS device, using an observational administrative database of Medicare fee-for-service beneficiaries undergoing PCI for AMICS. In addition, we used the example of the percutaneous microaxial LVAD for treating AMICS to broadly highlight how different observational analyses may answer different causal questions and rely on distinct assumptions.
Methods
A statistical analysis plan was created jointly by investigators at the Smith Center, the FDA Office of Clinical Evidence and Analysis, and the FDA Office of Cardiovascular Devices, both in the Center for Devices and Radiological Health. This study was reviewed by the institutional review board of Beth Israel Deaconess Medical Center and exempted from oversight under 45 CFR 46 101(b)(4).
Data Source and Study Population
All Medicare fee-for-service beneficiaries who were hospitalized with AMICS and underwent PCI between October 1, 2015, through December 31, 2019, were considered for inclusion. Patients were identified in the Medicare Provider Analysis and Review files using International Statistical Classification of Diseases, Tenth Revision (ICD-10), Clinical Modification codes and ICD-10 Procedure Coding System codes (eTable 1 in Supplement 1). The start of the study period coincided with the rollout of the ICD-10 billing coding system that introduced a specific code for the percutaneous microaxial LVAD.
Outcomes and Covariates
The outcomes examined included all-cause mortality and the composite of all-cause mortality or readmission at 30 days post-PCI. Deaths were identified in the Medicare Beneficiary Summary File, and readmissions were ascertained in the Medicare Provider Analysis and Review files. Covariates used in the analyses included patient characteristics (including race and ethnicity categories of Black, White, other [Hispanic was included in other]) and comorbidities, hospital characteristics, and procedural characteristics and are listed in eTable 2 in Supplement 1.
Research Questions and Causal Assumptions
We sought to evaluate the percutaneous microaxial LVAD for patients undergoing PCI for AMICS. Table 1 and eMethods in Supplement 1 provide more detailed descriptions of study methods. Briefly, we examined a family of questions, each using a different analytical approach that relies on distinct assumptions. These questions related to different potential treatment effects, as described below:
Table 1. Summary of Assumptions, Strengths, and Limitations of the Study Methods.
Treatment effect of interest | Method | Key assumption | Strength | Limitation |
---|---|---|---|---|
Effectiveness of percutaneous microaxial LVAD vs no percutaneous microaxial LVAD at time of PCI | Inverse probability of treatment weighting |
|
|
|
Effectiveness of percutaneous microaxial LVAD vs no percutaneous microaxial LVAD, in patients whose treatment was influenced by cross-sectional institutional preferences | Instrumental variable analysis |
|
|
|
Effectiveness of the percutaneous microaxial LVAD initiation vs no percutaneous microaxial LVAD, in patients whose treatment was influenced by longitudinal changes in institutinoal practice patterns | Instrumented difference-in-differences analysis |
|
|
|
Effectiveness of percutaneous microaxial LVAD vs IABP vs no mechanical support within 2 d of PCI and 30-d outcomes | Grace period with analysis using cloning, censoring, and weighting |
|
|
|
Abbreviations: IABP, intra-aortic balloon pump; IV, instrumental variable; LVAD, left ventricular assist device; PCI, percutaneous coronary intervention.
The effectiveness of initiating treatment on the day of PCI, using an inverse probability treatment weighting (IPTW) approach.12,13 The primary assumption in this analysis is that individuals receiving different treatments are exchangeable (ie, as if randomized) within levels of the covariates. Although this assumption cannot be tested using the data alone, large and directionally consistent between-group differences in prognostically important measured variables may indicate the possibility of between-group differences in additional unmeasured variables, particularly when supported by background knowledge about the nonrandom process of treatment choice.
The effectiveness of treatment in patients whose treatment was influenced by cross-sectional institutional preferences, using an instrumental variable (IV) approach. A major concern about analyses, such as IPTW, that rely on adjustment for measured variables is the possibility that unmeasured variables might influence both treatment and outcomes. For example, because percutaneous microaxial LVADs may be preferentially used in patients with AMICS who clinicians believe are at a higher risk of death or clinical deterioration compared with those receiving IABP or no MCS device, and the factors that drive clinical judgement are often not captured in claims data, there was a priori concern that approaches relying primarily on covariate adjustment could be biased due to unmeasured confounding. A family of approaches for addressing unmeasured confounding depends on identifying an IV. An IV has the following properties: (1) it is associated with treatment (relevance); (2) its effect on the outcome is not confounded (exchangeability or randomization); and (3) it has no effect on the outcome except through its effect on treatment (exclusion restriction). Under an additional assumption that no patient’s treatment responds opposite to encouragement by the instrument (monotonicity), IV approaches can estimate potential treatment effects among individuals whose treatment responds according to the instrument (compliers).14,15 When, as in our analyses, the model for IV analyses includes covariates, the effect estimated is a weighted average of complier effects in different covariate-defined subgroups, provided additional assumptions regarding model specification also hold.16,17
The effectiveness of treatment in patients whose treatment is influenced by longitudinal changes in institutional practice patterns, using an instrumented difference-in-differences design (DiD) approach. One way to address potential threats to the validity of the IV analysis based on cross-sectional preferences is to exploit variation in changes in practice patterns over time to evaluate treatment effectiveness. We examined changes in outcomes at hospitals whose use of percutaneous microaxial LVAD grew rapidly over time and compared them with contemporaneous changes at other hospitals whose use either grew less or declined, using a DiD approach.18,19 The DiD approach requires, among other conditions, that there are no unmeasured differences in secular trends of confounders between the groups with different use trends, a generalization of the monotonicity condition for IV to longitudinal settings, and an assumption that the potential treatment effect is stable over time. With a modified set of assumptions, DiD estimates the effect of treatment in the overall population (similar to IPTW). For that reason, the instrumented DiD analysis can best be viewed as a sensitivity analysis for triangulating the results of IPTW and IV analyses.
The effectiveness of initiating treatment within 2 days of PCI, using a grace period approach with cloning, censoring, and weighting. Because patients may receive MCS devices on days following the PCI, we considered strategies of receiving the percutaneous microaxial LVAD or IABP within a 2-day grace period of PCI for AMICS. Assignment of patients to groups based on the receipt of treatment after baseline (ie, after time zero, the date of PCI) is susceptible to selection bias (eg, immortal time and survival bias), particularly when early mortality rates are high.20 To avoid this problem, we considered each individual to be a candidate for 3 treatment strategies on the day of PCI: the percutaneous microaxial LVAD vs IABP vs no MCS device.21,22 Clones (identical copies) of each individual in the data set were assigned to each treatment strategy at time zero. A clone assigned to a treatment strategy was then (artificially) censored when the person’s treatment history was no longer compatible for the treatment strategy (eg, they received another device or the 2-day grace period ended without initiating the strategy they had been assigned to). Potential bias due to censoring was addressed by inverse probability of censoring weighting using baseline variables, as well as right-sided heart catheterization, intubation, and use of vasopressors as time-varying predictors of treatment initiation (during the grace period). The key assumption of this approach is that the covariates included in the censoring (treatment initiation) model are sufficient to render groups defined by treatment initiation exchangeable given covariates. However, with this approach, there is no baseline confounding because the treatment groups are identical copies of each other prior to any censoring.
Statistical Analyses
SAS statistical software version 9.4 (SAS Institute) was used. Analysis took place between March 2021 and December 2022.
Results
Resulting Data Set and Unadjusted Outcomes
Of 56 983 patients admitted with AMICS undergoing PCI during the study period, 23 478 were included in the analysis (Figure 1). A total of 14 264 (60.8%) were male and the mean (SD) age was 73.9 (9.8) years. Of these, 4063 (17.3%) received the percutaneous microaxial LVAD on the day of PCI and 19 415 (82.7%) did not. Of those who did not receive the percutaneous microaxial LVAD, 6964 (35.9%) received an IABP and 12 451 (64.1%) received no MCS. The unadjusted risk of death and 30-day readmission at 30 days were higher in patients treated with percutaneous microaxial LVAD compared with those not receiving the percutaneous microaxial LVAD (Figure 2, Table 2, and eTable 3 in Supplement 1).
Table 2. Death and All-Cause Readmission at 30 Days After Percutaneous Coronary Intervention Stratified by the Percutaneous Microaxial LVAD vs IABP or No MCSa.
Study method | Death at 30 d, % | Death and all-cause readmission at 30 d, % | ||||
---|---|---|---|---|---|---|
Percutaneous microaxial LVAD | IABP or no MCS | Risk difference (95% CI)b | Percutaneous microaxial LVAD | IABP or no MCS | Risk difference (95% CI)b | |
Unweighted, No. (%) | 2293 (56.4) | 7296 (37.6) | 18.9 (17.2 to 20.5) | 2470 (60.8) | 8601 (44.3) | 16.5 (14.8 to 18.1) |
IPTW analysis, % (95% CI)c | 53.8 (52.0 to 55.7) | 38.9 (38.1 to 39.7) | 14.9 (12.9 to 17.0) | 58.8 (57.1 to 60.6) | 45.6 (44.8 to 46.4) | 13.2 (11.3 to 15.2) |
Instrumental variable analysisc | NA | NA | 13.5 (3.9 to 23.2) | NA | NA | 9.1 (-0.9 to 19.1) |
Grace period analysis (2-d grace period), % (95% CI) | 55.1 (48.6 to 61.5) | 36.6 (35.6 to 37.6) | 18.4 (12.1 to 24.7) | 61.9 (54.3 to 69.6) | 43.1 (41.9 to 44.2) | 18.9 (11.5 to 26.3) |
Instrumented difference-in-differences analysis | ||||||
Between hospitals with rapidly increasing vs moderately increasing percutaneous microaxial LVAD use | NA | NA | 9.7 (−15.9 to 39.9) | NA | NA | 20.0 (−7.5 to 53.2) |
Between hospitals with rapidly increasing vs decreasing percutaneous microaxial LVAD use | NA | NA | 2.6 (−11.6 to 17.4) | NA | NA | 2.6 (−11.9 to 19.4) |
Between hospitals with moderately increasing vs decreasing percutaneous microaxial LVAD use | NA | NA | −4.7 (−33.7 to 25.1) | NA | NA | −15.0 (−45.1 to 16.1) |
Abbreviations: IABP, intra-aortic balloon pump; IPTW, inverse probability treatment weighting; LVAD, left ventricular assist device; MCS, mechanical circulatory support; NA, not applicable.
Propensity score was calculated from a hierarchical logistic regression with hospital ID included as a random intercept.
95% CI calculated from bootstrapping with 500 iterations.
Two-stage linear regressions were performed in this instrumental variable analysis, in the first-stage linear regression with receiving the percutaneous microaxial LVAD treatment vs not included as response variable, and the instrumental variable as well as all the patient and hospital characteristics as predictor variables (analysis of variance P < .001; F = 41.10).
Effectiveness of Initiating Treatment on the Day of PCI
There were notable differences in baseline characteristics of patients treated with different MCS strategies (Table 3 and eTable 4 in Supplement 1). Patients treated with percutaneous microaxial LVAD were younger and more likely to be male and have presented with non–ST-segment elevation myocardial infarction and a history of acute myocardial infarction and heart failure. Information on the severity of cardiogenic shock (eg, Society for Cardiovascular Angiography and Interventions shock classification), dosing of vasopressors, or hemodynamic parameters was not available in the data. Other markers of severity at baseline such as intubation (percutaneous microaxial LVAD: 2321 [57.1%] vs no percutaneous microaxial LVAD: 7039 [36.3%]; standardized mean difference [SMD], 0.43), treatment with vasopressors (percutaneous microaxial LVAD: 618 [15.2%] vs no percutaneous microaxial LVAD: 1909 [9.8%]; SMD, 0.16), and use of right-sided heart catheterization (percutaneous microaxial LVAD: 598 [14.7%] vs no percutaneous microaxial LVAD: 986 [5.1%]; SMD, 0.33) were more common in patients treated with percutaneous microaxial LVAD. These observed imbalances in markers of severity of illness captured in claims data (which were included in our analyses), combined with the lack of information on other known markers of illness severity, suggest considerable potential for residual unmeasured confounding because the measured variables provide an incomplete representation of a patient’s clinical status.
Table 3. Baseline Patient-Level and Hospital-Level Characteristics of Study Population Stratified by Percutaneous Microaxial LVAD vs IABP or No MCSa.
Characteristic | Preweighting, No. (%) | Postweighting, % | ||||
---|---|---|---|---|---|---|
Percutaneous microaxial LVAD (n = 4063) | IABP or no MCS (n = 19 415) | Standardized difference (×100) | Percutaneous microaxial LVAD (n = 4063) | IABP or no MCS (n = 19 415) | Standardized difference (×100) | |
Demographic | ||||||
Age, mean (SD), y | 72.8 (9.3) | 74.1 (9.9) | −13.6 | 73.5 (20.0) | 73.8 (11.3) | −2.2 |
Sex | ||||||
Male | 2751 (67.7) | 11 513 (59.3) | 17.5 | 64.2 | 60.9 | 6.7 |
Female | 1312 (32.3) | 7902 (40.7) | −17.5 | 35.8 | 39.1 | −6.7 |
Race | ||||||
Black | 302 (7.4) | 1328 (6.8) | 2.3 | 7.0 | 7.1 | −0.6 |
White | 3470 (85.4) | 16 952 (87.3) | −5.6 | 86.5 | 86.7 | −0.5 |
Otherb | 291 (7.2) | 1135 (5.8) | 5.3 | 6.5 | 6.2 | 1.3 |
Dual enrollee | 717 (17.6) | 3476 (17.9) | −0.7 | 17.7 | 17.9 | −0.5 |
Presenting AMI type | ||||||
STEMI | 2567 (63.2) | 13 976 (72.0) | −18.9 | 64.6 | 69.9 | −11.3 |
NSTEMI | 1496 (36.8) | 5439 (28.0) | 18.9 | 35.4 | 30.1 | 11.3 |
Cardiac arrest on presentation | 508 (12.5) | 1952 (10.1) | 7.7 | 11.6 | 10.6 | 3.4 |
Intubation | 2321 (57.1) | 7039 (36.3) | 42.8 | 45.4 | 40.5 | 10.0 |
Vasopressor | 618 (15.2) | 1909 (9.8) | 16.3 | 12.4 | 10.5 | 6.1 |
RHC/PA catheters | 598 (14.7) | 986 (5.1) | 32.7 | 8.1 | 6.7 | 5.2 |
Comorbidities | ||||||
Prior PCI | 578 (14.2) | 2765 (14.2) | −0.0 | 15.1 | 14.4 | 1.9 |
Prior CABG | 116 (2.9) | 670 (3.5) | −3.4 | 3.2 | 3.4 | −0.8 |
Acute myocardial infarction | 1376 (33.9) | 4423 (22.8) | 24.8 | 28.4 | 25.1 | 7.5 |
Atrial fibrillation | 617 (15.2) | 3124 (16.1) | −2.5 | 16.4 | 15.9 | 1.4 |
Chronic kidney disease | 2035 (50.1) | 9183 (47.3) | 5.6 | 49.7 | 47.9 | 3.6 |
Chronic obstructive pulmonary disease | 1173 (28.9) | 6409 (33.0) | −9.0 | 31.7 | 32.5 | −1.7 |
Heart failure | 2395 (58.9) | 9653 (49.7) | 18.6 | 55.4 | 51.8 | 7.3 |
Diabetes | 2042 (50.3) | 9049 (46.6) | 7.3 | 48.7 | 47.5 | 2.4 |
Hip/pelvic fracture | 99 (2.4) | 722 (3.7) | −7.4 | 2.9 | 3.5 | −3.0 |
Ischemic heart disease | 3875 (95.4) | 18 602 (95.8) | −2.1 | 95.7 | 95.8 | −0.3 |
Depression | 1234 (30.4) | 6420 (33.1) | −5.8 | 31.4 | 32.7 | −2.8 |
Stroke/transient ischemic attack | 690 (17.0) | 3617 (18.6) | −4.3 | 18.1 | 18.4 | −0.9 |
Breast cancer | 107 (2.6) | 709 (3.7) | −5.8 | 3.4 | 3.4 | −0.2 |
Colorectal cancer | 82 (2.0) | 544 (2.8) | −5.1 | 2.2 | 2.7 | −3.4 |
Prostate cancer | 251 (6.2) | 1128 (5.8) | 1.5 | 6.8 | 5.7 | 4.5 |
Lung cancer | 41 (1.3) | 435 (2.2) | −7.3 | 1.3 | 2.2 | −7.4 |
Endometrial cancer | 12 (0.3) | 121 (0.6) | −4.9 | 0.3 | 0.6 | −3.7 |
Asthma | 512 (12.6) | 2565 (13.2) | −1.8 | 12.7 | 13.1 | −1.1 |
Hyperlipidemia | 3154 (77.6) | 14 899 (76.7) | 2.1 | 77.6 | 77.0 | 1.4 |
Hypertension | 3286 (80.9) | 15 732 (81.0) | −0.4 | 81.0 | 81.1 | −0.3 |
Acquired hypothyroidism | 916 (22.5) | 4815 (24.8) | −5.3 | 23.5 | 24.4 | −2.1 |
Alcohol use disorders | 179 (4.4) | 1029 (5.3) | −4.2 | 5.3 | 5.1 | 0.7 |
Drug use disorder | 232 (5.7) | 1257 (6.5) | −3.2 | 6.1 | 6.4 | −1.0 |
Leukemias and lymphomas | 88 (2.2) | 472 (2.4) | −1.8 | 2.1 | 2.4 | −2.1 |
Liver disease, cirrhosis, and other liver conditions (excluding hepatitis) | 465 (11.4) | 2070 (10.7) | 2.5 | 11.0 | 10.7 | 0.8 |
Obesity | 1138 (28.0) | 5168 (26.6) | 3.1 | 28.6 | 26.9 | 3.9 |
Peripheral vascular disease | 1239 (30.5) | 6163 (31.7) | −2.7 | 31.4 | 31.7 | −0.7 |
Tobacco use disorders | 870 (21.4) | 4756 (24.5) | −7.3 | 23.6 | 24.1 | −1.2 |
Hospital characteristics | ||||||
Hospital size, No. of beds, mean (SD) | 476.1 (345.9) | 453.3 (308.0) | 7.0 | 466.4 (695.8) | 459.9 (360.5) | 1.2 |
Ownership | ||||||
For profit | 696 (17.1) | 2909 (15.0) | 5.9 | 15.2 | 15.7 | −1.3 |
Private nonprofit | 2970 (73.1) | 14 738 (75.9) | −6.5 | 75.8 | 74.9 | 2.2 |
Public | 397 (9.8) | 1768 (9.1) | 2.3 | 9.0 | 9.4 | −1.7 |
Teaching status | ||||||
Metropolitan teaching | 2940 (72.4) | 13 828 (71.2) | 2.5 | 72.1 | 71.3 | 1.9 |
Metropolitan nonteaching | 1106 (27.2) | 5534 (28.5) | −2.9 | 27.6 | 28.4 | −1.9 |
Rural | 17 (0.4) | 53 (0.3) | 2.5 | 0.3 | 0.3 | 0.3 |
Region | ||||||
Northeast | 543 (13.4) | 2818 (14.5) | −3.3 | 14.9 | 14.1 | 2.4 |
Midwest | 717 (17.6) | 3360 (17.3) | 0.9 | 17.4 | 17.5 | −0.3 |
South | 2050 (50.5) | 9478 (48.8) | 3.3 | 48.5 | 49.1 | −1.4 |
West | 753 (18.5) | 3759 (19.4) | −2.1 | 19.2 | 19.3 | −0.1 |
Minority-serving hospitalc | 845 (20.8) | 3497 (18.0) | 7.0 | 18.7 | 18.5 | 0.3 |
Hospital ADI, mean (SD) | 53.2 (21.3) | 52.9 (21.6) | 1.7 | 52.6 (46.2) | 53.1 (24.1) | −1.5 |
Abbreviations: ADI, area deprivation index; AMI, acute myocardial infarction; CABG, coronary artery bypass graft; LVAD, left ventricular assist device; IABP, intra-aortic balloon pump; MCS, mechanical circulatory support; NSTEMI, non–ST-segment elevation myocardial infarction; PCI, percutaneous coronary intervention; RHC/PA, right-sided heart catheter/ pulmonary artery catheter; STEMI, ST-segment elevation myocardial infarction.
Propensity score was calculated from a hierarchical logistic regression with hospital ID included as a random intercept.
Other is defined as American Indian/Alaska Native, Asian/Pacific Islander, and Hispanic/Latino.
Defined as those in the top 20% of institutions ranked by the proportion of inpatient admissions that were for Black patients during the study period.
After weighting, there were minimal imbalances in measured variables between groups, with the exception of acute myocardial infarction type (non–ST-segment elevation myocardial infarction: percutaneous microaxial LVAD: 35.4% vs no percutaneous microaxial LVAD: 30.1%; SMD, 0.11) and intubation status (percutaneous microaxial LVAD: 45.4% vs no percutaneous microaxial LVAD: 40.5%; SMD, 0.10). The distribution of the estimated probability of receiving a percutaneous microaxial LVAD for each treatment group is shown in eFigure 1A-C in Supplement 1.
After weighting, the mortality risk at 30 days post-PCI was higher in patients initially treated with percutaneous microaxial LVAD compared with patients not treated with percutaneous microaxial LVAD (53.8% [95% CI, 52.0%-55.7%] vs 38.9% [95% CI, 38.1%-39.7%], respectively; difference, 14.9% [95% CI, 12.9%-17.0%]) (Figure 2 and Table 2) and so is the risk of all-cause mortality or readmission at 30 days (58.8% [95% CI, 57.1%-60.6%] vs 45.6% [95% CI, 44.8%-46.4%]; difference, 13.2% [95% CI, 11.3%-15.2%]).
When treatments with IABP and no MCS device were evaluated separately, the adjusted 30-day mortality risk post-PCI was 41.3% (95% CI, 40.0%-42.7%) in those treated with IABP (percutaneous microaxial LVAD vs IABP difference, 11.4% [95% CI, 9.1%-13.7%]), and 37.2% (95% CI, 36.2%-38.2%) in those treated with no MCS (percutaneous microaxial LVAD vs no MCS difference, 15.5% [95% CI, 13.4%-17.6%]) (eTable 3 in Supplement 1). Similarly, the adjusted risk of mortality or readmission at 30 days post-PCI remained higher with the initial treatment with the percutaneous microaxial LVAD compared with other treatment strategies (57.8% [95% CI, 56.1%-59.5%] for percutaneous microaxial LVAD; 47.8% [95% CI, 46.4%-49.1%] for IABP; 43.9% [95% CI, 42.9%-44.9%] with no MCS; difference, 10.0% [95% CI, 7.8%-12.2%] for percutaneous microaxial LVAD vs IABP; difference, 13.9% [95% CI, 11.8%-15.9%] for percutaneous microaxial LVAD vs no MCS).
Effectiveness of Treatment Among Patients Whose Treatment Was Influenced by Cross-Sectional Institutional Preferences
The median percent use of percutaneous microaxial LVAD in the 2 years prior to each procedure among hospitals was 0% in the lowest fifth and 29.6% in the highest fifth of hospitals stratified by use, with skewed distribution (eFigure 2 in Supplement 1). Several variables were not balanced across groups defined by levels of use (eTable 5 in Supplement 1); the imbalances suggest that, in treating past institutional the percutaneous microaxial LVAD use as an IV, the exchangeability assumption may be violated by unmeasured variables that are common causes of hospital the percutaneous microaxial LVAD use and the outcome. Specifically, patients treated at hospitals with the highest prior use of percutaneous microaxial LVAD had greater use of vasopressors and right-sided heart catheterization, and lower rates of cardiac arrest at presentation. In addition, hospitals in the highest group of the percutaneous microaxial LVAD use were more likely to be metropolitan teaching hospitals, treated the highest percentage of Black patients, and were more likely to be classified as a minority-serving hospital (ie, those in the top 20% of institutions ranked by the proportion of inpatient admissions that were for Black patients during the study period) compared with hospitals with lower percutaneous microaxial LVAD use. In the IV analysis, there was a higher risk of 30-day mortality with percutaneous microaxial LVAD vs no percutaneous microaxial LVAD (difference, 13.5% [95% CI, 3.9%-23.2%]) (Figure 2 and Table 2). The difference in the composite outcome of mortality or readmission at 30 days of percutaneous microaxial LVAD vs no percutaneous microaxial LVAD was 9.1% (95% CI, −0.9% to 19.1%).
Effectiveness of Treatment Among Patients Whose Treatment Was Influenced by Longitudinal Changes in Institutional Preferences
The proportion of the percutaneous microaxial LVAD use among patients with AMICS undergoing PCI rose from 11.4% (648 of 5705) in 2016 to 22.4% (1266 of 5650) in 2019 (eFigure 3 in Supplement 1). There was significant variation among hospitals in longitudinal trends in the percutaneous microaxial LVAD use over the period analyzed, with hospitals in the lowest third reducing the percutaneous microaxial LVAD use from 18.6% (178 of 957) to 11.3% (125 of 1105), and hospitals in the highest third of growth increasing the percutaneous microaxial LVAD use from 6.4% (61 of 948) to 36.8% (395 of 1073) (eFigure 4, eFigure 5, and eTable 6 in Supplement 1). There were significant differences in trends in baseline characteristics across groups during the period analyzed, indicating the potential for violation of the assumption of no unmeasured differences in time-varying confounding between DiD-defined groups (eTable 7 in Supplement 1).
The risk difference of percutaneous microaxial LVAD vs no percutaneous microaxial LVAD for 30-day mortality varied from −4.7% (95% CI, −15.9% to 39.9%) when using moderately increasing vs declining percutaneous microaxial LVAD use hospitals as the DiD to 9.7% (95% CI, −15.9% to 39.9%) when using rapidly increasing vs moderately increasing the percutaneous microaxial LVAD use hospitals as the DiD. CIs for estimated DiD treatment effects were wide (Figure 2 and Table 2).
Effectiveness of Initiating Treatment Within 2 Days of PCI
In the analysis using a 2-day grace period for receiving MCS, the adjusted 30-day post-PCI cumulative incidence of mortality was 55.1% (95% CI, 48.6-61.5) in patients treated with percutaneous microaxial LVAD and 36.6% (95% CI, 35.6-37.6) in those not treated with percutaneous microaxial LVAD (difference, 18.4% [95% CI, 12.1% to 24.7%]) (Figure 2, Table 2). The risk difference in the composite of all-cause mortality or readmission at 30 days was 18.9% (95% CI, 11.5%-26.3%).
For the individual treatment groups, the adjusted 30-day post-PCI risk of death was 56.9% (95% CI, 48.15%-65.7%) in patients treated with percutaneous microaxial LVAD, 45.5% (95% CI, 41.4%-49.5%) in those treated with IABP, and 34.0% (95% CI, 32.0%-36.0%) in those treated with no MCS (difference, 11.4% [95% CI, 2.1%-20.7%] for percutaneous microaxial LVAD vs IABP; difference, 22.9% [95% CI, 14.4%-31.4%] for percutaneous microaxial LVAD vs no MCS) (eTable 3 in Supplement 1). The risk difference in the composite of mortality or readmission at 30 days was 11.6% (95% CI, 2.3%-20.9%) between percutaneous microaxial LVAD and IABP and 22.6% (95% CI, 13.9%-31.4%) between percutaneous microaxial LVAD and no MCS. Time-varying covariates (right-sided heart catheterization, intubation, and vasopressors) were strongly associated with the percutaneous microaxial LVAD initiation during the grace period. This finding, combined with the lack of other hemodynamic parameters that likely drove treatment during the grace period, suggest the potential for residual unmeasured confounding.
Discussion
We examined a family of causal questions regarding the potential effects of the percutaneous microaxial LVAD MCS device compared with alternative treatment strategies on short-term outcomes among Medicare fee-for-service patients presenting with AMICS undergoing PCI. Attempting to answer each question required distinct analytical approaches. First, we studied the effectiveness of the percutaneous microaxial LVAD use on the day of PCI using IPTW and found that it was associated with higher risk of 30-day mortality and readmission. Second, we analyzed the effectiveness of the percutaneous microaxial LVAD among patients whose treatment was by cross-sectional practice patterns using IV and found results consistent with IPTW analyses. Third, we exploited variation in longitudinal trends in the percutaneous microaxial LVAD use between hospitals using a DiD approach and obtained results compatible with the null but with very wide CIs. Finally, we investigated the effectiveness of percutaneous microaxial LVAD use within 48 hours of PCI using a grace period approach and observed that it was associated with increases in mortality and readmission at 30 days. Importantly, however, evidence from the data as well as background knowledge regarding drivers of treatment decisions and institutional differences, both cross-sectional and longitudinal, suggested that key assumptions for all approaches likely were strongly violated. Thus, we are not confident that any of the statistical estimates summarized above can be given a causal interpretation.
The mortality rate of patients presenting with AMICS remains high.23 There have been substantial efforts to improve outcomes of these patients through more refined characterization of shock phenotypes, protocol-driven use of invasive hemodynamic monitoring, and early use of MCS prior to clinical deterioration.24,25 These efforts have coincided with a general increase in the use of percutaneous microaxial LVADs nationally in the setting of AMICS, despite the lack of adequately powered randomized trials to date. Recent observational studies have raised the possibility that the devices could be associated with harm.5,6
Because administrative claims do not capture many important covariates driving clinical decision-making (eg, shock severity), residual confounding is a major concern for methods that rely on patient-level covariate adjustment at baseline or over time. While the presence of residual confounding cannot be empirically established without additional data and strong assumptions, the significantly higher rates of incomplete markers for shock severity prior to weighting, including vasopressor use, presence of heart failure, and need for intubation in the percutaneous microaxial LVAD group suggest that this group is at significantly higher risk of mortality compared with groups receiving IABP or no MCS. This is consistent with both the recommendation of expert consensus statements and our clinical experience that percutaneous microaxial LVADs are preferentially used over alternatives when a patient is believed to be at a greater risk of death or decompensation and requires a higher level of circulatory support.26 While administrative claims data sets may be particularly vulnerable to this bias due to lack of granular clinical information, many available registries also fail to collect structured data on hemodynamic measurements throughout the course of hospitalization including at the time of MCS placement, doses of vasopressors, or grade of shock severity (eg, Society for Cardiovascular Angiography and Interventions shock class) that may strongly influence treatment decisions and affect outcomes.
Considering these challenges, we posed a different set of questions to examine the association of the percutaneous microaxial LVAD use with outcomes using the IV and instrumented DiD methods, which are less commonly used approaches that may in some cases be useful to compare interventions in the presence of strong confounding by unmeasured variables.21,22 In our study, however, there were significant differences across levels of the IV in important patient characteristics including imperfect surrogates of patient severity of illness such as vasopressor use and right-sided heart catheterization, as well as hospital factors such as bed size, teaching status, and classification as minority-serving. CIs for the cross-sectional IV-based treatment effect were wide (including a null effect on combined 30-day mortality and readmission). The instrumented DiD method relies on several strong and untestable assumptions that there are no differences in the secular trends in unmeasured confounders between hospitals with different growth patterns.18 The results of the DiD analysis had very wide CIs compatible with both benefit or harm from the percutaneous microaxial LVAD. In addition, there was empirical evidence that longitudinal trends in patient characteristics between DiD-defined groups were different suggesting potential violations of assumptions.
Limitations
Our study has several additional limitations. First, we focused on causal questions that can be answered with commonly used methods for the analysis of observational studies, including those used in recent studies of the percutaneous microaxial LVAD; whether these are the most clinically relevant questions is less clear (eg, questions about dynamic treatment strategies that determine the use of circulatory support based on the patient’s time-varying clinical status may be more relevant). Second, in practice, the decision to initiate circulatory support is made during the course of the hospital stay by considering baseline patient characteristics, changes in hemodynamic measurements, markers of tissue perfusion, and response to treatment including the initiation of pressors and inotropes. These types of data with fine time resolution were not available in this study, as with most other administrative and registry data, but would be needed to account for the high force of mortality of AMICs and the rapid, dynamic decision-making in critical care settings. Last, our analyses were limited to patients enrolled in Medicare fee-for-service, limiting their applicability to younger patients with AMICS as well as those with other insurance types.
Observational studies evaluating the percutaneous microaxial LVAD in various settings have reported varied magnitudes of association between the device and outcomes.5,6,27 In our study, none of the analyses showed a significant association of the percutaneous microaxial LVAD with improved outcomes. However, given the expected unmeasured confounding in direct patient comparisons, we have low confidence that approaches relying on covariate adjustment using currently available data sets can validly estimate treatment effect of percutaneous microaxial LVAD among patients with AMICS. While new registries will not obviate the need for randomized trials to better define the benefit-risk profile of MCS, a more detailed AMICS registry that includes patients receiving different treatments and captures richer and finely time-stamped information on drivers of treatment selection, shock classification, and hemodynamic measurements could help overcome some limitations of presently available observational data. Furthermore, improved registry data may be useful to guide the design (eg, identifying promising new treatment strategies) and conduct of future randomized trials (eg, embedding trials within registries).28,29 Renewed efforts should be made to complete ongoing and planned randomized trials, including the DanGer Shock trial (NCT01633502) and RECOVER IV trial (NCT05506449).
More generally, observational analyses of cardiovascular devices should be conducted, analyzed, and interpreted carefully (eg, using the target-emulation framework30 or other systematic approaches for causal inference31) with attention to the precise causal questions being asked and the assumptions required to answer them. Prespecification of analyses, use of alternative methods that rely on different assumptions, sensitivity analyses, and the application of clinical subject matter expertise to understand the primary drivers of treatment use are critical elements in undertaking observational analyses that are meant to inform clinical care and public health decisions. Investigators often avoid the use of causal language when describing study goals while acknowledging the potential for residual confounding (usually relegated to limitations sections); it should be acknowledged, however, that the goal of many observational analyses is to estimate causal effects of treatments on outcomes. Our study demonstrates that, in some cases, the primary conclusion drawn from such analyses may be that the data are insufficient to support a credible causal interpretation and, therefore, the causal goal cannot be attained.
Conclusions
We used different approaches in an attempt to answer a family of questions regarding the effect of percutaneous microaxial left ventricular assist MCS devices with 30-day outcomes (ie, mortality and rehospitalization). The percutaneous microaxial LVAD was associated with worse outcomes in some analyses, while in other analyses, the association was too imprecise to draw meaningful conclusions. The distribution of patient and institutional characteristics between treatment groups or groups defined by changes in treatment use over time, combined with clinical knowledge of illness severity factors not captured in the data, suggested violations of key assumptions needed for valid causal inference with different observational analyses. Our findings suggest that commonly used observational data sets cannot support a causal interpretation of the estimates produced by different analyses used for the evaluation of percutaneous mechanical support devices in cardiogenic shock. Randomized clinical trials will allow valid comparisons across candidate treatment strategies and help resolve ongoing controversies.
References
- 1.Lauridsen MD, Rørth R, Lindholm MG, et al. Trends in first-time hospitalization, management, and short-term mortality in acute myocardial infarction-related cardiogenic shock from 2005 to 2017: a nationwide cohort study. Am Heart J. 2020;229:127-137. doi: 10.1016/j.ahj.2020.08.012 [DOI] [PubMed] [Google Scholar]
- 2.Ibanez B, James S, Agewall S, et al. ; ESC Scientific Document Group . 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: the Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J. 2018;39(2):119-177. doi: 10.1093/eurheartj/ehx393 [DOI] [PubMed] [Google Scholar]
- 3.O’Gara PT, Kushner FG, Ascheim DD, et al. 2013 ACCF/AHA guideline for the management of ST-elevation myocardial infarction: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013;61(4):e78-e140. doi: 10.1016/j.jacc.2012.11.019 [DOI] [PubMed] [Google Scholar]
- 4.Burkhoff D, Sayer G, Doshi D, Uriel N. Hemodynamics of mechanical circulatory support. J Am Coll Cardiol. 2015;66(23):2663-2674. doi: 10.1016/j.jacc.2015.10.017 [DOI] [PubMed] [Google Scholar]
- 5.Amin AP, Spertus JA, Curtis JP, et al. The evolving landscape of Impella use in the United States among patients undergoing percutaneous coronary intervention with mechanical circulatory support. Circulation. 2020;141(4):273-284. doi: 10.1161/CIRCULATIONAHA.119.044007 [DOI] [PubMed] [Google Scholar]
- 6.Dhruva SS, Ross JS, Mortazavi BJ, et al. Association of use of an intravascular microaxial left ventricular assist device vs intra-aortic balloon pump with in-hospital mortality and major bleeding among patients with acute myocardial infarction complicated by cardiogenic shock. JAMA. 2020;323(8):734-745. doi: 10.1001/jama.2020.0254 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Strom JB, Zhao Y, Shen C, et al. National trends, predictors of use, and in-hospital outcomes in mechanical circulatory support for cardiogenic shock. EuroIntervention. 2018;13(18):e2152-e2159. doi: 10.4244/EIJ-D-17-00947 [DOI] [PubMed] [Google Scholar]
- 8.Miller PE, Bromfield SG, Ma Q, et al. Clinical outcomes and cost associated with an intravascular microaxial left ventricular assist device vs intra-aortic balloon pump in patients presenting with acute myocardial infarction complicated by cardiogenic shock. JAMA Intern Med. 2022;182(9):926-933. doi: 10.1001/jamainternmed.2022.2735 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Basir MB, Schreiber TL, Grines CL, et al. Effect of early initiation of mechanical circulatory support on survival in cardiogenic shock. Am J Cardiol. 2017;119(6):845-851. doi: 10.1016/j.amjcard.2016.11.037 [DOI] [PubMed] [Google Scholar]
- 10.Chieffo A, Gramegna M, Pappalardo F. Letter by Chieffo et al regarding article, "the evolving landscape of Impella use in the United States among patients undergoing percutaneous coronary intervention with mechanical circulatory support". Circulation. 2020;142(6). doi: 10.1161/CIRCULATIONAHA.119.045169 [DOI] [PubMed] [Google Scholar]
- 11.Rizzo JA, Dove H. Intravascular microaxial left ventricular assist device vs intra-aortic balloon pump for cardiogenic shock. JAMA. 2020;324(3):303. doi: 10.1001/jama.2020.7551 [DOI] [PubMed] [Google Scholar]
- 12.Hernán MA, Robins JM. Randomized experiments. In: Causal Inference: What If. Chapman & Hall; 2020:13-24. [Google Scholar]
- 13.Robins JM, Hernán MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000;11(5):550-560. doi: 10.1097/00001648-200009000-00011 [DOI] [PubMed] [Google Scholar]
- 14.Kennedy EH, Lorch S, Small DS. Robust causal inference with continuous instruments using the local instrumental variable curve. J R Stat Soc Series B Stat Methodol. 2018;81(1):121-143. doi: 10.1111/rssb.12300 [DOI] [Google Scholar]
- 15.Glickman ME, Normand S-LT. The derivation of a latent threshold instrumental variables model. Stat Sin. 2000;10(2):517-544. [Google Scholar]
- 16.Angrist J, Pischke J-S. Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press; 2009. [Google Scholar]
- 17.Blandhol C, Bonney J, Mogstad M, Torgovitsky A. When is TSLS actually LATE? National Bureau of Economic Research Working Paper Series. Revised August 2022. Accessed May 16, 2023. doi: 10.3386/w29709 [DOI]
- 18.Abadie A. Semiparametric difference-in-differences estimators. Review of Economic Studies. 2005;72(1):1-19. doi: 10.1111/0034-6527.00321 [DOI] [Google Scholar]
- 19.Ye T, Ertefaie A, Flory J, Hennessy S, Small DS. Instrumented difference-in-differences. Biometrics. Accessed October 27, 2022. doi: 10.1111/biom.13783 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Secemsky EA, Yeh RW. Re: multivessel percutaneous coronary intervention in patients with ST-segment elevation myocardial infarction with cardiogenic shock. Epidemiology. 2018;29(6):e59-e60. doi: 10.1097/EDE.0000000000000884 [DOI] [PubMed] [Google Scholar]
- 21.McClellan M, Newhouse JP. The marginal cost-effectiveness of medical technology: a panel instrumental-variables approach. J Econom. 1997;77(1):39-64. doi: 10.1016/S0304-4076(96)01805-2 [DOI] [Google Scholar]
- 22.Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. J Am Stat Assoc. 1996;91(434):444-455. doi: 10.1080/01621459.1996.10476902 [DOI] [Google Scholar]
- 23.Virani SS, Alonso A, Aparicio HJ, et al. ; American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee . Heart disease and stroke statistics-2021 update: a report from the American Heart Association. Circulation. 2021;143(8):e254-e743. doi: 10.1161/CIR.0000000000000950 [DOI] [PubMed] [Google Scholar]
- 24.Baran DA, Grines CL, Bailey S, et al. SCAI clinical expert consensus statement on the classification of cardiogenic shock. Catheter Cardiovasc Interv. 2019;94(1):29-37. doi: 10.1002/ccd.28329 [DOI] [PubMed] [Google Scholar]
- 25.Basir MB, Kapur NK, Patel K, et al. ; National Cardiogenic Shock Initiative Investigators . Improved outcomes associated with the use of shock protocols: updates from the National Cardiogenic Shock Initiative. Catheter Cardiovasc Interv. 2019;93(7):1173-1183. doi: 10.1002/ccd.28307 [DOI] [PubMed] [Google Scholar]
- 26.Rihal CS, Naidu SS, Givertz MM, et al. ; Society for Cardiovascular Angiography and Interventions (SCAI); Heart Failure Society of America (HFSA); Society of Thoracic Surgeons (STS); American Heart Association (AHA), and American College of Cardiology (ACC) . 2015 SCAI/ACC/HFSA/STS clinical expert consensus statement on the use of percutaneous mechanical circulatory support devices in cardiovascular care: endorsed by the American Heart Association, the Cardiological Society of India, and Sociedad Latino Americana de Cardiologia Intervencion; Affirmation of Value by the Canadian Association of Interventional Cardiology-Association Canadienne de Cardiologie d’intervention. J Am Coll Cardiol. 2015;65(19):e7-e26. doi: 10.1016/j.jacc.2015.03.036 [DOI] [PubMed] [Google Scholar]
- 27.Dhruva SS, Ross JS, Mortazavi BJ, et al. Use of mechanical circulatory support devices among patients with acute myocardial infarction complicated by cardiogenic shock. JAMA Netw Open. 2021;4(2):e2037748. doi: 10.1001/jamanetworkopen.2020.37748 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Lauer MS, D’Agostino RB Sr. The randomized registry trial–the next disruptive technology in clinical research? N Engl J Med. 2013;369(17):1579-1581. doi: 10.1056/NEJMp1310102 [DOI] [PubMed] [Google Scholar]
- 29.Dahabreh IJ, Haneuse SJA, Robins JM, et al. Study designs for extending causal inferences from a randomized trial to a target population. Am J Epidemiol. 2021;190(8):1632-1642. doi: 10.1093/aje/kwaa270 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. 2016;183(8):758-764. doi: 10.1093/aje/kwv254 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Dang LE, Gruber S, Lee H, et al. A causal roadmap for generating high-quality real-world evidence. arXiv. Preprint posted May 11, 2023. doi: 10.48550/arXiv.2305.06850 [DOI] [PMC free article] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.