Key Points
Question
What patient characteristics are associated with benefit or harm of therapeutic-dose heparin in patients hospitalized for moderate or severe COVID-19, and how do methods to analyze heterogeneity of treatment effect (HTE) in clinical trial populations compare?
Findings
In an exploratory analysis of a multiplatform randomized trial of therapeutic-dose heparin for early-pandemic patients with moderate or severe COVID-19, 3 approaches for testing HTE—conventional subgroup analysis, risk-based analysis, and effect-based analysis—were congruent in findings that therapeutic-dose heparin was more likely to be beneficial in patients who were less severely ill at presentation or who had lower body mass index, and more likely to be harmful in sicker patients and those with higher body mass index.
Meaning
Benefits and harms of therapeutic-dose heparin varied by hospitalized COVID-19 patient characteristics, illustrating the importance of considering HTE in the design and analysis of randomized clinical trials.
Abstract
Importance
Randomized clinical trials (RCTs) of therapeutic-dose heparin in patients hospitalized with COVID-19 produced conflicting results, possibly due to heterogeneity of treatment effect (HTE) across individuals. Better understanding of HTE could facilitate individualized clinical decision-making.
Objective
To evaluate HTE of therapeutic-dose heparin for patients hospitalized for COVID-19 and to compare approaches to assessing HTE.
Design, Setting, and Participants
Exploratory analysis of a multiplatform adaptive RCT of therapeutic-dose heparin vs usual care pharmacologic thromboprophylaxis in 3320 patients hospitalized for COVID-19 enrolled in North America, South America, Europe, Asia, and Australia between April 2020 and January 2021. Heterogeneity of treatment effect was assessed 3 ways: using (1) conventional subgroup analyses of baseline characteristics, (2) a multivariable outcome prediction model (risk-based approach), and (3) a multivariable causal forest model (effect-based approach). Analyses primarily used bayesian statistics, consistent with the original trial.
Exposures
Participants were randomized to therapeutic-dose heparin or usual care pharmacologic thromboprophylaxis.
Main Outcomes and Measures
Organ support–free days, assigning a value of −1 to those who died in the hospital and the number of days free of cardiovascular or respiratory organ support up to day 21 for those who survived to hospital discharge; and hospital survival.
Results
Baseline demographic characteristics were similar between patients randomized to therapeutic-dose heparin or usual care (median age, 60 years; 38% female; 32% known non-White race; 45% Hispanic). In the overall multiplatform RCT population, therapeutic-dose heparin was not associated with an increase in organ support–free days (median value for the posterior distribution of the OR, 1.05; 95% credible interval, 0.91-1.22). In conventional subgroup analyses, the effect of therapeutic-dose heparin on organ support–free days differed between patients requiring organ support at baseline or not (median OR, 0.85 vs 1.30; posterior probability of difference in OR, 99.8%), between females and males (median OR, 0.87 vs 1.16; posterior probability of difference in OR, 96.4%), and between patients with lower body mass index (BMI <30) vs higher BMI groups (BMI ≥30; posterior probability of difference in ORs >90% for all comparisons). In risk-based analysis, patients at lowest risk of poor outcome had the highest propensity for benefit from heparin (lowest risk decile: posterior probability of OR >1, 92%) while those at highest risk were most likely to be harmed (highest risk decile: posterior probability of OR <1, 87%). In effect-based analysis, a subset of patients identified at high risk of harm (P = .05 for difference in treatment effect) tended to have high BMI and were more likely to require organ support at baseline.
Conclusions and Relevance
Among patients hospitalized for COVID-19, the effect of therapeutic-dose heparin was heterogeneous. In all 3 approaches to assessing HTE, heparin was more likely to be beneficial in those who were less severely ill at presentation or had lower BMI and more likely to be harmful in sicker patients and those with higher BMI. The findings illustrate the importance of considering HTE in the design and analysis of RCTs.
Trial Registration
ClinicalTrials.gov Identifiers: NCT02735707, NCT04505774, NCT04359277, NCT04372589
This exploratory study of a multiplatform randomized trial investigating the effects of therapeutic-dose heparin in early-pandemic hospitalized COVID-19 patients describes findings from 3 statistical approaches to detecting differences of treatment effect in clinically relevant patient subgroups.
Introduction
Thrombosis and inflammation contribute to critical illness or death in patients hospitalized for COVID-19.1 Multiple randomized clinical trials (RCTs) evaluated the benefit of therapeutic-dose heparin in these patients, with varying results. A multiplatform RCT (mpRCT)2 and the Therapeutic Anticoagulation vs Standard Care as a Rapid Response to the COVID-19 Pandemic (RAPID) RCT3 both reported clinical benefit from therapeutic-dose heparin in non–critically ill hospitalized COVID-19 patients; conversely, the mpRCT and other trials observed no benefit and probable harm from therapeutic- or intermediate-dose heparin in critically ill COVID-19 patients.4,5,6 Such divergent findings point to differences in heparin’s effect according to patient characteristics, also known as heterogeneity of treatment effect (HTE).
Estimates of between-group differences in RCT outcomes like those summarized above represent average treatment effects (ATEs) across all study participants. Ideally, clinicians need estimates of treatment effect for individual patients, but that would require knowledge of counterfactuals that cannot be observed (a patient’s outcome under treatment and without treatment).7,8 The individual treatment effect may vary widely within a population of patients because of differences among patients in relevant characteristics such as baseline health status, severity of infection, susceptibility to harm from anticoagulation, and multiple other factors.9,10 Although individual treatment effect is not estimable, treatment effects can be estimated for patient subgroups defined by a shared set of relevant characteristics (the conditional average treatment effect [cATE]). Subgroup analysis is a conventional approach to estimating cATE but has important limitations,11,12 so methods have been developed for more valid estimation of treatment effect in subgroups. These include risk-based analysis13,14 and effect-based analysis15 of HTE using data-driven modeling of cATEs with machine-learning techniques.16 These newer methods improve on conventional subgroup analysis8 but may yield dissimilar or even conflicting results, which can limit their utility in clinical decision-making.15,17
In this post hoc exploratory study, we applied these analytical approaches to pooled mpRCT data to empirically test the ability of these methods to identify heterogeneity in heparin’s treatment effects in patients with COVID-19 and to test for consistency between the results of these approaches as to which patients hospitalized for COVID-19 might benefit from the intervention.
Methods
Study Design and Inclusion Criteria
The mpRCT was a collaboration between the Antithrombotic Therapy to Ameliorate Complications of COVID-19 (ATTACC) trial,18 the Accelerating COVID-19 Therapeutic Interventions and Vaccines-4 Antithrombotics Inpatient platform trial (ACTIV-4a), and the Randomized, Embedded, Multifactorial Adaptive Platform Trial for Community-Acquired Pneumonia (REMAP-CAP).19 Eligibility criteria, interventions, outcome measures, and data collection were prospectively aligned across the 3 platforms to evaluate the effect of pragmatically defined regimens of therapeutic-dose heparin administered prophylactically or usual care pharmacologic thromboprophylaxis on mortality and organ support–free days to day 21 (see eTable 1 in Supplement 1 for details). Ethics and regulatory approval were obtained at each participating center. Written or oral informed consent, in accordance with regional regulations, was obtained from all patients or their surrogates.
The mpRCT prospectively stratified the primary analysis based on illness severity, evaluating distinct treatment effects in hospitalized patients with laboratory-confirmed COVID-19 with severe illness (critically ill or requiring intensive care unit [ICU]–level care) or moderate illness (hospitalized non–critically ill patients not requiring ICU-level care). Patients with moderate illness were further analyzed according to baseline D-dimer level. Intensive care unit–level care was defined as use of respiratory or cardiovascular organ support (oxygen delivered by high-flow nasal cannula, noninvasive or invasive ventilation, or use of vasopressors or inotropes). The mpRCT used bayesian hierarchical modeling to separately estimate the posterior probability of treatment benefit in the primary end point in patients with severe COVID-19, moderate COVID-19 with elevated baseline D-dimer (more than 2 times the upper limit of normal), moderate COVID-19 without elevated baseline D-dimer, and moderate COVID-19 with missing baseline D-dimer data. Adaptive analyses were planned to determine the final sample size based on accumulating information about the probability of superiority or futility. The mpRCT completed enrollment in patients with severe COVID-19 in December 2020, based on a statistical conclusion of futility, and subsequently completed enrollment in patients with moderate COVID-19 in January 2021, based on a statistical conclusion of superiority across all D-dimer groups. These mpRCT results were described in 2 separate reports2,4 including one-at-a-time subgroup analyses exploring HTE in moderate and severe COVID-19 separately.2,4 In the present study, we report analyses conducted across the entire population in the mpRCT with laboratory-confirmed COVID-19, including both moderately and severely ill patients.
Outcomes
The primary outcome for the mpRCT was organ support–free days, evaluated on an ordinal scale that combined in-hospital death (assigned a value of −1) and the number of days free of cardiovascular or respiratory organ support up to day 21 among patients who survived to hospital discharge. Patients who never required organ support were assigned a value of 22. Patients who were discharged from the hospital before day 21 were assumed to be alive and free of organ support from discharge through day 21. Any death during the index hospitalization through 90 days was assigned the worst score on the outcome scale (–1). This end point reflects both survival to hospital discharge and the need for and duration of ICU-level interventions, with higher values indicating better outcomes.
A secondary outcome for this study was survival to hospital discharge, a key component of the primary outcome.
Statistical Analysis
Heterogeneity of treatment effect was evaluated using 3 strategies planned after publication of the original mpRCT results (Figure 1).
Figure 1. Three Strategies to Evaluate HTE of Therapeutic-Dose Heparin for COVID-19.
ACTIV-4a indicates Accelerating COVID-19 Therapeutic Interventions and Vaccines-4 Antithrombotics Inpatient trial; ARD, observed absolute rate difference in hospital survival; ATTACC, Antithrombotic Therapy to Ameliorate Complications of COVID-19 trial; cARD, conditional absolute rate difference in hospital survival; cATE, conditional average treatment effect (the treatment effect within a subgroup); HTE, heterogeneity of treatment effect; OR, odds ratio; OSFDs, organ support–free days; REMAP-CAP, Randomized, Embedded, Multifactorial Adaptive Platform Trial for Community-Acquired Pneumonia. Strategy 1 consists of conventional tests for HTE using subgroup analyses. Differences in treatment effect among subgroups are evaluated in a regression model with an independent variable representing any one of several potential effect modifiers (patient, disease, or management characteristics) that might influence treatment effect. A separate model is computed for each potential effect modifier. Strategy 2 is a risk-based approach to test for HTE according to risk of outcomes (OSFDs and hospital survival) estimated using a risk model derived and internally validated in the trial data. Patient and disease characteristics are handled as predictors of outcome (candidate risk predictors) and combined in a single risk model to compute a single candidate effect modifier, the predicted risk of outcome. See Methods section of text and the eAppendix in Supplement 1 for details. Strategy 3 is an effect-based approach to test for HTE according to predicted treatment effect computed from a model combining multiple variables potentially associated with treatment effect (trained on part of the data, the training data set) and comparing predicted vs observed treatment effect on the remaining data (test data set). Patient and disease characteristics are used to compute a model of the difference in outcome with and without treatment, which is used to compute a single candidate effect modifier, the predicted treatment effect.
Conventional Subgroup Analysis Approach
Differences in treatment effect were explored within prespecified clinical subgroups defined by variables that were deemed to potentially moderate treatment effect (age, sex, race and ethnicity [which were ascertained by patient self-report based on selection from fixed categories with the option of “other”], body mass index [BMI, calculated as weight in kilograms divided by height in meters squared], illness severity [moderate vs severe], baseline respiratory support requirements, baseline D-dimer, prerandomization thromboprophylaxis dosing, and time since hospital admission), using conventional subgroup analysis. In this approach, the influence of each potential effect modifier on treatment effect is evaluated one at a time in separate models, in isolation from all other potential effect modifiers. Race and ethnicity were ascertained by patient self-report from a selection of fixed categories as these characteristics may influence both the risk of outcome and the magnitude of treatment effect. The treatment effect in each subgroup (cATE) was estimated using a bayesian cumulative logistic regression model with a statistical interaction (product term) with treatment effect to calculate the proportional odds ratio (OR) for organ support–free days, where an OR greater than 1 indicated treatment benefit and an OR less than 1 indicated treatment harm.
Each subgroup analysis model was adjusted for illness severity, age, sex, baseline D-dimer, trial site/country, and enrollment time period (in 2-week intervals). Weakly informative Dirichlet prior probability distributions were specified for the baseline probabilities for each value of organ support–free days. Independent standard normal priors were specified for the treatment effects in each subgroup analysis. The model was fit using a Markov chain Monte Carlo algorithm with 100 000 samples from the joint posterior distribution, which allowed calculation of the posterior distributions for the ORs, including medians and 95% credible intervals, and the posterior probability of superiority of therapeutic-dose heparin (indicated by an OR >1). The magnitude of statistical evidence for differences in treatment effect was quantified in each subgroup analysis by computing the posterior probability that the OR was greater in one subgroup vs the others. Analyses were repeated for the secondary outcome of survival to discharge.
Risk-Based Approach
Differences in treatment effect were explored across levels of risk of poor outcome (effect modification by predicted risk) using previously described methods.13,14 First, risk was estimated for each patient, as follows. A cumulative logistic regression risk prediction model for organ support–free days was internally derived from the population using a prespecified set of candidate risk predictors; final variables for inclusion were selected based on the model with the lowest Bayes information criterion among models with all possible combinations of the following: age, sex, race, BMI, cardiac history, diabetes, chronic kidney disease, chronic respiratory disease, chronic liver disease, immunosuppression, baseline vasopressor requirement, baseline respiratory support, D-dimer, neutrophil count, lymphocyte count, creatinine, and platelet count. Statistical interactions (product terms) between candidate risk predictors were not tested because it was not computationally possible to test all possible combinations. Missing data were multiply imputed using a multivariate imputation by chained equations algorithm. The model was used to compute risk for each patient in terms of a risk score, computed as the linear combination of their covariate values and the final risk model coefficients (log OR), multiplied by –1 to obtain a positive value. Patients were ranked by risk score and grouped by deciles of risk.
The risk score was then evaluated as a potential effect modifier by computing cATE within each decile of risk, specifying a statistical interaction (product term) between risk decile and treatment assignment in a bayesian cumulative logistic regression model (constructed as described above for the conventional subgroup analysis models). Smoothing of the treatment effects across quantiles of risk score was induced with a first-order normal dynamic linear model prior distribution (dynamic borrowing). Additional analyses were performed with independent priors to estimate the treatment effects by decile without smoothing (borrowing).
Effect-Based Approach
Differences in treatment effect were evaluated across levels of predicted treatment effect (effect modification by predicted effect) using a nonparametric causal forest method (eFigure 1 in Supplement 1).20 Because causal forest methods are most easily applied to binary outcomes, this analysis used survival status at hospital discharge (alive or dead) as the outcome of interest. The treatment effect was quantified in terms of the absolute difference in survival rate. In this application, the causal forest method estimates cATE for each patient as a conditional absolute rate difference (cARD), the difference between weighted survival outcome averages among treated and control patients with similar values for potential effect modifiers (the more similar the values, the higher the weights).
Before applying the causal forest method and to reduce computational complexity for identifying effect modifiers in a modest sample size, a prognostic model of the probability of survival at hospital discharge was computed by random forest modeling using the same set of candidate risk predictors used to construct the risk-based model. The causal forest model of treatment effect on hospital survival was constructed using the predicted probability of survival for each patient along with all candidate effect modifiers. The resulting model estimated the cARD for each patient. Positive cARD values indicate a predicted improvement in survival due to therapeutic-dose heparin, while negative values indicate a predicted decrease in survival.
A specialized Monte-Carlo cross-validation procedure with 100 repetitions was used to estimate the observed absolute difference in rate of hospital survival within deciles of predicted cARD, infer the 95% confidence limits, and test relevant null hypotheses.21 Specifically, we tested whether the observed absolute differences in the rate of survival increased monotonically across deciles of the cARD predicted for each patient, and whether the observed difference in rate of survival in the lowest decile of predicted cARD was significantly different from the rest (this latter test was defined post hoc).
Contributions of each candidate effect modifier to cARD prediction were quantified by Shapley additive explanation (SHAP) scores,22 which allocate credit for each cARD prediction among the variables. Specifically, a SHAP score captures the difference in the cARD attributable to the difference in the candidate effect modifier value between patients. For example, a BMI SHAP of −0.06 for a 1-unit increase in BMI is associated with an absolute 6% decrease in the cARD. See the eAppendix in Supplement 1 for details. To further characterize patients in the lowest decile and compare them with others, we compared observed baseline characteristics between groups.
All statistical analyses are detailed in the eAppendix in Supplement 1 and were performed using R version 4.1.3 (R Foundation) and Stan version 2.21.0 (NumFOCUS). The statistical analysis plan for this study is available in Supplement 2.
Results
The original primary mpRCT analyses included 1098 patients hospitalized with severe COVID-19 and 2219 with moderate COVID-19. Since publication, the primary end point was ascertained for 4 additional patients with severe COVID-19, and 1 patient with moderate COVID-19 was removed from the analysis set because they were double-counted in 2 platforms. Thus, 3320 patients were included for analysis in the present study. Baseline characteristics for the population are shown in Table 1, with missing data summarized in eTable 2 in Supplement 1. In the overall population (n = 3320), the median number of organ support–free days was 22 (IQR, 8-22 days); 17.3% of patients (n = 575) died in the hospital. In patients with moderate COVID-19 (n = 2218), the median number of organ support–free days was 22 (IQR, 22-22 days) and hospital mortality was 7.8% (n = 173). In patients with severe COVID-19 (n = 1102), the median number of organ support–free days was 3 (IQR, –1 to 16 days) and hospital mortality was 36.5% (n = 402).
Table 1. Characteristics of Participants in the Multiplatform Randomized Clinical Trial of Therapeutic-Dose Heparin for Moderate and Severe COVID-19 Enrolled Between April 2020 and January 2021 for Whom Primary Outcome Is Known.
| Characteristics | Therapeutic-dose heparin (n = 1707) | Usual care pharmacologic thromboprophylaxis (n = 1613) |
|---|---|---|
| Age, median (IQR), y | 60 (50-69) | 60 (51-69) |
| Sex, No. (%) | ||
| Female | 613 (36) | 633 (39) |
| Male | 1094 (64) | 980 (61) |
| Race, No. (%)a | ||
| Aboriginal | 111 (7) | 80 (5) |
| Asian | 107 (6) | 108 (7) |
| Black | 224 (13) | 165 (10) |
| White | 928 (54) | 892 (55) |
| Other | 127 (8) | 145 (9) |
| Unknown | 210 (12) | 223 (14) |
| Hispanic ethnicity, No./total (%) | 479/1073 (45) | 438/963 (45) |
| Baseline COVID-19 severity state, No. (%)b | ||
| Moderate | 1171 (69) | 1047 (65) |
| Severe | 536 (31) | 566 (35) |
| Region of enrollment, No. (%) | ||
| North America | 873 (51) | 801 (50) |
| Europe | 585 (34) | 595 (37) |
| South America | 245 (14) | 213 (13) |
| Australia/Asia | 4 (<1) | 4 (<1) |
| Body mass index, median (IQR)c | 30.0 (26.5-35.2) | 30.2 (26.6-34.9) |
| Comorbidities, No. (%) | ||
| Cardiovascular diseased | 658 (19.8) | 581 (17.5) |
| Diabetes | 521 (15.7) | 501 (15.1) |
| Chronic respiratory diseasee | 359 (10.8) | 330 (9.9) |
| Chronic kidney disease | 139 (4.2) | 112 (3.4) |
| Immunosuppressive diseasef | 118 (3.6) | 119 (3.6) |
| Chronic liver disease | 20 (0.6) | 14 (0.4) |
| Vasopressor use at baseline, No. (%) | 95 (2.9) | 109 (3.3) |
| Respiratory support at baseline, No. (%) | ||
| None/supplementary | 1130 (66) | 998 (62) |
| High-flow nasal oxygen | 202 (12) | 223 (14) |
| Noninvasive ventilation | 237 (14) | 225 (14) |
| Invasive mechanical ventilation | 138 (8) | 167 (10) |
| Prerandomization thromboprophylaxis dosing, No. (%)g | ||
| Low | 522 (31) | 479 (30) |
| Intermediate | 138 (8) | 156 (10) |
| Subtherapeutic | 18 (1) | 8 (<1) |
| Therapeutic | 51 (3) | 13 (1) |
| Unknown | 978 (57) | 957 (59) |
| Laboratory values, median (IQR) | ||
| D-dimer ratio, upper limit of normal at site | 1.67 (1.00-2.82) | 1.63 (1.00-2.94) |
| Neutrophil count, × 109/L | 6.06 (4.00-8.90) | 6.10 (4.05-9.00) |
| Lymphocyte count, × 109/L | 0.81 (0.60-1.20) | 0.85 (0.60-1.25) |
| Creatinine, mg/dL | 0.89 (0.72-1.13) | 0.86 (0.70-1.09) |
| Platelet count, × 109/L | 229 (176-300) | 227 (175-298) |
Race was ascertained by patient self-report from a selection of fixed categories. When multiple races were selected, race is reported as “other.”
Severe COVID-19 was defined as a need for intensive care unit–level care with organ support at baseline, including high-flow nasal oxygen greater than 20 L/min, noninvasive ventilation, invasive mechanical ventilation, or vasopressors. Moderate COVID-19 was defined as hospitalization for COVID-19 without need for intensive care unit–level of care with organ support.
Calculated as weight in kilograms divided by height in meters squared.
Cardiovascular disease was defined as baseline history of heart failure, myocardial infarction, coronary artery disease, peripheral arterial disease, or cerebrovascular disease (stroke or transient ischemic attack) in the ATTACC (Antithrombotic Therapy to Ameliorate Complications of COVID-19) platform and ACTIV-4a (Multicenter, Adaptive, Randomized Controlled Platform Trial of the Safety and Efficacy of Antithrombotic Strategies in Hospitalized Adults With COVID-19) and as a baseline history of New York Heart Association class IV symptoms in REMAP-CAP (Randomized, Embedded, Multifactorial Adaptive Platform Trial for Community-Acquired Pneumonia).
Chronic respiratory disease was defined as a baseline history of asthma, chronic obstructive pulmonary disease, bronchiectasis, interstitial lung disease, primary lung cancer, pulmonary hypertension, active tuberculosis, or receipt of home oxygen therapy.
Immunosuppressive disease was defined as concurrently having any of the following conditions: HIV, leukemia, metastatic cancer, myeloma, lupus, multiple sclerosis, rheumatoid arthritis, psoriatic arthritis, Crohn disease, granulomatosis with polyangiitis, sarcoidosis, monoclonal gammopathy of unknown significance, ankylosing spondylitis, and psoriasis; or as receiving chemotherapy or radiation, transplant, or high-dose or long-term steroid treatment.
Prerandomization anticoagulants were categorized into 1 of 4 dose intensity categories, which increase in dose intensity level from low (lowest), to intermediate, to subtherapeutic, to therapeutic (highest). Dosing equivalent categories were chosen based on consensus and also considering guidance from the American Society of Hematology and were assigned retrospectively.2
In the overall mpRCT population, therapeutic-dose heparin was not associated with an increase in organ support–free days (median value for the posterior distribution of the OR, 1.05; 95% credible interval, 0.91-1.22).
Conventional Subgroup Analysis
The effect of therapeutic-dose heparin on organ support–free days differed substantially between moderate and severe COVID-19 groups (median OR, 1.29 vs 0.85; posterior probability of difference in OR, 99.8%), which the original trials defined as not requiring (moderate) vs requiring (severe) critical care–level organ support at baseline; it also differed between males and females (median OR, 1.16 vs 0.86; posterior probability of difference in OR, 96.4%) and between the subgroup with a BMI of less than 30 in comparison with subgroups with a BMI of 30 or higher (posterior probability of difference in OR, >90% for all comparisons) (Figure 2). The remaining subgroup analyses are presented in Figure 2 and in eTable 3 in Supplement 1. Body mass index was distributed similarly between patients with moderate COVID-19 (median, 30.0; IQR, 26.5-34.8) and severe COVID-19 (median, 30.4; IQR, 26.6-35.4).
Figure 2. Heterogeneity of Treatment Effect Evaluation by Conventional Subgroup Analysis.
CrI indicates credible interval; OR, odds ratio. aPosterior probability of an OR greater than 1. bBody mass index is calculated as weight in kilograms divided by height in meters squared.
Similarly, the effect of therapeutic-dose heparin on hospital survival differed substantially between moderate and severe COVID-19 groups (median OR, 1.22 vs 0.82; posterior probability of difference in OR, 96.3%), between patients with a BMI of less than 30 and those with a higher BMI (posterior probability of difference in OR, >90% for all comparisons), and according to baseline respiratory support (posterior probability of difference in OR, >90% for all comparisons) (Figure 2). Treatment effect on hospital survival was not meaningfully different between females and males (median OR, 0.85 vs 1.02; posterior probability of difference in OR, 79%).
Risk-Based Approach
The estimated ORs (and log ORs) for each variable included in the final derived model of cumulative odds of organ support–free days are shown in Table 2. The risk of having a low value for organ support–free days was distributed bimodally across the mpRCT population (eFigure 2 in Supplement 1) and within each of the participating platforms (eFigure 3 in Supplement 1). Patients were ranked by predicted risk and grouped by decile (eTable 4 in Supplement 1). Increasing risk score was associated with progressively lower values for organ support–free days; in the lowest decile, the risk of death or organ dysfunction was very low while the proportion of patients with death or prolonged organ failure was highest in the highest decile, with a consistent gradient across deciles (eFigure 4 in Supplement 1).
Table 2. Risk Model for Organ Support–Free Days Used in Risk-Based Heterogeneity of Treatment Effect Analysis.
| Model variablea | Risk model coefficients | |
|---|---|---|
| Odds ratio (95% CI)b | Log odds ratioc | |
| Age, per 10 y | 0.73 (0.68-0.77) | −0.32 |
| Female sex | 1.29 (1.10-1.51) | 0.25 |
| Body mass index, per 5 units | 0.92 (0.88-0.97) | −0.08 |
| Diabetes | 0.78 (0.67-0.91) | −0.25 |
| Immunosuppressive disease or therapyd | 0.58 (0.44-0.77) | −0.54 |
| Baseline organ support | ||
| Vasopressor requirement | 0.49 (0.35-0.70) | −0.71 |
| High-flow nasal oxygen | 0.07 (0.05-0.08) | −2.70 |
| Noninvasive ventilation | 0.05 (0.04-0.06) | −3.10 |
| Invasive mechanical ventilation | 0.04 (0.03-0.05) | −3.29 |
| Neutrophil count, per 5 × 109/L | 0.74 (0.68-0.81) | −0.30 |
| Lymphocyte count, per 5 × 109/L | 1.35 (1.10-1.66) | 0.30 |
| Platelet count, per 25 × 109/L | 1.07 (1.04-1.09) | 0.06 |
Unadjusted for treatment assignment, and no statistical interactions between model variables were specified in the risk model.
Confidence intervals are reported because the internally derived risk scores were derived from a frequentist ordinal logistic model.
The log odds ratio is used to compute the risk score components.
Immunosuppressive disease or therapy was defined as concurrently having any of the following conditions: HIV, leukemia, metastatic cancer, myeloma, lupus, multiple sclerosis, rheumatoid arthritis, psoriatic arthritis, Crohn disease, granulomatosis with polyangiitis, sarcoidosis, monoclonal gammopathy of unknown significance, ankylosing spondylitis, and psoriasis; or as receiving chemotherapy or radiation, transplant, or high-dose or long-term steroid treatment.
The effect of therapeutic-dose heparin on organ support–free days and survival to hospital discharge for each risk score decile is shown in Figure 3. The observed treatment effect in the 60% of the cohort at lowest risk (groups 1-6; risk score ≤3.46) suggested probable benefit from therapeutic-dose heparin (posterior probability of OR >1, >80%). The observed treatment effect in the 30% at highest risk (groups 8-10; risk score >5.14) suggested that harm from therapeutic-dose heparin was more probable than not (posterior probability of OR >1, <50%). A similar pattern of results was obtained from estimates of cATE obtained without dynamic borrowing (eFigure 5 in Supplement 1) and for survival to hospital discharge (Figure 3). The implications of risk-based HTE in terms of absolute risks for poor clinical outcomes are shown for representative patients with varying risk profiles in eTable 5 in Supplement 1; although patients at lowest risk exhibited the highest relative benefit (in terms of OR) (Figure 3), the absolute risk difference is relatively constant for representative patients with low or moderate risk scores because treatment effect varies inversely with risk. In a representative patient with a high risk score, treatment increases the absolute risk of a poor outcome.
Figure 3. Heterogeneity of Treatment Effect Evaluation by Risk-Based Analysis.
CrI indicates credible interval; OR, odds ratio. Risk-based heterogeneity of treatment effect for organ support–free days is shown by risk deciles (ranging from lowest, group 1, to highest, group 10). All patients in risk groups 8 through 10 required respiratory organ support at baseline vs 2 of 1992 (0.1%) patients in risk groups 1 through 6 (see eTable 2 in Supplement 1). Clinical benefit was deemed substantially more probable than not (posterior probability of an OR >1 above 80%) in risk groups 1 through 6. A similar pattern was observed for hospital survival, although the posterior probability of benefit from heparin was lower for hospital survival vs organ support–free days.
aPosterior probability of an OR greater than 1.
The level of respiratory support at baseline was a major determinant of risk group assignment. All 996 patients in risk groups 8 through 10 (Figure 3) were receiving either high-flow nasal oxygen, noninvasive ventilation, or invasive mechanical ventilation at baseline (eFigure 6 in Supplement 1), whereas very few patients in risk groups 1 through 6 were receiving these respiratory support strategies at baseline (2/1992 [0.1%]). Most patients receiving high-flow nasal oxygen were in risk groups 8 through 10 (292/425 [69%]).
Effect-Based Approach
The distribution of observed absolute differences in rates of hospital survival obtained by cross-validation in each decile of predicted cARD is shown in Figure 4. Observed hospital survival did not monotonically increase with deciles of predicted benefit (P = .38). The point estimate for absolute risk reduction in the lowest decile suggested a non–statistically significant association between therapeutic-dose heparin and possible harm (absolute risk reduction, –5.7%; 95% CI, –22.4% to 10.6%), and the effect of therapeutic-dose heparin on mortality differed between patients in the lowest decile in comparison with all others (post hoc P = .05 for statistical interaction). SHAP scores indicated that lower predicted risk of death and lower BMI exhibited the strongest associations with greater cARD (eFigure 7 in Supplement 1). Patients in the lowest cARD decile group (in whom the treatment was associated with possible harm) (Figure 4) tended to have high BMI and were more likely to require ICU admission at baseline (eTable 6 in Supplement 1).
Figure 4. Heterogeneity of Treatment Effect Evaluation by Effect-Based Analysis.
Effect-based heterogeneity of treatment effect for hospital survival shown by deciles of predicted conditional absolute rate difference (cARD) in hospital survival derived from repeated cross-validation using a causal machine-learning algorithm (n = 100 repetitions).
Discussion
Using 3 analytic strategies, this exploratory analysis of an mpRCT of therapeutic-dose heparin for patients hospitalized for COVID-19 found that the treatment effect varied substantially according to the baseline risk of poor outcome, primarily reflected by the degree of organ support required at baseline. The threshold level of risk that defined whether heparin was more likely to be beneficial or harmful appears to coincide with the transition from moderate COVID-19 (no organ support at baseline) to severe COVID-19 (requirement for respiratory or cardiovascular organ support at baseline). Higher BMI was also associated with a higher risk of harm with therapeutic-dose heparin. Importantly, baseline severity of illness and BMI were associated with differential treatment effect irrespective of the analytical approach used to characterize HTE. This consistency is noteworthy in view of the previously reported differences in results obtained by different HTE modeling strategies17 and supports the relevance of these characteristics to inform decisions about treatment with therapeutic-dose heparin in patients hospitalized for COVID-19, despite the exploratory nature of the analysis.
The mpRCT was designed to estimate treatment effect according to severity of illness and, among moderately ill patients, baseline D-dimer level. This design decision was motivated in part by anticipation of potential HTE based on these factors, and partly by the fact that therapeutic-dose heparin was conceptualized as a “treatment” for severe COVID-19 in patients with severe illness and as a means to prevent progression to severe illness in patients with moderate COVID-19. Nevertheless, the trial was predicated on anticipated potential benefit in both moderately and severely ill patients; if anything, prior observational evidence was suggestive of greater potential risk of thrombosis in critically ill patients, suggesting greater potential benefit.23 In retrospect, this trial design decision was fortuitous, as the present analysis suggests that the distinction between moderate and severe COVID-19 appears to differentiate patients who derive benefit or harm from therapeutic-dose heparin. If the trial had been primarily designed to enroll a broad cohort of all hospitalized patients, regardless of severity of illness, it would likely have reported a single average treatment effect indicative of neither benefit nor harm.
To appraise the credibility of observed HTE, it is important both to consider the consistency of the result across multiple analytical techniques and to establish whether there is a plausible mechanistic basis for HTE.24 Heterogeneity of treatment effect according to baseline severity of illness was observed across the 3 strategies used to evaluate HTE in this study. This suggests that therapeutic-dose heparin should be avoided in severely ill patients requiring organ support and that its benefit is limited to hospitalized patients who do not require organ support at baseline. Of note, this analysis cannot determine whether heparin should be continued or discontinued when patients transition from moderate to severe illness. Elevated BMI was also identified as an important predictor of harm from therapeutic-dose heparin across all 3 HTE strategies. Because BMI was unrelated to severity state, clinicians may consider shifting the risk-benefit decision-making for patients with a BMI greater than 30 in patients with moderate COVID-19. Finally, subgroup analyses suggested that female sex was associated with a low probability of benefit from therapeutic-dose heparin and male sex was associated with a high probability of benefit. Sex was not an influential contributor to HTE in the risk-based and effect-based analyses.
Limitations
The mechanisms accounting for HTE according to baseline risk and severity of illness, BMI, and sex are uncertain. It is possible that the putative effect of heparin on inflammation and thrombosis may only be relevant before organ injury and dysfunction are established. Obesity is generally associated with an increased risk of thrombosis and vascular inflammation,25,26 particularly in combination with COVID-19,27 so one might expect therapeutic-dose heparin to be beneficial rather than harmful in this population. However, dosing heparin to target therapeutic levels may be challenging in patients with high BMI. Observational data suggest that among patients hospitalized for COVID-19, male patients are at higher risk of thrombosis and death in comparison with female patients, and the excess mortality in male patients was attributable to higher thrombotic risk.28 This observation might account for the observed differences in treatment effect by sex, although the underlying sex-specific mechanism remains uncertain.
The generalizability of the mpRCT results generated during the first year of the pandemic to the contemporary management of COVID-19 warrants consideration. Although rising population immunity due to widespread vaccination and infection and the circulation of less virulent variants of SARS-CoV-2 have markedly reduced the risk of developing severe disease, the mechanisms (such as exaggerated host thrombotic and inflammatory responses) responsible for organ dysfunction and death among patients who do develop severe disease are likely unchanged. The putative benefits of heparin therefore probably remain relevant for some patients, although the continued evolution of SARS-CoV-2 bears accounting for when generating inferences from trials conducted early in the pandemic. The findings of the present analysis highlight the importance of assessing risk in determining whether therapeutic-dose heparin should be prophylactically administered in patients with moderate COVID-19 without established thrombosis, and the risk score and effect-based predictors provide a basis for conducting this risk assessment.
The results of the multiplatform trial illustrate the importance of considering HTE in the design of trials and the utility of adaptive trial design methods to prospectively account for HTE. Under traditional approaches to trial design, the primary “cost” of designing a trial to estimate separate treatment effects in independent strata within the trial population is a marked increase in sample size requirement, an increase that may ultimately be unnecessary if the treatment effect is actually similar across strata. Such designs may be perceived as a “risk” in terms of increased costs and time to reach trial conclusions. Adaptive trial designs using bayesian hierarchical statistical models can mitigate this risk by dynamically borrowing information about treatment effect between subgroups. This entails that there is no major adverse effect on sample size requirement unless there is substantial HTE, in which case divergent treatments effect can be identified and reported, as in the mpRCT.
Because the sources and determinants of HTE are often unknown at the outset of clinical trials, it is challenging to prospectively account for HTE in trial design. Future innovations in adaptive trial design could utilize early trial phases to apply data-driven techniques for risk-based and effect-based modeling on initial trial data to detect strong HTE signals. These signals could drive adaptations in the trial design to account for HTE discovered during the trial.
Conclusions
The effect of therapeutic-dose heparin administered prophylactically in patients hospitalized for COVID-19 appears to vary substantially according to the baseline risk of having a poor outcome, the severity of COVID-19 at baseline, and BMI. Patients with moderate COVID-19 who require only low-flow supplemental oxygen (or none) are more likely to benefit from treatment; patients with severe COVID-19 who require organ support or ICU-level care do not benefit from therapeutic-dose heparin. The findings illustrate the importance of considering HTE in the design and analysis of clinical trials.
Section Editor: Christopher Seymour, MD, Associate Editor, JAMA (christopher.seymour@jamanetwork.org).
eAppendix. Additional Methods
eFigure 1. Cross-validated Estimation and Inference for Average Treatment Effects Per Decile of Predicted Individual Absolute Risk Reduction
eFigure 2. Distribution of Risk Scores in mpRCT Population
eFigure 3. Distribution of Risk Scores for Severity States in the mpRCT
eFigure 4. Distribution of OSFDs and Risk of Death (OSFD = –1) for Risk Deciles
eFigure 5. Risk-Based Heterogeneity of Treatment Effect Without Dynamic Borrowing
eFigure 6. Utilization of Respiratory Support at the Time of Randomization According to Risk Decile
eFigure 7. SHAP Distributions in Effect-Based Heterogeneity of Treatment Effect Modelling
eTable 1. Cross-trial Comparison of Trial Design
eTable 2. Missing Data in Variables Considered for HTE Analysis
eTable 3. Additional Subgroup Analyses
eTable 4. Distribution of Risk Scores for Each Decile of Risk Defining the Risk Groups
eTable 5. Comparison of Baseline Covariates Between a Group Consisting of the Lowest 10% of Predicted iARRs From Effect-Based Modeling, and a Group Consisting of the Remaining 90% of Patients
eTable 6. Comparison of Baseline Covariates Between a Group Consisting of the Lowest 10% of Predicted cARD From Effect-Based Modeling, and a Group Consisting of the Remaining Patients
Statistical Analysis Plan
Nonauthor Collaborators. Investigators for the REMAP-CAP, ATTACC, and ACTIV-4a Trials
Data Sharing Statement
References
- 1.Godoy LC, Goligher EC, Lawler PR, Slutsky AS, Zarychanski R. Anticipating and managing coagulopathy and thrombotic manifestations of severe COVID-19. CMAJ. 2020;192(40):E1156-E1161. doi: 10.1503/cmaj.201240 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lawler PR, Goligher EC, Berger JS, et al. ; ATTACC Investigators; ACTIV-4a Investigators; REMAP-CAP Investigators . Therapeutic anticoagulation with heparin in noncritically ill patients with Covid-19. N Engl J Med. 2021;385(9):790-802. doi: 10.1056/NEJMoa2105911 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sholzberg M, Tang GH, Rahhal H, et al. ; RAPID Trial Investigators . Effectiveness of therapeutic heparin versus prophylactic heparin on death, mechanical ventilation, or intensive care unit admission in moderately ill patients with covid-19 admitted to hospital: RAPID randomised clinical trial. BMJ. 2021;375(2400):n2400. doi: 10.1136/bmj.n2400 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Goligher EC, Bradbury CA, McVerry BJ, et al. ; REMAP-CAP Investigators; ACTIV-4a Investigators; ATTACC Investigators . Therapeutic anticoagulation with heparin in critically ill patients with Covid-19. N Engl J Med. 2021;385(9):777-789. doi: 10.1056/NEJMoa2103417 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Sadeghipour P, Talasaz AH, Rashidi F, et al. ; INSPIRATION Investigators . Effect of intermediate-dose vs standard-dose prophylactic anticoagulation on thrombotic events, extracorporeal membrane oxygenation treatment, or mortality among patients with COVID-19 admitted to the intensive care unit: the INSPIRATION randomized clinical trial. JAMA. 2021;325(16):1620-1630. doi: 10.1001/jama.2021.4152 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Spyropoulos AC, Goldin M, Giannis D, et al. ; HEP-COVID Investigators . Efficacy and safety of therapeutic-dose heparin vs standard prophylactic or intermediate-dose heparins for thromboprophylaxis in high-risk hospitalized patients with COVID-19: the HEP-COVID randomized clinical trial. JAMA Intern Med. 2021;181(12):1612-1620. doi: 10.1001/jamainternmed.2021.6203 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Iwashyna TJ, Burke JF, Sussman JB, Prescott HC, Hayward RA, Angus DC. Implications of heterogeneity of treatment effect for reporting and analysis of randomized trials in critical care. Am J Respir Crit Care Med. 2015;192(9):1045-1051. doi: 10.1164/rccm.201411-2125CP [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Angus DC, Chang CH. Heterogeneity of treatment effect: estimating how the effects of interventions vary across individuals. JAMA. 2021;326(22):2312-2313. doi: 10.1001/jama.2021.20552 [DOI] [PubMed] [Google Scholar]
- 9.Rothwell PM. Can overall results of clinical trials be applied to all patients? Lancet. 1995;345(8965):1616-1619. doi: 10.1016/S0140-6736(95)90120-5 [DOI] [PubMed] [Google Scholar]
- 10.Seymour CW, Gomez H, Chang CH, et al. Precision medicine for all? challenges and opportunities for a precision medicine approach to critical illness. Crit Care. 2017;21(1):257. doi: 10.1186/s13054-017-1836-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Rothwell PM. Treating individuals 2: subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet. 2005;365(9454):176-186. doi: 10.1016/S0140-6736(05)17709-5 [DOI] [PubMed] [Google Scholar]
- 12.Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet. 2000;355(9209):1064-1069. doi: 10.1016/S0140-6736(00)02039-0 [DOI] [PubMed] [Google Scholar]
- 13.Kent DM, Rothwell PM, Ioannidis JPA, Altman DG, Hayward RA. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials. 2010;11(1):85. doi: 10.1186/1745-6215-11-85 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA. 2007;298(10):1209-1212. doi: 10.1001/jama.298.10.1209 [DOI] [PubMed] [Google Scholar]
- 15.Kent DM, Paulus JK, van Klaveren D, et al. The Predictive Approaches to Treatment Effect Heterogeneity (PATH) statement. Ann Intern Med. 2020;172(1):35-45. doi: 10.7326/M18-3667 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Athey S, Imbens G. Recursive partitioning for heterogeneous causal effects. Proc Natl Acad Sci U S A. 2016;113(27):7353-7360. doi: 10.1073/pnas.1510489113 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.van Klaveren D, Balan TA, Steyerberg EW, Kent DM. Models with interactions overestimated heterogeneity of treatment effects and were prone to treatment mistargeting. J Clin Epidemiol. 2019;114:72-83. doi: 10.1016/j.jclinepi.2019.05.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Houston BL, Lawler PR, Goligher EC, et al. Anti-Thrombotic Therapy to Ameliorate Complications of COVID-19 (ATTACC): study design and methodology for an international, adaptive bayesian randomized controlled trial. Clin Trials. 2020;17(5):491-500. doi: 10.1177/1740774520943846 [DOI] [PubMed] [Google Scholar]
- 19.Angus DC, Berry S, Lewis RJ, et al. The REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) study: rationale and design. Ann Am Thorac Soc. 2020;17(7):879-891. doi: 10.1513/AnnalsATS.202003-192SD [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Wager S, Athey S. Estimation and inference of heterogeneous treatment effects using random forests. J Am Stat Assoc. 2018;113(523):1228-1242. doi: 10.1080/01621459.2017.1319839 [DOI] [Google Scholar]
- 21.Chernozhukov V, Demirer M, Duflo E, Fernández-Val I. Generic Machine Learning Inference on Heterogenous Treatment Effects in Randomized Experiments. National Bureau of Economic Research; 2018. [Google Scholar]
- 22.Lundberg SM, Erion G, Chen H, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2(1):56-67. doi: 10.1038/s42256-019-0138-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Bilaloglu S, Aphinyanaphongs Y, Jones S, Iturrate E, Hochman J, Berger JS. Thrombosis in hospitalized patients with COVID-19 in a New York City health system. JAMA. 2020;324(8):799-801. doi: 10.1001/jama.2020.13372 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Schandelmaier S, Briel M, Varadhan R, et al. Development of the Instrument to Assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ. 2020;192(32):E901-E906. doi: 10.1503/cmaj.200077 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Saltiel AR, Olefsky JM. Inflammatory mechanisms linking obesity and metabolic disease. J Clin Invest. 2017;127(1):1-4. doi: 10.1172/JCI92035 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Blokhin IO, Lentz SR. Mechanisms of thrombosis in obesity. Curr Opin Hematol. 2013;20(5):437-444. doi: 10.1097/MOH.0b013e3283634443 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Vaughan CJ, Cronin H, Ryan PM, Caplice NM. Obesity and COVID-19: a Virchow’s triad for the 21st century. Thromb Haemost. 2020;120(11):1590-1593. doi: 10.1055/s-0040-1714216 [DOI] [PubMed] [Google Scholar]
- 28.Cohen KR, Anderson D, Ren S, Cook DJ. Contribution of the elevated thrombosis risk of males to the excess male mortality observed in COVID-19: an observational study. BMJ Open. 2022;12(2):e051624. doi: 10.1136/bmjopen-2021-051624 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eAppendix. Additional Methods
eFigure 1. Cross-validated Estimation and Inference for Average Treatment Effects Per Decile of Predicted Individual Absolute Risk Reduction
eFigure 2. Distribution of Risk Scores in mpRCT Population
eFigure 3. Distribution of Risk Scores for Severity States in the mpRCT
eFigure 4. Distribution of OSFDs and Risk of Death (OSFD = –1) for Risk Deciles
eFigure 5. Risk-Based Heterogeneity of Treatment Effect Without Dynamic Borrowing
eFigure 6. Utilization of Respiratory Support at the Time of Randomization According to Risk Decile
eFigure 7. SHAP Distributions in Effect-Based Heterogeneity of Treatment Effect Modelling
eTable 1. Cross-trial Comparison of Trial Design
eTable 2. Missing Data in Variables Considered for HTE Analysis
eTable 3. Additional Subgroup Analyses
eTable 4. Distribution of Risk Scores for Each Decile of Risk Defining the Risk Groups
eTable 5. Comparison of Baseline Covariates Between a Group Consisting of the Lowest 10% of Predicted iARRs From Effect-Based Modeling, and a Group Consisting of the Remaining 90% of Patients
eTable 6. Comparison of Baseline Covariates Between a Group Consisting of the Lowest 10% of Predicted cARD From Effect-Based Modeling, and a Group Consisting of the Remaining Patients
Statistical Analysis Plan
Nonauthor Collaborators. Investigators for the REMAP-CAP, ATTACC, and ACTIV-4a Trials
Data Sharing Statement