Abstract
Background
The clinical impact of different prophylactic anticoagulation regimens among hospitalized patients with coronavirus disease 2019 (COVID-19) remains unclear. We pooled evidence from available randomized controlled trials (RCTs) to provide insights on this topic.
Methods and results
We searched for RCTs comparing treatment with an escalated-dose (intermediate-dose or therapeutic-dose) vs. a standard-dose prophylactic anticoagulation regimen in critically and non-critically ill COVID-19 patients requiring hospitalization and without a formal indication for anticoagulation. The primary efficacy endpoint was all-cause death, and the primary safety endpoint was major bleeding. Seven RCTs were identified, including 5154 patients followed on an average of 33 days. Compared to standard-dose prophylactic anticoagulation, escalated-dose prophylactic anticoagulation was not associated with a reduction of all-cause death [17.8% vs. 18.6%; risk ratio (RR) 0.96, 95% confidence interval (CI) 0.78–1.18] but was associated with an increase in major bleeding (2.4% vs. 1.4%; RR 1.73, 95%CI 1.15–2.60). Compared to prophylactic anticoagulation used at a standard dose, an escalated dose was associated with lower rates of venous thromboembolism (2.5% vs. 4.7%; RR 0.55, 95%CI 0.41–0.74) without a significant effect on myocardial infarction (RR 0.80, 95%CI 0.47–1.36), stroke (RR 0.94, 95%CI 0.43–2.09), or systemic arterial embolism (RR 1.20, 95%CI 0.29–4.95). There were no significant interactions in the subgroup analysis for critically and non-critically ill patients.
Conclusions
Our findings provide comprehensive and high-quality evidence for the use of standard-dose prophylactic anticoagulation over an escalated-dose regimen as routine standard of care for hospitalized patients with COVID-19 who do not have an indication for therapeutic anticoagulation, irrespective of disease severity.
Study registration
This study is registered in PROSPERO (CRD42021257203).
Keywords: Anticoagulant therapy , Coronavirus disease 2019 , Death, Thrombosis, Bleeding
Introduction
Coronavirus disease 2019 (COVID‐19) is an infectious disease caused by severe acute respiratory syndrome coronavirus‐2 (SARS‐CoV‐2).1 By August 24, 2021, >212 million cases of COVID‐19 have been reported in 190 countries or regions, resulting in >4.96 million deaths.2 Among hospitalized patients, COVID-19 infection has been associated with endotheliitis, intense endothelial activation, inflammation, and coagulopathy.1 Patients with moderate or severe disease have consistently shown to have elevated D-dimer levels.1,3 Moreover, autopsy studies have found a high incidence of macro- and microthrombi, with thromboembolic events representing an important clinical manifestation.3 Accordingly, the use of prophylactic anticoagulation at a standard-dose regimen has been recommended for the prevention of thromboembolic events.3 In addition to their effects on micro- and macrothromboembolism, heparin-based products have also been suggested to have anti‐inflammatory and antiviral properties, including the ability to directly interact with the spike S1 protein of SARS‐CoV‐2.3
The occurrence of thromboembolic events despite standard-dose prophylactic anticoagulation and growing observations of heparin resistance have suggested the need for the use of prophylactic anticoagulation at an escalated dose for selected patients with COVID-19.4 Observational studies and a meta-analysis of observational studies have suggested that the use of prophylactic anticoagulation at an escalated dose, composed of intermediate- or therapeutic-dosing regimens, is associated with reduced mortality in critically ill COVID-19 patients with coagulopathy.5,6 However, randomized controlled trials (RCTs) assessing the safety and efficacy of prophylactic anticoagulation at an escalated dose compared with a standard dose have found contrasting results.7–13 Of note, none of these RCTs were powered to test the superiority for individual clinical endpoints such as all-cause death or major bleeding. Meta-analyses of RCTs can provide a more precise and powered estimation of the benefit of a specific anticoagulation dosing regimen on such hard and clinically relevant endpoints. Moreover, such estimates can generate interim insights to guide clinical practice and future research.
We therefore performed a systematic review and meta-analysis of available RCTs to determine the safety and efficacy of escalated-dose prophylactic vs. standard-dose prophylactic anticoagulation on critically and non-critically ill hospitalized patients with COVID-19 without a formal indication for therapeutic anticoagulation.
Methods
Search strategy and selection criteria
This meta-analysis was done according to the Cochrane Collaboration and Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (see Supplementary material online, Table S1).14 We included RCTs comparing treatment with escalated-dose (defined as intermediate-dose or therapeutic-dose) vs. a standard-dose prophylactic anticoagulation regimen in critically and non-critically ill COVID-19 patients without a formal indication for anticoagulation.
There was no restriction on the type of anticoagulation used. We included heparinoids [unfractionated heparin (UFH), low-molecular weight heparin (LMWH), or pentasaccharides], vitamin K antagonists (VKAs), direct oral anticoagulants (DOACs), and non-oral direct anticoagulants. Therapeutic, intermediate, or standard doses of anticoagulation regimens used for prophylaxis were defined according to the definitions of the trials [i.e. therapeutic-dose prophylactic anticoagulation: enoxaparin 1 mg/kg twice daily or UFH 10 000 UI subcutaneous (SC) three times a day, or rivaroxaban 20 mg daily; intermediate-dose prophylactic anticoagulation: enoxaparin 1 mg/kg SC daily or UFH 10 000 units SC twice daily; and standard-dose prophylactic anticoagulation: enoxaparin 40 mg SC daily or UFH 5000 units SC twice daily. If indicated, all doses were adjusted by creatinine clearance and body mass index]. For this meta-analysis, therapeutic- and intermediate-dosing regimens were pooled into one group and defined as escalated dose, which was compared to the standard-dose regimen. The rationale for this approach is that both the therapeutic and intermediate doses provide superior anticoagulant effect compared to the standard dose.15
Preprint (not peer-reviewed) status was not considered as exclusion criteria. In line with other meta-analysis related to COVID-19 treatment, we considered this strategy acceptable because of the urgency for timely evidence regarding effective treatment of COVID-19 patients.16 If a preprint article became published during the performance of this study, the analysis was updated with published data. From June 1 to July 15, 2021, we did a systematic digital search using MEDLINE (via PubMed), Cochrane, Embase, Web of Science databases, and medRxiv. In addition, we searched preprints, abstracts, presentations, unpublished data from annual meetings of the following societies: European Society of Cardiology, European Association of Percutaneous Cardiovascular Interventions, American Heart Association, American College of Cardiology, Transcatheter Cardiovascular Therapeutics, and Society of Cardiovascular Angiography and Interventions. Search terms were ‘COVID-19’, ‘SARS-CoV-2’, ‘anticoagulation’, ‘prophylactic’, ‘hospitalized’, ‘severe’, ‘critically-ill’, ‘non-critically-ill’, ‘therapeutic’, ‘moderate’, ‘intensive care unit’, ‘clinical trial’, in addition to combinations of these terms (see Supplementary material online, Table S2 for the full search strategy). An experienced medical librarian reviewed literature search terms. Two investigators (L.O.P., M.G.) independently screened titles and abstracts for eligibility as well as the full text, supplementary material, online appendices, and reference lists of each eligible study to confirm the inclusion criteria and to identify further published studies. The same two investigators independently performed data extraction. There were no restrictions with respect to the language used, publication status, or publication date. Disagreements were solved by consensus.
Data analysis
The risk of bias was independently assessed by two investigators (L.O.P., M.G.) according to the Cochrane Collaboration risk-of-bias-tool 2 (RoB 2).17 The primary efficacy endpoint was all-cause death at the longest follow-up available in the respective trials. Secondary efficacy endpoints were venous thromboembolism (VTE) (deep vein thrombosis or pulmonary embolism), myocardial infarction (MI), stroke, and systemic arterial embolism. The primary safety endpoint was major bleeding. Secondary safety outcomes were any bleeding and minor bleeding. Major, minor, or any bleeding were defined according to trial definitions. We prioritized the Bleeding Academic Research Consortium definition when available.18 If not reported, we chose the Thrombolysis in Myocardial Infarction criteria or International Society on Thrombosis and Haemostasis. Details on endpoint definitions are provided in the appendix (see Supplementary material online, Table S3).
Risk ratios (RRs) with 95% confidence intervals (95%CIs) were calculated with RevMan software version 5.3 (Cochrane Collaboration) to provide a practical interpretation of effect estimates. The Cochran's Q test and Higgins’ I² statistics were used to estimate heterogeneity among studies, with I² less than 25% indicating low heterogeneity, 25–50% indicating moderate heterogeneity, and more than 50% indicating high heterogeneity.17 For moderate-to-high heterogeneity, the random-effects model with inverse variance weighting was used, whereas the Mantel-Haenszel fixed-effect model was used for low heterogeneity. P-values less than 0.05 were considered significant.
All analyses were run according to the pre-specified subgroups of critically vs. non-critically ill patients to provide both an overall and a specific treatment effect estimate according to clinical status. A difference between the estimates of these subgroups was considered significant for Pinteraction < 0.10.19
To explore whether a single study significantly affected our overall findings, we ran a sensitivity analysis by sequentially removing each single study from the pooled effect estimates. Moreover, to address varying durations of follow-up across studies, we also performed a further sensitivity analysis by using incidence risk ratios (IRRs) and associated 95%CIs using R 3.6 (The R Project for Statistical Computing, Vienna). Furthermore, fixed- and random-effects meta-regression analyses were performed to evaluate the impact of concomitant antiplatelet therapy use on the treatment effects of anticoagulation regimens. The ‘number needed to treat’ (NNT) or the ‘number needed to harm’ (NNH) to prevent or cause and adverse event were calculated according to the absolute risk differences. The presence of publication bias was investigated by visual estimation of funnel plots. This study is registered with PROSPERO (CRD42021257203).
Results
Studies and patient characteristics
Using our search strategy, we screened 2408 potentially relevant articles. The PRISMA flow diagram describing the search and study selection process is available in Supplementary material online, Table S4. Seven relevant studies were identified. Our analyses included a total of 5154 patients: 562 from INSPIRATION7 comparing intermediate vs. standard prophylactic dose with LMWH/UFH in critically ill patients; 614 from ACTION8 comparing therapeutic- vs. standard-dose prophylactic anticoagulation with rivaroxaban (non-critically ill patients) or enoxaparin (critically ill patients); 1098 from REMAP-CAP, ACTIV-4a, and ATTACC12 comparing therapeutic- vs. standard-dose prophylactic anticoagulation with LMWH/UFH in critically ill patients; 2219 from REMAP-CAP, ACTIV-4a, and ATTACC13 comparing therapeutic- vs. standard-dose prophylactic anticoagulation with LMWH/UFH in non-critically ill patients; 20 from HESACOVID9 comparing therapeutic- vs. standard-dose prophylactic anticoagulation with LMWH/UFH in critically ill patients; RAPID10 comparing therapeutic- vs. standard-dose prophylactic anticoagulation with LMWH/UFH in non-critically ill patients; and 176 from Perepu et al.11 comparing intermediate- vs. standard-dose prophylactic anticoagulation with LMWH/UFH in critically ill patients. The mean follow-up duration was 33 days. INSPIRATION had the longest follow-up (90 days), whereas HESACOVID had the shortest (14 days). COVID-19 infection was confirmed by a polymerase chain reaction in the majority of trials. The median time from hospital admission to randomization was 4 days (interquartile range 2–4 days). The main trial characteristics are shown in Table 1. A summary of baseline and clinical characteristics of each trial is shown in Supplementary material online, Table S5. A detailed list of the different anticoagulation regimens and outcomes definitions are shown in Supplementary material online, Table S3. In the pooled analysis, the rate of treatment with LMWH was 87.4%; enoxaparin was used in 83.1% of patients treated with LMWH (see Supplementary material online, Table S6). The risk of bias for each study and estimate of the overall risk of bias are reported in Supplementary material online, Table S7 and publication bias by visual inspection of funnel plots in Supplementary material online, Table S8.
Table 1.
Study | Methodology | Treatment arms | Time period | Location | Clinical status | Confirmed by rt-PCR (%) | Sample size | Follow-up Duration (days) |
---|---|---|---|---|---|---|---|---|
INSPIRATION | Open label, RCT | Intermediate-dose (enoxaparin/UFH) vs. standard-dose prophylactic anticoagulationb | From July 29, to November 19, 2020 | Iran | Critically ill | 100 | 562 | 90 |
ACTION | Open label, RCT | Therapeutic-dose anticoagulation with rivaroxaban (stable patients) or enoxaparin (unstable patients) vs. standard-dose (enoxaparin/UFH) prophylactic anticoagulationb | From June 24, 2020, to February 26, 2021 | Brazil | Non-critically ill (94%) and critically ill (6%) | 100 | 614 | 30 |
REMAP-CAP, ACTIV-4a, and ATTACC | Open label, RCT | Therapeutic-dose (enoxaparin/UFH) vs. standard-dose prophylactic anticoagulationc | From April 21, to December 19 2020 | UK, USA, Canada, Brazil, and other | Critically ill | 91.6 | 1098 | 21 |
REMAP-CAP, ACTIV-4a, and ATTACC | Open label, RCT | Therapeutic-dose (enoxaparin/UFH) vs. standard-dose prophylactic anticoagulationc | From April 21, 2020 to January 22, 2021 | UK, USA, Canada, Brazil, and other | Non-critically ill | NA | 2219 | 21 |
HESACOVID | Open label, RCT | Therapeutic-dose (enoxaparin/UFH) vs. standard-dose (enoxaparin/UFH) prophylactic anticoagulationb | From April 2020 to July 2020 | Brazil | Critically ill | 100 | 20 | 14 |
RAPIDa | Open label, RCT | Intermediate-dose (enoxaparin/UFH) vs. standard-dose prophylactic anticoagulationb | From May 2020 to April 2021 | Canada, Saudi Arabia, Brazil, and other | Non-critically ill | 100 | 465 | 28 |
Perepu et al. | Open label, RCT | Therapeutic-dose (enoxaparin/UFH) vs. standard-dose prophylactic anticoagulation (enoxaparin/UFH)b | From April 2020 to January 2021 | USA | Critically ill | 100 | 176 | 30 |
RCT, randomized controlled trial; UFH, unfractionated heparin; UK, United Kingdom; USA, United States of America; rt-PCR, reverse transcription polymerase chain reaction; NA, not available.
Preprint manuscript.
Defined according to trial protocol.
Standard-dose prophylactic anticoagulation was defined according to local site protocols.
Primary outcomes
Seven studies were included for the primary outcomes of all-cause death and major bleeding. Overall, the incidence of all-cause death was 17.8% (466/2616) in the escalated-dose and 18.6% (473/2535) in the standard-dose prophylactic anticoagulation group. The incidence of major bleeding was 2.4% (63/2620) and 1.4% (36/2532) in the escalated-dose and standard-dose anticoagulation groups, respectively. Compared to standard-dose prophylactic anticoagulation, escalated-dose prophylactic anticoagulation was not associated with a reduction of all-cause death (RR 0.96, 95%CI 0.78–1.18, I2 = 56%) but was associated with an increase in major bleeding (RR 1.73, 95%CI 1.15–2.60, I2 = 0%) (Figure 1).
Secondary outcomes
Seven studies were included for the outcome of VTE. The incidence of VTE was 2.5% (66/2621) with the escalated-dose and 4.7% (119/2528) with the standard-dose prophylactic anticoagulation. An escalated-dose regimen was associated with lower rates of VTE events compared to the standard dose (RR 0.55, 95%CI 0.41–0.74, I2 = 0%) (Figure 2). Moreover, when compared to prophylactic anticoagulation at the standard dose, the reduction in VTE events in patients treated with the escalated dose was mainly driven by a significant lower rate of PE (RR 0.39, 95%CI 0.26–0.58, I2 = 0%), without any effect on the rate of deep venous thrombosis (RR 1.03, 95%CI 0.60–1.75, I2 = 0%) (see Supplementary material online, Table S9).
Six trials were included for the outcome of MI. The incidence of MI was 0.9% (23/2611) and 1.2% (29/2518) in the escalated-dose and standard-dose groups, respectively, resulting in no significant differences between regimens (RR 0.80, 95%CI 0.47–1.36, I2 = 0%) (Figure 2).
Four trials were included for the outcomes of stroke and systemic arterial embolism. The incidence of stroke was 0.5% (11/2296) and 0.5% (12/2195) in the escalated-dose and standard-dose groups, respectively, resulting in no significant differences between regimens (RR 0.94, 95%CI 0.43–2.09, I2 = 0%) (Figure 2). The incidence of systemic arterial embolism was 0.4% (9/2296) and 0.3% (6/2195) in the escalated-dose and standard-dose groups, respectively, resulting in no significant differences between regimens (RR 1.20, 95%CI 0.29–4.95, I2 = 27%) (Figure 2).
Four studies were included for the outcome of any bleeding and minor bleeding. The incidence of any bleeding was 9.5% (65/683) with prophylactic anticoagulation at the escalated dose and 4.2% (29/686) at the standard dose. The incidence of minor bleeding was 5.3% (36/683) and 2.2% (15/686) in the escalated-dose and standard-dose groups, respectively. Prophylactic anticoagulation at the escalated dose was associated with higher rates of any bleeding (RR 2.01, 95%CI 1.08–3.74, I2 = 47%) and minor bleeding (RR 2.23, 95%CI 1.04–4.81, I2 = 30%) compared to prophylactic anticoagulation at the standard dose (Figure 3).
Sensitivity and subgroup analyses
Sensitivity analysis by sequentially excluding one trial at a time showed that individual study data did not generally influence any of the included outcomes. Exceptions were major bleeding after exclusion of REMAP-CAP, ACTIV-4a, and ATTACC non-critically ill patients. While after exclusion of the HESACOVID, ACTION, or INSPIRATION trial, the results of the outcomes any and minor bleeding were affected (see Supplementary material online, Table S10).
Subgroup analysis showed consistent results for all included outcomes among both critically ill and non-critically ill patients, except for any bleeding, which was significantly higher in non-critically ill, but not in critically ill patients (Pinteraction = 0.03) (Figures 1–3).
The results were consistent in the sensitivity analyses performed with IRR (see Supplementary material online, Table S11).
At meta-regression analysis, we did not find any association between the concomitant use of antiplatelet therapy and the treatment effects of the anticoagulation regimens in both fixed-effects and random-effects models (see Supplementary material online, Table S12).
Numbers needed to treat
The NNT and NNH calculations reflecting the number of treated patients needed to prevent or cause each outcome are shown in Figure 4. The NNT for all-cause death was 119, whereas the NNT for VTE was 46. The NNH for major, minor, and any bleeding events were 102, 32, and 16, respectively.
Discussion
In this comprehensive meta-analysis including all available RCTs comparing prophylactic anticoagulation at an escalated vs. standard dose in hospitalized COVID-19 patients without a formal indication to be on therapeutic anticoagulation, we found that the escalated dosing regimen: (1) was not associated with lower risk of all-cause death; (2) was associated with significantly higher rates of any, major, and minor bleeding; (3) was associated with significantly lower rates of VTE, driven by a reduction in PE, but not of arterial thrombotic events; (4) showed consistent safety and efficacy profiles among both critically and non-critically ill patients.
Thromboprophylaxis is a guideline-recommended therapy for preventing VTE in selected hospitalized medical and surgical patients without an indication for anticoagulation therapy.20 Current evidence suggests that thromboprophylaxis reduces the risk of VTE in hospitalized patients with risk factors.21 However, thromboprophylaxis does not eliminate the risk of VTE or VTE-related death. Of note, in acutely ill medical patients, thromboprophylaxis does not reduce mortality.21 The potential clinical benefits of thromboprophylaxis on VTE prevention are associated with an increase in major and non-major bleeding, especially in the presence of risk factors such as frailty, active gastroduodenal ulcer, previous bleeding, and low platelet count.22 Nevertheless, a risk-balance assessment suggests that the benefits outweigh the risks, and therefore, it is considered a standard of care in selected patients.20
Because of the pro-coagulant state and increased risk of thromboembolic events in COVID-19 patients, the use of anticoagulation for prophylactic purposes is recommended.23 Anticoagulation can indeed be of potential benefit for the treatment and prevention of venous and arterial micro- and macrothrombosis.1 An early finding suggested that using an escalated rather than a standard prophylactic anticoagulant dosing regimen decreased mortality in patients with severe COVID-19 and coagulopathy.5 However, this study was not randomized, it excluded patients with bleeding diathesis, and included only patients from China. A meta-analysis of observational studies including 25 719 hospitalized COVID-19 patients found that an escalated prophylactic anticoagulant dosing regimen was associated with a 50% reduction in in-hospital mortality.6 Nevertheless, the absence of data from RCTs is a major limitation of these findings. Several RCTs were designed worldwide to address the critical clinical question on the safety and efficacy of anticoagulation in COVID-19 patients.7–9 However, none of these trials were powered for testing the superiority of treatment regimens with respect to all-cause death or major bleeding.
In this meta-analysis including 5154 hospitalized COVID-19 patients from 7 RCTs, we found that the use of prophylactic anticoagulation at an escalated dose was not associated with a significant reduction in all-cause death compared to the standard dose regardless of the clinical status (i.e. critically and non-critically ill patients). Heterogeneity among trials for all-cause death was high (56%) possibly reflecting multiple causes of death including both bleeding and thromboembolic events. Compared to the standard prophylactic anticoagulant dose, the escalated-dosing regimen significantly reduced VTE, with an NNT of 46, but significantly increased the risk of bleeding, with an NNH of 102. There were no significant differences on arterial thrombotic events. Overall, the cumulative evidence to support the routine use of escalated-dose prophylactic anticoagulation in clinical practice is limited. An escalated dose might be considered only on an individual basis, when thrombotic risk is deemed too heavily outweigh the bleeding risk. It should be highlighted that of the included trial, the REMAP-CAP, ACTIV-4a, and ATTACC non-critically ill patients met its primary composite endpoint of organ support—free days.13 However, this endpoint was not included in this analysis because it was only measured in the REMAP-CAP, ACTIV-4a, and ATTACC trials.
COVID-19 patients may exhibit complex coagulopathy states. While at the beginning of the disease patients may exhibit a pro-thrombotic state, those who advance to severe forms of the disease may exhibit a disseminated intravascular coagulopathy (DIC)-like state.24 Therefore, as the severity of the COVID-19 increase, the bleeding risk may also increase. In the present analysis, major bleeding and VTE were both more frequent among critically vs. non-critically ill patients (2.6% vs. 1.5% for bleeding and 5.7% vs. 2.4% for VTE), but critical-illness status did not significantly interact with treatment effects on these outcomes. Timing of administration of anticoagulation may be crucial. On average, the median time from symptom onset to randomization in the included trials was close to 10 days (7–8 days from symptom onset to hospitalization and 2–4 days from hospitalization to randomization). At this time point, some patients can be ending the pro-coagulant state and transitioning into a DIC-like state, albeit bleeding rates are not as high. Thus, in patients in whom the predominant state is pro-coagulant, prophylactic anticoagulation at an escalated dose may be potentially beneficial. However, in patients who advance to a DIC-like state, such escalated regimen may be harmful. Of note, observational studies have reported that major bleeding events may be related to higher risk of death in COVID-19 patients and should be considered as important as thrombotic events.25 Optimizing patient selection and the window for treatment with anticoagulant therapy can be challenging and warrants further research. The application of prediction models to identify patients at higher risk of thrombotic events may improve patient selection who can benefit from escalated doses of prophylactic anticoagulation.26 Moreover, special populations such as the elderly may benefit from tailored dose anticoagulation to minimize the risk of adverse events.27 However, this approach should be tested in dedicated RCTs.
In clinical practice, knowing the inflammatory status of an individual patient may be of potential utility given its association with the risk of bleeding. Observational studies have suggested that high C-reactive protein levels may be associated with an increased bleeding risk.28 Therefore, it may be hypothesized that anti-inflammatory therapies may modulate the risk of bleeding. Given the pro-inflammatory status which characterizes patients with COVID-19, the link between anti-inflammatory therapies and bleeding outcomes is important and warrants further investigation.
Prophylactic anticoagulation at an escalated dosing regimen was associated with a lower rate of VTE, driven by a reduction in PE, without any effect on arterial thrombotic events. Although arterial thrombotic events occur less frequently than VTE, they have greater impact on adverse prognosis.29,30 This may be attributed to the fact that VTE is commonly represented by in situ PE in COVID-19 patients.1,31 Taking into consideration the differential prognostic impact of venous vs. arterial thrombotic events, including all-cause death, it appears that VTE prevention does not compensate for the increase in bleeding risk associated with the use of prophylactic anticoagulation at an escalated-dosing regimen.
Ultimately, we performed a subgroup analysis according to clinical status (i.e. critically and non-critically ill). The rationale for this analysis is that several factors differ from critically and non-critically ill patients, including indication for VTE prophylaxis, vital prognosis, coagulopathy state, bleeding risk profile, drug selection, via of drug administration, and drug interactions. We only found a significant interaction between subgroups in the endpoint of any bleeding. However, we believe this interaction should be considered cautiously because the endpoint of any bleeding was reported by only three trials including critically ill patients and one trial including non-critically ill patients. Moreover, this significant interaction was not found for the outcomes of major or minor bleeding endpoints. Overall, all results were consistent independently of the clinical status of the patient. This finding has important clinical implications because previous observational data suggested that critically ill patients may benefit more from an escalated anticoagulant dosing regimen than non-critically ill patients.6 The results of this meta-analysis of RCTs do not support these findings.
Limitations
This meta-analysis has limitations. First, trial-level rather than individual-level data were used to assess outcomes. Second, due to the limited size of specific groups, we did not perform detailed subgroup analyses according to baseline characteristics. However, all analyses were performed according to a pre-specified subgroup analysis by clinical status (critically vs. non-critically ill), which is the main prognostic determinant. Third, ACTION was the only trial using a DOAC (i.e. rivaroxaban) as anticoagulation strategy compared to the other trials (using UFH/LMWH). Nevertheless, we ran sensitivity analyses for all the endpoints removing the ACTION trial, and the overall study findings remained consistent. Fourth, the INSPIRATION and Perepu et al. trials used an intermediate dose of anticoagulation (enoxaparin/UFH) rather than a full anticoagulant dose. However, the study findings were consistent even after the exclusion of the INSPIRATION or Perepu et al. trials results. Fifth, one preprint trial was included in this meta-analysis. Nonetheless, there is a low probability of a change in the raw data during the peer-review process and, in light of the urgent status due to the pandemic, it is essential to provide in a timely fashion updated evidence to guide clinical practice. Sixth, since we performed a meta-analysis of aggregated data, we cannot rule out the risk of ecological bias. Moreover, since less than 10 studies were included in this analysis, the accuracy of funnel plots as well of meta-regression analyses may be debated. Therefore, we cannot rule out the presence of publication and reporting bias of the studied outcomes. Ultimately, over 30 RCTs are still enrolling patients with the aim of comprehensively evaluating the role of anticoagulation in COVID-19 patients using a wide variety of drugs at different regimens in diverse clinical settings.32 Therefore, once these further studies will be available, future updated meta-analyses, possibly performed at a patient-level, including a network design, and multivariate rather than univariate analyses, will play an essential role in defining the safety and efficacy of therapeutic vs. standard prophylactic anticoagulation in critically and non-critically ill patients with COVID-19.
Conclusions
In hospitalized patients with COVID-19 without an indication for therapeutic anticoagulation, although the use of prophylactic anticoagulation at an escalated dose was associated with a reduction in the risk of VTE compared to a standard dose, it was not associated with a reduction in all-cause death or other adverse ischemic events but with a significant increase in major bleeding. These findings were consistent among critically and non-critically ill patients. The overall findings of this meta-analysis of RCTs do not support the routine use of prophylactic anticoagulation at an escalated over a standard dose in hospitalized patients with COVID-19.
Supplementary Material
Acknowledgement
This work was not funded. M.G. is supported by a grant from the Fondazione Enrico ed Enrica Sovena (Rome, Italy).
Contributor Information
Luis Ortega-Paz, Cardiovascular Institute, Hospital Clinic, IDIBAPS, Barcelona 08036, Spain.
Mattia Galli, Cardiovascular Medicine, Fondazione Policlinico Universitario A Gemelli IRCCS, Rome 00168, Italy; Division of Cardiology, University of Florida College of Medicine, ACC Building 5th floor, 655 West 8th Street, Jacksonville, FL 32209, USA.
Davide Capodanno, Division of Cardiology, A.O.U. “Policlinico-Vittorio Emanuele,” University of Catania, Catania 95124, Italy.
Francesco Franchi, Division of Cardiology, University of Florida College of Medicine, ACC Building 5th floor, 655 West 8th Street, Jacksonville, FL 32209, USA.
Fabiana Rollini, Division of Cardiology, University of Florida College of Medicine, ACC Building 5th floor, 655 West 8th Street, Jacksonville, FL 32209, USA.
Behnood Bikdeli, Cardiovascular Medicine Division, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA; Center for Outcomes Research and Evaluation (CORE), Yale School of Medicine, New Haven, CT 06510, USA; Cardiovascular Research Foundation (CRF), New York, NY 10019, USA.
Roxana Mehran, Zena and Michael A. Wiener Cardiovascular Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA.
Gilles Montalescot, Sorbonne Université, ACTION Study Group, Institut de Cardiologie (Assistance Publique - Hôpitaux de Paris) Hôpital Pitié-Salpêtrière, INSERM UMRS 1166, Paris 75013, France.
C Michael Gibson, Cardiovascular Division, Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA 02215, USA.
Renato D Lopes, Duke Clinical Research Institute, Duke University Medical Center, Durham, NC 27710, USA; Cardiovascular Division, Brazilian Clinical Research Institute, São Paulo, 01404-000, Brazil.
Felicita Andreotti, Cardiovascular Medicine, Fondazione Policlinico Universitario A Gemelli IRCCS, Rome 00168, Italy.
Dominick J Angiolillo, Division of Cardiology, University of Florida College of Medicine, ACC Building 5th floor, 655 West 8th Street, Jacksonville, FL 32209, USA.
Conflict of interest: D.C. declares that he has received consulting and speaker's fee from Amgen, Boehringer Ingelheim, Daiichi Sankyo, Sanofi, outside the present work. F.F. declares that he has received consulting fees or honoraria from AstraZeneca, Bayer, and Sanofi, outside the present work. F.R. declares that he has received honoraria from Chiesi, outside the present work. B.B. reports that he is a consulting expert, on behalf of the plaintiff, for litigation related to two specific brand models of IVC filters. R.M. reports institutional research grants from Abbott, Abiomed, Applied Therapeutics, Arena, AstraZeneca, Bayer, Biosensors, Boston Scientific, Bristol-Myers Squibb, CardiaWave, CellAegis, CERC, Chiesi, Concept Medical, CSL Behring, DSI, Insel Gruppe AG, Medtronic, Novartis Pharmaceuticals, OrbusNeich, Philips, Transverse Medical, Zoll; personal fees from ACC, Boston Scientific, California Institute for Regenerative Medicine (CIRM), Cine-Med Research, Janssen, WebMD, SCAI; consulting fees paid to the institution from Abbott, Abiomed, AM-Pharma, Alleviant Medical, Bayer, Beth Israel Deaconess, CardiaWave, CeloNova, Chiesi, CSL Behring, Concept Medical, DSI, Duke University, Idorsia Pharmaceuticals, Medtronic, Novartis, Philips; Equity < 1% in Applied Therapeutics, Elixir Medical, STEL, CONTROLRAD (spouse); Scientific Advisory Board for AMA, Biosensors (spouse); Faculty CRF (no fee). G.M. reports consulting or speaker's fees from Abbott, Amgen, AstraZeneca, Bayer, Boehringer Ingelheim, Boston Scientific, Bristol-Myers Squibb, Cell Prothera, CSL Behring, Europa, Idorsia, IRIS-Servier, Medtronic, MSD, Novartis, Pfizer, Quantum Genomics, Sanofi-Aventis. C.M.G. receives research support from Johnson & Johnson. He receives consulting support from Astra-Zenca, Johnson & Johnson, Janssen & Bayer, outside the present work. R.D.L. reports significant grants and personal fees from Bristol-Myers Squibb and Pfizer, modest personal fees from Boehringer Ingelheim and Bayer AG and modest grants from Amgen Inc, GlaxoSmithKline, Medtronic PLC, and Sanofi Aventis. F.A. reports receiving speaker or consultancy fees from Amgen, Bayer, BMS/Pfizer, Daiichi Sankyo and Radcliffe Cardiology. D.J.A. declares that he has received consulting fees or honoraria from Abbott, Amgen, Aralez, AstraZeneca, Bayer, Biosensors, Boehringer Ingelheim, Bristol-Myers Squibb, Chiesi, Daiichi-Sankyo, Eli Lilly, Haemonetics, Janssen, Merck, PhaseBio, PLx Pharma, Pfizer, Sanofi, and The Medicines Company and has received payments for participation in review activities from CeloNova and St Jude Medical. D.J.A. also declares that his institution has received research grants from Amgen, AstraZeneca, Bayer, Biosensors, CeloNova, CSL Behring, Daiichi-Sankyo, Eisai, Eli Lilly, Gilead, Idorsia, Janssen, Matsutani Chemical Industry Co., Merck, Novartis, Osprey Medical, Renal Guard Solutions, and the Scott R. MacKenzie Foundation. Other authors have nothing to declare.
Data availability statement: Extracted data are available on request to the corresponding author.
References
- 1. Ortega-Paz L, Capodanno D, Montalescot G, Angiolillo DJ. Coronavirus disease 2019-associated thrombosis and coagulopathy: review of the pathophysiological characteristics and implications for antithrombotic management. J Am Heart Assoc 2021;10:e019650. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis 2020;20:533–534. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Bikdeli B, Madhavan MV, Jimenez D, Chuich T, Dreyfus I, Driggin E, Nigoghossian C, Ageno W, Madjid M, Guo Y, Tang LV, Hu Y, Giri J, Cushman M, Quere I, Dimakakos EP, Gibson CM, Lippi G, Favaloro EJ, Fareed J, Caprini JA, Tafur AJ, Burton JR, Francese DP, Wang EY, Falanga A, McLintock C, Hunt BJ, Spyropoulos AC, Barnes GD, Eikelboom JW, Weinberg I, Schulman S, Carrier M, Piazza G, Beckman JA, Steg PG, Stone GW, Rosenkranz S, Goldhaber SZ, Parikh SA, Monreal M, Krumholz HM, Konstantinides SV, Weitz JI, Lip GYH, Global Covid-19 Thrombosis Collaborative Group EbtINE, the Iua SbtESCWGoPC, Right Ventricular F . COVID-19 and thrombotic or thromboembolic disease: implications for prevention, antithrombotic therapy, and follow-up: JACC state-of-the-art review. J Am Coll Cardiol 2020;75:2950–2973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. White D, MacDonald S, Bull T, Hayman M, de Monteverde-Robb R, Sapsford D, Lavinio A, Varley J, Johnston A, Besser M, Thomas W. Heparin resistance in COVID-19 patients in the intensive care unit. J Thromb Thrombolysis 2020;50:287–291. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Tang N, Bai H, Chen X, Gong J, Li D, Sun Z. Anticoagulant treatment is associated with decreased mortality in severe coronavirus disease 2019 patients with coagulopathy. J Thromb Haemost 2020;18:1094–1099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Parisi R, Costanzo S, Di Castelnuovo A, de Gaetano G, Donati MB, Iacoviello L. Different anticoagulant regimens, mortality, and bleeding in hospitalized patients with COVID-19: a systematic review and an updated meta-analysis. Semin Thromb Hemost 2021;47:372–391. [DOI] [PubMed] [Google Scholar]
- 7. Bikdeli B, Talasaz AH, Rashidi F, Bakhshandeh H, Rafiee F, Rezaeifar P, Baghizadeh E, Matin S, Jamalkhani S, Tahamtan O, Sharif-Kashani B, Beigmohammadi MT, Farrokhpour M, Sezavar SH, Payandemehr P, Dabbagh A, Moghadam KG, Khalili H, Yadollahzadeh M, Riahi T, Abedini A, Lookzadeh S, Rahmani H, Zoghi E, Mohammadi K, Sadeghipour P, Abri H, Tabrizi S, Mousavian SM, Shahmirzaei S, Amin A, Mohebbi B, Parhizgar SE, Aliannejad R, Eslami V, Kashefizadeh A, Dobesh PP, Kakavand H, Hosseini SH, Shafaghi S, Ghazi SF, Najafi A, Jimenez D, Gupta A, Madhavan MV, Sethi SS, Parikh SA, Monreal M, Hadavand N, Hajighasemi A, Maleki M, Sadeghian S, Piazza G, Kirtane AJ, Van Tassell BW, Stone GW, Lip GYH, Krumholz HM, Goldhaber SZ, Sadeghipour P. Intermediate-dose versus standard-dose prophylactic anticoagulation in patients with COVID-19 admitted to the intensive care unit: 90-day results from the INSPIRATION randomized trial. Thromb Haemost 2021. [DOI] [PubMed] [Google Scholar]
- 8. Lopes RD, de Barros ESPGM, Furtado RHM, Macedo AVS, Bronhara B, Damiani LP, Barbosa LM, de Aveiro Morata J, Ramacciotti E, de Aquino Martins P, de Oliveira AL, Nunes VS, Ritt LEF, Rocha AT, Tramujas L, Santos SV, Diaz DRA, Viana LS, Melro LMG, de Alcantara Chaud MS, Figueiredo EL, Neuenschwander FC, Dracoulakis MDA, Lima R, de Souza Dantas VC, Fernandes ACS, Gebara OCE, Hernandes ME, Queiroz DAR, Veiga VC, Canesin MF, de Faria LM, Feitosa-Filho GS, Gazzana MB, Liporace IL, de Oliveira Twardowsky A, Maia LN, Machado FR, de Matos Soeiro A, Conceicao-Souza GE, Armaganijan L, Guimaraes PO, Rosa RG, Azevedo LCP, Alexander JH, Avezum A, Cavalcanti AB, Berwanger O, Investigators ACC-BI . Therapeutic versus prophylactic anticoagulation for patients admitted to hospital with COVID-19 and elevated D-dimer concentration (ACTION): an open-label, multicentre, randomised, controlled trial. Lancet 2021;397:2253–2263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Lemos ACB, do Espirito Santo DA, Salvetti MC, Gilio RN, Agra LB, Pazin-Filho A, Miranda CH. Therapeutic versus prophylactic anticoagulation for severe COVID-19: a randomized phase II clinical trial (HESACOVID). Thromb Res 2020;196:359–366. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Sholzberg M, Tang GH, Rahhal H, AlHamzah M, Kreuziger LB, Ni Ainle F, Alomran F, Alayed K, Alsheef M, AlSumait F, Pompilio CE, Sperlich C, Tangri S, Tang T, Jaksa P, Suryanarayan D, Almarshoodi M, Castellucci L, James PD, Lillicrap D, Carrier M, Beckett A, Colovos C, Jayakar J, Arsenault MP, Wu C, Doyon K, Andreou ER, Dounaevskaia V, Tseng EK, Lim G, Fralick M, Middeldorp S, Lee AYY, Zuo F, da Costa BR, Thorpe KE, Negri EM, Cushman M, Juni P, investigators RT. Heparin for moderately ill patients with Covid-19. medRxiv 2021:2021.07.08.21259351. [Google Scholar]
- 11. Perepu US, Chambers I, Wahab A, Ten Eyck P, Wu C, Dayal S, Sutamtewagul G, Bailey SR, Rosenstein LJ, Lentz SR. Standard prophylactic versus intermediate dose enoxaparin in adults with severe COVID-19: a multi-center, open-label, randomized controlled trial. J Thromb Haemost 2021;19:2225–2234 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. The REMAP-CAP, ACTIV-4a, and ATTACC Investigators . Therapeutic anticoagulation with heparin in critically ill patients with Covid-19. N Engl J Med 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. The REMAP-CAP, ACTIV-4a, and ATTACC Investigators . Therapeutic anticoagulation with heparin in noncritically ill patients with Covid-19. N Engl J Med 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, Group P-P . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Rappaport SH, Clark JM, Delibert S, Maynard KM, Prasad P, Kaufman DC, Pietropaoli AP, Quill CM, Groth CM. Anti-FXa activity with intermediate-dose thromboprophylaxis in COVID-19. Am J Respir Crit Care Med 2020;202:1731–1733. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Flumignan RL, Tinoco JDS, Pascoal PI, Areias LL, Cossi MS, Fernandes MI, Costa IK, Souza L, Matar CF, Tendal B, Trevisani VF, Atallah AN, Nakano LC. Prophylactic anticoagulants for people hospitalised with COVID-19. Cochrane Database Syst Rev 2020;10:CD013739. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA, Cochrane Bias Methods G, Cochrane Statistical Methods G . The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Mehran R, Rao SV, Bhatt DL, Gibson CM, Caixeta A, Eikelboom J, Kaul S, Wiviott SD, Menon V, Nikolsky E, Serebruany V, Valgimigli M, Vranckx P, Taggart D, Sabik JF, Cutlip DE, Krucoff MW, Ohman EM, Steg PG, White H. Standardized bleeding definitions for cardiovascular clinical trials: a consensus report from the Bleeding Academic Research Consortium. Circulation 2011;123:2736–2747. [DOI] [PubMed] [Google Scholar]
- 19. Richardson M, Garner P, Donegan S. Interpretation of subgroup analyses in systematic reviews: a tutorial. Clinical Epidemiology and Global Health 2019;7:192–198. [Google Scholar]
- 20. Kearon C, Akl EA, Ornelas J, Blaivas A, Jimenez D, Bounameaux H, Huisman M, King CS, Morris TA, Sood N, Stevens SM, Vintch JRE, Wells P, Woller SC, Moores L. Antithrombotic therapy for VTE disease: CHEST Guideline and Expert Panel Report. Chest 2016;149:315–352. [DOI] [PubMed] [Google Scholar]
- 21. Samama MM, Cohen AT, Darmon JY, Desjardins L, Eldor A, Janbon C, Leizorovicz A, Nguyen H, Olsson CG, Turpie AG, Weisslinger N. A comparison of enoxaparin with placebo for the prevention of venous thromboembolism in acutely ill medical patients. Prophylaxis in Medical Patients with Enoxaparin Study Group. N Engl J Med 1999;341:793–800. [DOI] [PubMed] [Google Scholar]
- 22. Decousus H, Tapson VF, Bergmann JF, Chong BH, Froehlich JB, Kakkar AK, Merli GJ, Monreal M, Nakamura M, Pavanello R, Pini M, Piovella F, Spencer FA, Spyropoulos AC, Turpie AG, Zotz RB, Fitzgerald G, Anderson FA, Investigators I . Factors at admission associated with bleeding risk in medical patients: findings from the IMPROVE investigators. Chest 2011;139:69–79. [DOI] [PubMed] [Google Scholar]
- 23. Cuker A, Tseng EK, Nieuwlaat R, Angchaisuksiri P, Blair C, Dane K, Davila J, DeSancho MT, Diuguid D, Griffin DO, Kahn SR, Klok FA, Lee AI, Neumann I, Pai A, Pai M, Righini M, Sanfilippo KM, Siegal D, Skara M, Touri K, Akl EA, Bou Akl I, Boulos M, Brignardello-Petersen R, Charide R, Chan M, Dearness K, Darzi AJ, Kolb P, Colunga-Lozano LE, Mansour R, Morgano GP, Morsi RZ, Noori A, Piggott T, Qiu Y, Roldan Y, Schunemann F, Stevens A, Solo K, Ventresca M, Wiercioch W, Mustafa RA, Schunemann HJ. American Society of Hematology 2021 guidelines on the use of anticoagulation for thromboprophylaxis in patients with COVID-19. Blood Adv 2021;5:872–888. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Thachil J, Cushman M, Srivastava A. A proposal for staging COVID-19 coagulopathy. Res Pract Thromb Haemost 2020;4:731–736. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Demelo-Rodriguez P, Farfan-Sedano AI, Pedrajas JM, Llamas P, Siguenza P, Jaras MJ, Quintana-Diaz M, Fernandez-Capitan C, Bikdeli B, Jimenez D, Monreal M, Investigators R-B . Bleeding risk in hospitalized patients with COVID-19 receiving intermediate- or therapeutic doses of thromboprophylaxis. J Thromb Haemost 2021;19:1981–1989. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Ramirez GA, Calvisi SL, Rebecca DEL, Valentina DAP, Borio G, Gallina G, Farolfi F, Cavallo L, Pascali M, Castellani J, Baccellieri D, Guzzo F, Baiardo Redaelli M, Azzolini ML, Alba AC, Zangrillo A, Bozzolo EP, Scotti R, Giuseppe DIL, Piemonti L, Rovere Querini P, D'Angelo A, Tresoldi M, group CO-B . A novel evidence-based algorithm to predict thromboembolism in patients with COVID-19: preliminary data from a single-centre cohort. Minerva Med 2021. [DOI] [PubMed] [Google Scholar]
- 27. Marini M, Zilio F, Martin M, Strazzanti M, Quintarelli S, Guarracini F, Coser A, Giacopelli D, Bonmassari R. COVID-19 pandemic and elderly: is the curtain dropped for urgent pacemaker implantations? Minerva Cardioangiol 2020. [DOI] [PubMed] [Google Scholar]
- 28. Arevalos V, Ortega-Paz L, Rodriguez-Arias JJ, Calvo M, Castrillo L, Salazar A, Roque M, Dantas AP, Sabate M, Brugaletta S. Myocardial injury in COVID-19 patients: association with inflammation, coagulopathy and in-hospital prognosis. J Clin Med 2021;10:2096. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Malas MB, Naazie IN, Elsayed N, Mathlouthi A, Marmor R, Clary B. Thromboembolism risk of COVID-19 is high and associated with a higher risk of mortality: a systematic review and meta-analysis. EClinicalMedicine 2020;29:100639. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Gonzalez-Fajardo JA, Ansuategui M, Romero C, Comanges A, Gomez-Arbelaez D, Ibarra G, Garcia-Gutierrez A. Mortality of COVID-19 patients with vascular thrombotic complications. Med Clin (Engl Ed) 2021;156:112–117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Wang T, Chen R, Liu C, Liang W, Guan W, Tang R, Tang C, Zhang N, Zhong N, Li S. Attention should be paid to venous thromboembolism prophylaxis in the management of COVID-19. Lancet Haematol 2020;7:e362–e363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Talasaz AH, Sadeghipour P, Kakavand H, Aghakouchakzadeh M, Kordzadeh-Kermani E, Van Tassell BW, Gheymati A, Ariannejad H, Hosseini SH, Jamalkhani S, Sholzberg M, Monreal M, Jimenez D, Piazza G, Parikh SA, Kirtane AJ, Eikelboom JW, Connors JM, Hunt BJ, Konstantinides SV, Cushman M, Weitz JI, Stone GW, Krumholz HM, Lip GYH, Goldhaber SZ, Bikdeli B. Recent randomized trials of antithrombotic therapy for patients with COVID-19: JACC state-of-the-art review. J Am Coll Cardiol 2021;77:1903–1921. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.