Abstract
Background
We and others have previously demonstrated that the endothelium is a primary target of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and L-arginine has been shown to improve endothelial dysfunction. However, the effects of L-arginine have never been evaluated in coronavirus disease 2019 (COVID-19).
Methods
This is a parallel-group, double-blind, randomized, placebo-controlled trial conducted on patients hospitalized for severe COVID-19. Patients received 1.66 g L-arginine twice a day or placebo, administered orally. The primary efficacy endpoint was a reduction in respiratory support assessed 10 and 20 days after randomization. Secondary outcomes were the length of in-hospital stay, the time to normalization of lymphocyte number, and the time to obtain a negative real-time reverse transcription polymerase chain reaction (RT-PCR) for SARS-CoV-2 on nasopharyngeal swab. This clinical trial had been registered at ClinicalTrials.gov, identifier: NCT04637906.
Findings
We present here the results of the initial interim analysis on the first 101 patients. No treatment-emergent serious adverse events were attributable to L-arginine. At 10-day evaluation, 71.1% of patients in the L-arginine arm and 44.4% in the placebo arm (p < 0.01) had the respiratory support reduced; however, a significant difference was not detected 20 days after randomization. Strikingly, patients treated with L-arginine exhibited a significantly reduced in-hospital stay vs placebo, with a median (interquartile range 25th,75th percentile) of 46 days (45,46) in the placebo group vs 25 days (21,26) in the L-arginine group (p < 0.0001); these findings were also confirmed after adjusting for potential confounders including age, duration of symptoms, comorbidities, D-dimer, as well as antiviral and anticoagulant treatments. The other secondary outcomes were not significantly different between groups.
Interpretation
In this interim analysis, adding oral L-arginine to standard therapy in patients with severe COVID-19 significantly decreases the length of hospitalization and reduces the respiratory support at 10 but not at 20 days after starting the treatment.
Funding
Both placebo and L-arginine were kindly provided by Farmaceutici Damor S.p.A., Naples
Keywords: COVID-19, Endothelial dysfunction, SARS-CoV-2, Arginine, Clinical trial, Coronavirus, Immune response
Research in context.
Evidence before this study
The severe acute respiratory syndrome coronavirus 2 (SARS-CoV- 2) infection has spread worldwide, causing disease and mortality, as well as social disruption and economic loss. Endothelial cells have been shown to be a cardinal target in COVID-19, and L-arginine is known to improve endothelial dysfunction. PubMed and medRxiv preprint searches were updated on June 24, 2021 using the search criteria "SARS-CoV-2" "COVID-19" “L-arginine”. These searches retrieved no items.
Added value of this study
The first interim analysis of this randomized, double-blind, placebo-controlled, parallel-group trial shows that patients hospitalized for severe COVID-19 treated with L-arginine displayed a significantly reduced in-hospital stay vs placebo (log-rank p < 0.0001). Furthermore, at the 10-day evaluation, 71.1% of patients in the L-arginine group and 44.4% in the placebo group (p < 0.01) had reduced the respiratory support; however, a significant difference was not detected 20 days after randomization, most likely because most of the participants in the L-arginine arm had been already discharged from the hospital by this time.
Implications of all the available evidence
There is concrete evidence that endothelial dysfunction is a fundamental feature of COVID-19. However, no clinical trial has actually shown that intervention improving endothelial function could be beneficial in COVID-19. Our interim findings indicate for the first time that adding L-arginine orally to standard therapy in patients with severe COVID-19 significantly reduces the length of hospital stay and respiratory support.
Alt-text: Unlabelled box
1. Introduction
The clinical course of coronavirus disease 2019 (COVID-19) consists of two main phases: viral infection and immune/inflammatory response, requiring distinct therapeutic approaches to stop virus replication and to attenuate the inflammatory state that is commonly observed in COVID-19 patients and may contribute to multiorgan failure [1], [2], [3], [4]. Although several approved drugs and investigational agents have shown some activity against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the treatment of COVID-19 remains a critical challenge [5], [6], [7], [8], [9], [10], [11], [12], [13].
Clinical and preclinical evidence supports the view that the endothelium is a key target organ in COVID-19, providing a mechanistic rationale behind its systemic manifestations [14]. Hence, COVID-19 can be considered a systemic vascular disease affecting multiple organs due to endothelial damage [14,15]. More recently, amino acid metabolism has been shown to be a crucial factor in the pathophysiology of COVID-19 [16]; specifically, decreased plasma L-arginine levels along with enhanced arginase activity have been reported in COVID-19 patients, especially in the most severe forms [17,18].
On these grounds, given the well-established beneficial effects of L-arginine on endothelial function [19], we hypothesized that oral L-arginine supplementation could be helpful for contrasting the inflammatory state in COVID-19. Therefore, we designed the present single-center double-blind randomized, placebo-controlled trial with a parallel group scheme, to test the hypothesis that, compared to placebo, the addition of oral L-arginine to the hospital standard therapy is an efficacious treatment for patients hospitalized for COVID-19.
2. Methods
2.1. Study design and participants
We designed an investigator-initiated, individually randomized, placebo-controlled, double-blind trial aiming to assess the effectiveness and safety of oral L-arginine in adults (aged ≥ 18 years) admitted to hospital with severe COVID-19, defined as previously reported by other investigators [15,20]. The trial (registered at ClinicalTrials.gov, Identifier: NCT04637906) was conducted at the Domenico Cotugno Hospital (Naples, Italy). The study protocol is reported in Appendix 1.
Ethical approval was obtained from the institutional Ethical Committee of the Hospital. Written informed consent was collected from all patients, or their legal representative if they were unable to provide consent. The trial was performed in accordance with the principles of the Declaration of Helsinki and the International Conference on Harmonization-Good Clinical Practice guidelines.
Eligible hospitalized patients were men and non-pregnant women with COVID-19 who were at least 18-year-old and were RT-PCR positive for SARS-CoV-2; patients were screened by applying the following eligibility criteria.
2.2. Inclusion criteria
Presence of all the following conditions:
-
•
Pneumonia confirmed by chest imaging;
-
•
Oxygen saturation of 93% or lower on room air;
-
•
Ratio of alveolar oxygen partial pressure to fractional inspired oxygen (PaO2/FiO2, or P/F) of 300 or less;
-
•
Lymphocytopenia, defined as lymphocytes < 1500/μL or < 20% of white blood cells.
Exclusion criteria:
-
•
History of L-arginine intolerance;
-
•
Diagnosis of chronic pulmonary disease - currently under treatment;
-
•
Pregnancy or breastfeeding;
-
•
Neutropenia due to neoplasms of the hematopoietic system or other organs with invasion of the bone marrow;
-
•
Use of immunosuppressive drugs, other than corticosteroids, or cytotoxic chemotherapy within the previous three weeks;
-
•
Symptoms onset > 15 days before enrollment;
-
•
Enrollment into an investigational treatment study for COVID-19 in the 30 days before screening;
-
•
Refusal to provide written informed consent.
2.3. Procedures
Eligible patients were assigned, with a 1:1 ratio, based on a computer-generated randomization table, to add to the hospital standard therapy 1 bottle containing 1.66 g of L-arginine (Bioarginina®, Farmaceutici Damor S.p.A.) or 1 bottle of identical aspect not containing L-arginine, twice a day orally for the whole hospitalization period. The formulation of Bioarginina® used in this study consists of oral vials containing 1.66 g of L-arginine, whose taste is fully buffered by the presence of sucrose and by an acidity regulator (anhydrous citric acid); therefore, there were no differences in appearance, smell, or taste between treatment and placebo. The dose was chosen based on a previous study proving that such a regimen was safe and significantly effective in ameliorating the oxidative metabolism of professional water polo players [21]. Respiratory support was defined as follows (from the more to the less intense one): NIV: non-invasive ventilation; CPAP: continuous positive airway pressure; HFNC: high-flow nasal cannula; LTOT: Long-term oxygen therapy [22]. Since symptom duration has been shown to have an effect on prognosis [13], the actual duration of symptoms (≤ 8-days vs > 8-days) was taken into account during randomization. Protocol adherence was assessed daily during the hospitalization period. Patients, ward doctors, trial personnel and outcome assessors were unaware of the type of supplementation provided. All clinical interventions, including the use of antibiotics, corticosteroids, anticoagulants, ventilation strategy, and laboratory investigations were at the discretion of the treating physicians according to clinical needs for both treatment groups.
2.4. Clinical and laboratory parameters
A single follow-up form was completed by the trial staff when each patient was discharged or had died, whichever occurred first. Information was recorded regarding the patients’ adherence to the assigned treatment, administration of other treatments, duration of symptoms on admission, duration and type of respiratory support, renal dialysis or hemofiltration. In addition, we obtained routine health care and registry data, including information on vital status (or date and cause of death), discharge from the hospital, as well as respiratory and renal support therapy. The P/F ratio was calculated as previously described [23,24], obtaining PaO2 from arterial blood gas test; FiO2 was considered 0.21 when the patient was breathing room air [25].
Before randomization, the demographic characteristics and other clinical and laboratory data required by the routine diagnostic activity were collected. These parameters were evaluated daily during the hospitalization period according to routine hospital practice. The safety assessment included daily monitoring for adverse events, clinical laboratory testing, 12-lead electrocardiogram, and daily measurement of vital signs.
2.5. Objectives of the study
The primary clinical endpoint was a reduction in respiratory support (meaning a transition to a less intense assistance, NIV → CPAP → HFNC → LTOT → Room air), evaluated 10 and 20 days after randomization.
Secondary outcomes were the length of in-hospital stay, the time to lymphocyte number normalization, and the time to obtain a negative RT-PCR for SARS-CoV-2 on nasopharyngeal swab.
Safety outcomes included treatment-emergent adverse events, serious adverse events, and premature discontinuations of study drug.
2.6. Statistical analysis
Since no reliable data were available for an accurate calculation of the sample size, we harnessed data from our preliminary clinical experiences conducted with the administration of L-arginine in COVID-19 patients and those of a historical group of COVID-19 patients undergoing the same therapeutic regimen except the daily oral supplementation of L-arginine. Based on these data, we hypothesized a difference of at least 35% in the primary outcome between the two study groups. According to this hypothesis, we determined that it was necessary to enroll a population of 290 patients to detect a statistically significant difference with a 2-side α level of 0.05 and a power of 80%.
Two interim analyses were planned for the assessment of efficacy and tolerability after completion of the first 100 and 200 patients. The study discontinuation threshold was set at p < 0.01 for efficacy and p < 0.01 for tolerability. Enrollment was planned to continue until the Data and Safety Monitoring Board (DSMB) recommended stopping the trial for evidence of efficacy, futility, or harm, based on evaluation of all the available data, including data internal and external to the trial.
The characteristics of the participants are reported as the mean ± standard deviation (SD) for approximately normally distributed continuous variables, as the median (interquartile range [IQR]: 25th percentile, 75th percentile) for severely skewed continuous variables and as numerical values (percentages) for categorical variables. Normal distribution of the data was verified via normal probability plots and confirmed with the skewness/kurtosis test for normality. Bivariate tests were used to assess the association between the treatment with L-arginine and the baseline characteristics and treatments of the patients included in the study. Statistical significance was determined by a p value < 0.05. In the statistical analysis, differences for continuous variables were evaluated using two-sample t-test for approximately normally distributed variables and Mann-Whitney U test for severely skewed variables. Chi-square or Fisher tests were used to measure associations between dichotomous and categorical variables.
Multivariable logistic regression analysis was performed in order to investigate the association between L-arginine treatment and the primary outcome (reduction in respiratory support), adjusting for potential confounders; a backward selection method was applied to create the final model. Log-rank test was used to compare the length of in-hospital stay between the L-arginine and the placebo groups and a Cox proportional hazards regression analysis was employed to relate L-arginine treatment to length of hospitalization adjusting for likely confounders. All analyses were performed using SPSS 26.0.
2.7. Role of the funding source
Both placebo and L-arginine were kindly provided by Farmaceutici Damor S.p.A., Naples, Italy, which had no role in the design and conduct of the study, collection, management, analysis, and interpretation of the data, preparation, review, or approval of the manuscript, and decision to submit the manuscript for publication. GF, RI, and BT had full access to all data in the study and take final responsibility for manuscript submission.
3. Results
Between November 2020 and April 2021, 210 patients were screened, of whom 101 were eligible (Fig. 1); 48 patients were assigned to receive L-arginine and 53 to receive placebo; no patient in either group withdrew her/his previously given written informed consent after randomization, so 48 and 53 patients were included in the intention-to-treat (ITT) population for the active and the placebo treatment, respectively. Patients were randomized and received study treatments with a mean of 7.8 days after symptoms onset. At randomization, all patients were receiving oxygen, with or without non-invasive ventilation. Since 8 patients randomized to receive placebo and 3 in the L-arginine arm were transferred to the intensive care unit (ICU) before starting the study, we had in the per protocol (PP) analysis a total of 90 patients who started and successfully completed the efficacy and safety evaluation period (Fig. 1). A 100% protocol adherence was achieved for this interim analysis. The demographic, anthropometric, and clinical characteristics of the 90 patients are shown in Table 1. The most common comorbidity in both arms was hypertension, followed by coronary heart disease, and diabetes. There was no difference in the number of days between onset of symptoms and starting treatment in patients of the control group and those in the L-arginine group.
Fig. 1.
Study flowchart.
Table 1.
Baseline characteristics and treatments of the per-protocol analysis population. Data are presented as mean ± SD for continuous variables normally distributed and as median (IQR: 25th percentile, 75th percentile) for non-normally distributed continuous variables; percentages are reported for categorical variables. Parameters in bold denote a significant difference (p < 0.05). ALT: Alanine aminotransferase; AST: aspartate aminotransferase; BUN: blood urea nitrogen; CPAP: continuous positive airway pressure; CRP: C Reactive Protein; HFNC: high-flow nasal cannula; LMWH: low molecular weight heparin; LTOT: long-term oxygen therapy; NIV: non-invasive ventilation.
| Placebo (n = 45) | L-arginine (n = 45) | |
|---|---|---|
| Gender (M/W) (%) | 86.7/13.3 | 75.6/24.4 |
| Age (years) | 65.9 ± 11.7 | 57.4 ± 13.2 |
| Hypertension (%) | 42.2 | 31.1 |
| Coronary artery disease (%) | 17.8 | 11.1 |
| Smokers (%) | 8.9 | 6.7 |
| Obesity (%) | 11.1 | 8.9 |
| Diabetes (%) | 8.9 | 11.1 |
| Time between onset of symptoms and admission (days) | 7 (6, 10) | 6.5 (5, 10) |
| White blood cells (n/μL) | 10319.7 ± 5532.7 | 9210.0 ± 3310.0 |
| Lymphocytes (n/μL) | 8.5 (4.8, 12.6) | 7.3 (5.1, 10.4) |
| CRP (mg/L)a | 8.2 (4.8, 13.05) | 5.3 (3.1, 13) |
| D-dimer (ng/mL) | 397 (219, 1540) | 256 (183, 441) |
| AST (U/L) | 31 (26, 50) | 34 (26, 42) |
| ALT (U/L) | 42 (31, 62) | 41 (33.5, 60.5) |
| Creatinine (mg/dL) | 0.9 (0.75, 1.1) | 0.8 (0.7, 0.9) |
| BUN (mg/dL)a | 73.0 ± 45.6 | 58.1 ± 25.0 |
| Sodium (mmol/L)b | 137.5 ± 4.5 | 137.8 ± 4.4 |
| Potassium (mmol/L)b | 4.6 ± 0.9 | 4.4 ± 0.5 |
| Asthenia (%) | 91.1 | 93.3 |
| Dyspnea (%) | 95.6 | 95.6 |
| Cough (%) | 33.3 | 28.9 |
| Fever (%) | 75.6 | 73.3 |
| Sputum (%) | 8.9 | 2.2 |
| PaO2 (kPa) | 10.4 ± 1.39 | 9.92 ± 1.4 |
| PaCO2 (kPa) | 4.6 ± 0.6 | 5.09 ± 0.6 |
| P/F (PaO2/FiO2) | 154.6 ± 49.5 | 161.7 ± 62.3 |
| Remdesivir (%) | 24.4 | 31.1 |
| LMWH (%) | 93.3 | 91.1 |
| Steroids (%) | 100 | 100 |
| Respiratory support: | ||
| LTOT (%) | 42.2 | 20.0 |
| HFNC (%) | 73.3 | 62.2 |
| CPAP (%) | 2.2 | 13.3 |
| NIV (%) | 0 | 11.1 |
available in 43 patients in the placebo group and in 44 patients in the L-arginine group;
available in 44 patients in both the placebo group and the L-arginine group.
The primary clinical endpoint was the reduction in respiratory support (according to the sequence NIV → CPAP → HFNC → LTOT → Room Air) assessed 10 and 20 days after randomization. At 10-day evaluation, 44.4% in the placebo group and 71.1% in the L-arginine group (p < 0.01) had reduced the respiratory support (Fig. 2).
Fig. 2.
Respiratory support at baseline and at day 10 (A) and percentage of patients in which the respiratory support was reduced, evaluated 10 days after randomization (B) in the per-protocol analysis; *: p < 0.01.
CPAP: continuous positive airway pressure; HFNC: high-flow nasal cannula; LTOT: long-term oxygen therapy; NIV: non-invasive ventilation.
A multivariable logistic regression was performed in order to examine the magnitude of association between the treatment with L-arginine and the primary outcome and to better identify explanatory variables that were associated with a reduction of the respiratory support (Table 2). When adjusting the main analysis for likely confounders including age, gender, symptom duration, hypertension, diabetes, treatment with remdesivir or heparin, and values of creatinine, lymphocytes, and D-dimer, the odds of having a reduction in respiratory support were 6.6-fold higher in those who were in the treatment group compared with those in the placebo group (OR, 6.62; 95% CI, 1.55, 28.22; p = 0.01).
Table 2.
Multivariable logistic regression analysis assessing in the per-protocol analysis the association of L-arginine with the primary outcome at 10 days. CI: confidence interval; LMWH: low molecular weight heparin; OR: odds ratio; symptom duration before hospital admission was categorized as ≤ 8 and > 8 days.
| OR | 95% CI |
p | ||
|---|---|---|---|---|
| Lower | Upper | |||
| Age | 1.058 | 0.991 | 1.129 | 0.090 |
| Gender | 3.449 | 0.793 | 15.011 | 0.099 |
| Symptom duration | 0.710 | 0.199 | 2.531 | 0.598 |
| Hypertension | 0.590 | 0.147 | 2.361 | 0.456 |
| Diabetes | 0.011 | 0.001 | 0.389 | 0.013 |
| Creatinine | 0.749 | 0.209 | 2.683 | 0.658 |
| Lymphocytes | 1.109 | 0.994 | 1.238 | 0.065 |
| D-dimer | 1.001 | 1.000 | 1.001 | 0.060 |
| LMWH | 0.824 | 0.090 | 7.534 | 0.864 |
| Remdesivir | 1.142 | 0.302 | 4.323 | 0.845 |
| L-arginine | 6.622 | 1.554 | 28.223 | 0.01 |
On the contrary, 20 days after randomization we failed to detect any significant difference in the primary outcome between the study arms (placebo group: 13 out of 25, 52%; L-arginine group: 4 out of 13, 30.8%; non-significant). Similarly, the P/F ratio was significantly different at 10 days (Placebo: 186.4 ± 68.3 vs L-arginine: 228.3 ± 93; p = 0.02) but not at 20 days (Placebo: 231.0 ± 93 vs L-arginine: 263.5 ± 103).
The multivariable logistic regression performed on the ITT population at 10 and 20 days after randomization is presented as supplementary material (Table S1 and Table S2, respectively). Among the secondary outcome measures, differences between active treatment group vs placebo group were statistically significant only for the time to hospital discharge, which was significantly shorter in the L-arginine arm (Fig. 3). Specifically, the median time to hospital discharge in the L-arginine group and placebo group was 25 and 46 days, respectively, and the distributions in the two groups differed significantly (p < 0.001). This association remained significant in a Cox regression analysis (Table 3) using the fully adjusted multivariable model including age, gender, symptom duration, hypertension, diabetes, treatment with remdesivir or heparin, and values of creatinine, lymphocytes, and D-dimer (HR, 41.6; 95% CI, 12.18, 142.10; p < 0.0001). The variables used in the regression analyses were selected according to the available literature on COVID-197-9,11.
Fig. 3.
Kaplan-Meier curves assessing in the per-protocol analysis the length of in-hospital stay.
Table 3.
Cox proportional hazards model assessing in the per-protocol analysis the effect of L-arginine on length of in-hospital stay, adjusting for potential confounders. CI: confidence interval; HR: hazard ratio; LMWH: low molecular weight heparin; symptom duration before hospital admission was categorized as ≤ 8 and > 8 days.
| HR | 95% CI |
p | ||
|---|---|---|---|---|
| Lower | Upper | |||
| Age | 0.971 | 0.944 | 0.998 | 0.035 |
| Gender | 0.918 | 0.463 | 1.819 | 0.805 |
| Symptom duration | 1.794 | 0.992 | 3.245 | 0.053 |
| Hypertension | 0.798 | 0.441 | 1.445 | 0.457 |
| Diabetes | 1.244 | 0.445 | 3.478 | 0.678 |
| Creatinine | 0.846 | 0.544 | 1.315 | 0.457 |
| Lymphocytes | 1.02 | 0.979 | 1.064 | 0.340 |
| D-dimer | 0.999 | 0.999 | 1.001 | 0.768 |
| LMWH | 0.874 | 0.355 | 2.148 | 0.769 |
| Remdesivir | 0.986 | 0.558 | 1.744 | 0.962 |
| L-arginine | 41.599 | 12.178 | 142.102 | <0.001 |
The rate of lymphocyte number normalization was similar between the 2 study arms (17.5% and 24.4% in the placebo group and 36.1% and 20% in the L-arginine group at days 10 and 20, respectively; both not statistically significant). The time to obtain a negative RT-PCR for SARS-CoV-2 on nasopharyngeal swab was comparable between groups (20.4 ± 13.8 days in the placebo group and 24.8 ± 14.5 days in the L-arginine group, p = 0.21).
The 11 patients transferred to the ICU (8 randomized to the placebo group, 3 to the L-arginine group) deceased shortly after the transfer, before they could actually start the treatment with L-arginine or placebo. Since these patients were transferred to the ICU right after the randomization, they have been included in the ITT but not in the PP analysis. Besides, 3 other patients in the placebo arm (0 in the L-arginine arm) died after day 20. Thus, the total mortality was 3.3% in the PP analysis (Placebo: 6.7%, L-arginine: 0) and 13.9% in the ITT analysis (Placebo: 20.8%, L-arginine: 6.3%; p = 0.035).
3.1. Adverse events
Four serious adverse events were reported (pneumomediastinum, pancreatitis, and pulmonary embolism in the placebo group, pneumothorax in the L-arginine group) which were considered unrelated to the study treatment by the investigators.
4. Discussion
In this randomized clinical trial involving adults with severe COVID-19, L-arginine plus standard care compared with standard care alone significantly reduced the need for respiratory support and, foremost, the duration of hospitalization. L-arginine was not associated with increased risk of adverse events in these critically ill COVID-19 patients. Notably, the safety of oral L-arginine had been previously ascertained in other trials [26,27].
We selected the reduction of respiratory support as the primary endpoint because it comprises both pathophysiologic and clinical determinants. Thus, in order to assess the role of L-arginine supplementation in the reduction of respiratory support, we performed a multivariate analysis which demonstrated that treatment with L-arginine was the main determinant of this phenomenon, ruling out the possibility that it could be merely due to the younger age of this group of patients. This conclusion is corroborated by the observation that L-arginine supplementation was the main determinant in the reduction of in-hospital stay duration; the in-hospital stay median (IQR 25th,75th percentile) number of days was 46 (45,46) in the placebo group vs 25 (21,26) in the L-arginine group (p < 0.0001, Mann-Whitney); this finding was also confirmed after adjustment for potential confounders including age.
On the contrary, there were no significant improvements in the time to normalization of lymphopenia nor in the time to obtain a negative RT-PCR for SARS-CoV-2 on nasopharyngeal swab, two findings that may appear in contrast with a beneficial effect of L-arginine in the treatment of COVID-19. However, L-arginine may exert its favorable effect through means of an improvement of endothelial function without any interference with virus replication, which strongly affects the time to obtain a negative RT-PCR for SARS-CoV-2 on nasopharyngeal swab. Indeed, the currently accepted view of COVID-19 pathogenesis includes a viral tissue injury followed by an inflammatory host immune response, which drives hypercytokinemia and aggressive inflammation, resulting in endotheliitis, increased apoptotic activity, thrombotic events, and intravascular coagulation [14,15,28,29]. The excessive inflammatory response against SARS-CoV-2 is thought to orchestrate disease severity in patients with COVID-19 and is associated with profound lymphopenia and substantial mononuclear cell infiltration in lungs, heart, spleen, lymph nodes, intestine, and kidney, as confirmed in postmortem analyses [4,30]. In vitro studies of cell lines as well as immunohistochemichal and electron microscopy analyses of human tissues suggested the presence of SARS-CoV-2 within endothelial cells [31], [32], [33], [34]. Endothelial cell infection with consecutive inflammatory cell recruitment and endothelial dysfunction may explain the impaired microcirculation observed across vascular beds in COVID-19, triggering vasoconstriction, ischemia, and a pro-coagulant state [14]. Therefore, endotheliitis has been suggested as the major cause of systemic impaired microcirculatory function observed in different vascular beds in COVID-19 patients [14]. Our findings are consistent with previous reports indicating that after the early stage the disease may be dominated by immunopathological elements, with active viral replication playing a secondary role [4,15,[35], [36], [37], [38], [39]].
Further supporting our data, L-arginine depletion has been shown to increase the production of reactive oxygen species, exacerbating inflammation [40].
The exact mechanisms linking SARS-CoV-2 infection and lymphopenia remain not fully defined [41], [42], [43]. SARS-CoV-2 is internalized into cells via ACE2, which is widely expressed by cardiopulmonary tissues and certain hematopoietic cells like monocytes and macrophages [44]. Owing to low expression of ACE2 on T cells [45], direct viral attack of T cells via ACE2 can hardly explain the occurrence of lymphopenia. Alternatively, the reduction in lymphocyte count may be attributed to increased cell apoptosis [46]. Several investigators reported a negative correlation between T cell numbers and the concentration of cytokines including interleukin-6 (IL-6), IL-10, interferon-γ (IFN-γ), and tumor necrosis factor-α (TNF-α) in COVID-19 patients [46,47]. Highly dysregulated cytokine release might promote T lymphocyte apoptosis by activating extrinsic and intrinsic apoptosis pathways during SARS-CoV-2 infection [48,49]. Whatever the case, the observation that in our study L-arginine is able to shorten the disease duration without reducing the time to normalization of lymphocyte number is not surprising even because the whole study population during hospitalization received glucocorticoids, which in severe COVID-19 patients may promote lymphocyte depletion [9,36,42,50]. The activity of arginase–the enzyme responsible for metabolizing L-arginine to ornithine and urea–could also have a negative impact on T cell function, which has been shown to depend on L-arginine supply [51], [52], [53], [54]. In vitro assays revealed that T cell proliferative capacity is significantly reduced among COVID-19 patients and can be restored through L-arginine supplementation [18]. Tadié and collaborators have demonstrated that COVID-19 patients with severe acute respiratory distress syndrome present an elevated number of myeloid-derived suppressor cells, which was directly correlated to enhanced arginase activity, effectively depleting L-arginine from the microenvironment [18,55]. Ergo, the effects of L-arginine on T cell function may be of greater importance than the mere impact on lymphocyte number.
Intriguingly, diabetes was significantly associated with the primary outcome but not with the hospitalization length; this observation could be related to the previously reported detrimental role of diabetes on lung function [56,57]; the pathophysiology of pulmonary disease in diabetic patients is complex and multifactorial and some of the proposed underlying mechanisms include endothelial dysfunction, microangiopathy of alveolar capillaries, oxidative stress, autonomic neuropathy, and alterations of connective tissue [58], [59], [60].
At day 20 after randomization, we did not detect any significant difference in the rate of patients that transitioned to a lower respiratory support; however a nearly twice the number of participants in the L-arginine arm had been already discharged from the hospital by this time, leaving on a small number of patients remaining-likely the sickest patients of the cohort; therefore, we can speculate that many of the patients with earlier discharge would have demonstrated improvements had they been analyzed at day 20.
The main limitation of our study is the relatively small number of patients included in this first interim analysis; therefore, further studies in larger populations are warranted, as also suggested by the high confidence interval (CI) around L-arginine in Tables 2 and 3. Additionally, we did not demonstrate that L-arginine administration actually increased arginine availability and we do not have data on respiratory rates and weight evolution. Of note, the enrollment took place in a setting dedicated to patients with severe forms of COVID-19; since all patients were hospitalized because of severe COVID-19, the generalizability of the results to patients with milder COVID-19 remains unclear. Some may argue that it might be easier for the sicker patients to transition to a lower level of respiratory support, potentially biasing the study towards the treatment arm; however, as shown in the stacked bar chart depicted in Fig. 2A, the main effects of L-arginine were especially evident in transitioning from HFNC to LTOT and from CPAP to HFNC.
Strengths of this trial included its blinded, placebo-controlled design, high adherence to the study protocol, rigorous monitoring for safety and adverse events, and rapid recruitment. Moreover, all patients were analyzed according to their randomization group, and follow-up was completed. We included patients with respiratory symptoms for up to 15 days prior to randomization while some trials of antiviral medications limit enrollment to patients with symptoms for a shorter duration, most likely in an effort to enrich the population for patients most likely to benefit.
Finally, it should be noted that both endothelial dysfunction and T cell impairment are consequences of low L-arginine bioavailability and contribute to complications of COVID-19, making a role for L-arginine supplementation biologically plausible. Thus, our results may have important clinical implications for COVID-19 treatment especially in low-resource environments were vaccination is not widely available given the safety and the affordability of oral L-arginine.
Taken together, the results of this interim analysis indicate that adding L-arginine to standard therapy in patients with severe COVID-19 significantly decreases the length of hospitalization and reduces the respiratory support assessed 10, but not 20, days after starting the treatment. The other secondary outcomes were not significantly different between groups.
Declaration of Competing Interest
We declare no competing interests.
Acknowledgments
Data sharing statement
All relevant data are available within the manuscript.
Contributors
Giuseppe Fiorentino*: Conceptualisation, Data Curation. Antonietta Coppola*: Data Collection, Data Curation. Raffaele Izzo: Formal analysis, Validation, Writing - Original draft. Anna Annunziata: Data Collection. Mariano Bernardo: Data Collection. Angela Lombardi: Data Analysis, Writing - Review and editing. Valentina Trimarco: Validation, Data Curation. Gaetano Santulli: Conceptualisation, Study Design, Data Analysis, Writing - Review and editing Bruno Trimarco: Conceptualisation, Study Design, Writing - Original draft. *: contributed equally.
Funding
Both placebo and L-arginine were kindly provided by Farmaceutici Damor S.p.A., Naples.
Footnotes
Supplementary material associated with this article can be found in the online version at doi:10.1016/j.eclinm.2021.101125.
Appendix. Supplementary materials
References
- 1.Cummings M.J., Baldwin M.R., Abrams D. Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York City: a prospective cohort study. Lancet. 2020;395(10239):1763–1770. doi: 10.1016/S0140-6736(20)31189-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gambardella J., Coppola A., Izzo R., Fiorentino G., Trimarco B., Santulli G. Role of endothelial miR-24 in COVID-19 cerebrovascular events. Crit Care. 2021;25(1):306. doi: 10.1186/s13054-021-03731-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Estenssoro E., Loudet C.I., Rios F.G. Clinical characteristics and outcomes of invasively ventilated patients with COVID-19 in Argentina (SATICOVID): a prospective, multicentre cohort study. Lancet Respir Med. 2021;9(9):989–998. doi: 10.1016/S2213-2600(21)00229-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Tay M.Z., Poh C.M., Renia L., MacAry P.A., Ng L.F.P. The trinity of COVID-19: immunity, inflammation and intervention. Nat Rev Immunol. 2020;20(6):363–374. doi: 10.1038/s41577-020-0311-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Paules C.I., Fauci A.S. COVID-19: the therapeutic landscape. Med (N Y) 2021;2(5):493–497. doi: 10.1016/j.medj.2021.04.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rubin D., Chan-Tack K., Farley J., Sherwat A. FDA approval of remdesivir-a step in the right direction. N Engl J Med. 2020;383(27):2598–2600. doi: 10.1056/NEJMp2032369. [DOI] [PubMed] [Google Scholar]
- 7.Gottlieb R.L., Nirula A., Chen P. Effect of bamlanivimab as monotherapy or in combination with etesevimab on viral load in patients with mild to moderate COVID-19: a randomized clinical trial. JAMA. 2021;325(7):632–644. doi: 10.1001/jama.2021.0202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Goligher E.C., Bradbury C.A., McVerry B.J. Therapeutic anticoagulation with heparin in critically ill patients with COVID-19. N Engl J Med. 2021;385(9):777–789. doi: 10.1056/NEJMoa2103417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Horby P., Lim W.S., Emberson J.R. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021;384(8):693–704. doi: 10.1056/NEJMoa2021436. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Sanders J.M., Monogue M.L., Jodlowski T.Z., Cutrell J.B. Pharmacologic treatments for coronavirus disease 2019 (COVID-19): a review. JAMA. 2020;323(18):1824–1836. doi: 10.1001/jama.2020.6019. [DOI] [PubMed] [Google Scholar]
- 11.Li L., Zhang W., Hu Y. Effect of convalescent plasma therapy on time to clinical improvement in patients with severe and life-threatening COVID-19: a randomized clinical trial. JAMA. 2020;324(5):460–470. doi: 10.1001/jama.2020.10044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hall M.D., Anderson J.M., Anderson A. Report of the national institutes of health SARS-CoV-2 antiviral therapeutics summit. J Infect Dis. 2021;224(Supplement 1):S1–S21. doi: 10.1093/infdis/jiab305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Chen P., Nirula A., Heller B. SARS-CoV-2 neutralizing antibody LY-CoV555 in outpatients with Covid-19. N Engl J Med. 2021;384(3):229–237. doi: 10.1056/NEJMoa2029849. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Sardu C., Gambardella J., Morelli M.B., Wang X., Marfella R., Santulli G. Hypertension, thrombosis, kidney failure, and diabetes: is COVID-19 an endothelial disease? A comprehensive evaluation of clinical and basic evidence. J Clin Med. 2020;9(5):1417. doi: 10.3390/jcm9051417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Budinger G.R.S., Misharin A.V., Ridge K.M., Singer B.D., Wunderink R.G. Distinctive features of severe SARS-CoV-2 pneumonia. J Clin Invest. 2021;131(14):e149412. doi: 10.1172/JCI149412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Barberis E., Timo S., Amede E. Large-scale plasma analysis revealed new mechanisms and molecules associated with the host response to SARS-CoV-2. Int J Mol Sci. 2020;21(22):8623. doi: 10.3390/ijms21228623. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rees C.A., Rostad C.A., Mantus G. Altered amino acid profile in patients with SARS-CoV-2 infection. Proc Natl Acad Sci USA. 2021;118(25) doi: 10.1073/pnas.2101708118. e2101708118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Reizine F., Lesouhaitier M., Gregoire M. SARS-CoV-2-induced ARDS associates with MDSC expansion, lymphocyte dysfunction, and arginine shortage. J Clin Immunol. 2021;41(3):515–525. doi: 10.1007/s10875-020-00920-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gambardella J., Khondkar W., Morelli M.B., Wang X., Santulli G., Trimarco V. Arginine and endothelial function. Biomedicines. 2020;8(8):277. doi: 10.3390/biomedicines8080277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Angus D.C., Derde L., Al-Beidh F. Effect of hydrocortisone on mortality and organ support in patients with severe COVID-19: The REMAP-CAP COVID-19 corticosteroid domain randomized clinical trial. JAMA. 2020;324(13):1317–1329. doi: 10.1001/jama.2020.17022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Gambardella J., Fiordelisi A., Spigno L. Effects of chronic supplementation of L-arginine on physical fitness in water polo players. Oxid Med Cell Longev. 2021;2021 doi: 10.1155/2021/6684568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Pisani L., Betti S., Biglia C. Effects of high-flow nasal cannula in patients with persistent hypercapnia after an acute COPD exacerbation: a prospective pilot study. BMC Pulm Med. 2020;20(1):12. doi: 10.1186/s12890-020-1048-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Horovitz J.H., Carrico C.J., Shires G.T. Pulmonary response to major injury. Arch Surg. 1974;108(3):349–355. doi: 10.1001/archsurg.1974.01350270079014. [DOI] [PubMed] [Google Scholar]
- 24.Petersson J., Glenny R.W. Gas exchange and ventilation-perfusion relationships in the lung. Eur Respir J. 2014;44(4):1023–1041. doi: 10.1183/09031936.00037014. [DOI] [PubMed] [Google Scholar]
- 25.Sharma S., Hashmi M.F., Burns B. StatPearls; Treasure Island (FL): 2021. Alveolar gas equation. [PubMed] [Google Scholar]
- 26.Bednarz B., Jaxa-Chamiec T., Maciejewski P. Efficacy and safety of oral l-arginine in acute myocardial infarction. Results of the multicenter, randomized, double-blind, placebo-controlled ARAMI pilot trial. Kardiol Pol. 2005;62(5):421–427. [PubMed] [Google Scholar]
- 27.Onalo R., Cooper P., Cilliers A. Randomized control trial of oral arginine therapy for children with sickle cell anemia hospitalized for pain in Nigeria. Am J Hematol. 2021;96(1):89–97. doi: 10.1002/ajh.26028. [DOI] [PubMed] [Google Scholar]
- 28.Gambardella J., Santulli G. What is linking COVID-19 and endothelial dysfunction? Updates on nanomedicine and bioengineering from the 2020 AHA scientific sessions. Eur Heart J Cardiovasc Pharmacother. 2021;7(3):e2–e3. doi: 10.1093/ehjcvp/pvaa145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Vabret N., Britton G.J., Gruber C. Immunology of COVID-19: current state of the science. Immunity. 2020;52(6):910–941. doi: 10.1016/j.immuni.2020.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Dorward D.A., Russell C.D., Um I.H. Tissue-specific immunopathology in fatal COVID-19. Am J Respir Crit Care Med. 2021;203(2):192–201. doi: 10.1164/rccm.202008-3265OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Varga Z., Flammer A.J., Steiger P. Endothelial cell infection and endotheliitis in COVID-19. Lancet. 2020;395(10234):1417–1418. doi: 10.1016/S0140-6736(20)30937-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Ackermann M., Verleden S.E., Kuehnel M. Pulmonary vascular endothelialitis, thrombosis, and angiogenesis in Covid-19. N Engl J Med. 2020;383(2):120–128. doi: 10.1056/NEJMoa2015432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Fox S.E., Li G., Akmatbekov A. Unexpected features of cardiac pathology in COVID-19 infection. Circulation. 2020;142(11):1123–1125. doi: 10.1161/CIRCULATIONAHA.120.049465. [DOI] [PubMed] [Google Scholar]
- 34.Stahl K., Brasen J.H., Hoeper M.M., David S. Direct evidence of SARS-CoV-2 in gut endothelium. Intensive Care Med. 2020;46(11):2081–2082. doi: 10.1007/s00134-020-06237-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Datta S.D., Talwar A., Lee J.T. A proposed framework and timeline of the spectrum of disease due to SARS-CoV-2 infection: illness beyond acute infection and public health implications. JAMA. 2020;324(22):2251–2252. doi: 10.1001/jama.2020.22717. [DOI] [PubMed] [Google Scholar]
- 36.Taboada M., Caruezo V., Naveira A., Atanassoff P.G. Corticosteroids and the hyper-inflammatory phase of the COVID-19 disease. J Clin Anesth. 2020;66 doi: 10.1016/j.jclinane.2020.109926. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Garcia L.F. Immune response, inflammation, and the clinical spectrum of COVID-19. Front Immunol. 2020;11:1441. doi: 10.3389/fimmu.2020.01441. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Manson J.J., Crooks C., Naja M. COVID-19-associated hyperinflammation and escalation of patient care: a retrospective longitudinal cohort study. Lancet Rheumatol. 2020;2(10):e594–e602. doi: 10.1016/S2665-9913(20)30275-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Lavine J.S., Bjornstad O.N., Antia R. Immunological characteristics govern the transition of COVID-19 to endemicity. Science. 2021;371(6530):741–745. doi: 10.1126/science.abe6522. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Lucas R., Czikora I., Sridhar S. Arginase 1: an unexpected mediator of pulmonary capillary barrier dysfunction in models of acute lung injury. Front Immunol. 2013;4:228. doi: 10.3389/fimmu.2013.00228. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Huang I., Pranata R. Lymphopenia in severe coronavirus disease-2019 (COVID-19): systematic review and meta-analysis. J Intensive Care. 2020;8:36. doi: 10.1186/s40560-020-00453-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Xiang Q., Feng Z., Diao B. SARS-CoV-2 induces lymphocytopenia by promoting inflammation and decimates secondary lymphoid organs. Front Immunol. 2021;12 doi: 10.3389/fimmu.2021.661052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Cheng L.L., Guan W.J., Duan C.Y. Effect of recombinant human granulocyte colony-stimulating factor for patients with coronavirus disease 2019 (COVID-19) and lymphopenia: a randomized clinical trial. JAMA Intern Med. 2021;181(1):71–78. doi: 10.1001/jamainternmed.2020.5503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Gomez-Rial J., Rivero-Calle I., Salas A., Martinon-Torres F. Role of monocytes/macrophages in Covid-19 pathogenesis: implications for therapy. Infect Drug Resist. 2020;13:2485–2493. doi: 10.2147/IDR.S258639. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Song X., Hu W., Yu H. Little to no expression of angiotensin-converting enzyme-2 on most human peripheral blood immune cells but highly expressed on tissue macrophages. Cytometry A. 2020 doi: 10.1002/cyto.a.24285. In press. [DOI] [PubMed] [Google Scholar]
- 46.Diao B., Wang C., Tan Y. Reduction and functional exhaustion of T cells in patients with coronavirus disease 2019 (COVID-19) Front Immunol. 2020;11:827. doi: 10.3389/fimmu.2020.00827. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Xu B., Fan C.Y., Wang A.L. Suppressed T cell-mediated immunity in patients with COVID-19: a clinical retrospective study in Wuhan, China. J Infect. 2020;81(1):e51–e60. doi: 10.1016/j.jinf.2020.04.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Domingo P., Mur I., Pomar V., Corominas H., Casademont J., de Benito N. The four horsemen of a viral apocalypse: the pathogenesis of SARS-CoV-2 infection (COVID-19) EBioMedicine. 2020;58 doi: 10.1016/j.ebiom.2020.102887. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Costela-Ruiz V.J., Illescas-Montes R., Puerta-Puerta J.M., Ruiz C., Melguizo-Rodriguez L. SARS-CoV-2 infection: the role of cytokines in COVID-19 disease. Cytokine Growth Factor Rev. 2020;54:62–75. doi: 10.1016/j.cytogfr.2020.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Yang L., Liu S., Liu J. COVID-19: immunopathogenesis and immunotherapeutics. Signal Transduct Target Ther. 2020;5(1):128. doi: 10.1038/s41392-020-00243-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Rodriguez P.C., Ochoa A.C. Arginine regulation by myeloid derived suppressor cells and tolerance in cancer: mechanisms and therapeutic perspectives. Immunol Rev. 2008;222:180–191. doi: 10.1111/j.1600-065X.2008.00608.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Geiger R., Rieckmann J.C., Wolf T. L-arginine modulates T cell metabolism and enhances survival and anti-tumor activity. Cell. 2016;167(3):829–842. doi: 10.1016/j.cell.2016.09.031. e13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Wherry EJ. T cell exhaustion. Nat Immunol. 2011;12(6):492–499. doi: 10.1038/ni.2035. [DOI] [PubMed] [Google Scholar]
- 54.Rodriguez P.C., Hernandez C.P., Morrow K. L-arginine deprivation regulates cyclin D3 mRNA stability in human T cells by controlling HuR expression. J Immunol. 2010;185(9):5198–5204. doi: 10.4049/jimmunol.1001224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Dean M.J., Ochoa J.B., Sanchez-Pino M. Severe COVID-19 Is characterized by an impaired Type I interferon response and elevated levels of arginase producing granulocytic myeloid derived suppressor cells. Front Immunol. 2021;12:695972. doi: 10.3389/fimmu.2021.695972. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Goldman M.D. Lung dysfunction in diabetes. Diabetes Care. 2003;26(6):1915–1918. doi: 10.2337/diacare.26.6.1915. [DOI] [PubMed] [Google Scholar]
- 57.Pitocco D., Fuso L., Conte E.G. The diabetic lung–a new target organ? Rev Diabet Stud. 2012;9(1):23–35. doi: 10.1900/RDS.2012.9.23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Lopez-Lopez J.G., Moral-Sanz J., Frazziano G. Diabetes induces pulmonary artery endothelial dysfunction by NADPH oxidase induction. Am J Physiol Lung Cell Mol Physiol. 2008;295(5):L727–L732. doi: 10.1152/ajplung.90354.2008. [DOI] [PubMed] [Google Scholar]
- 59.Kolahian S., Leiss V., Nurnberg B. Diabetic lung disease: fact or fiction? Rev Endocr Metab Disord. 2019;20(3):303–319. doi: 10.1007/s11154-019-09516-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Tai H., Jiang X.L., Yao S.C. Vascular endothelial function as a valid predictor of variations in pulmonary function in T2DM patients without related complications. Front Endocrinol. 2021;12 doi: 10.3389/fendo.2021.622768. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.



