Abstract
Background & aims
Severe acute COVID-19 has taken on pandemic proportions with growing interest in identification of prognostic factors for mortality. Standardized bioelectrical impedance (BI) phase angle (SPhA), which is PhA adjusted by age and sex, has been related to mortality in patients with several diseases but never investigated in COVID-19. Inflammation, a consequence of COVID-19 infection, affects fluid status (hydration) and can be identified with PhA. The aim of this study was to determine the predictive role of PhA on 90 days survival of adults with COVID-19.
Methods
We studied 127 consecutive patients diagnosed with COVID-19. BI measurements determined with a 50 kHz phase-sensitive BI device, body composition parameters and laboratory markers were evaluated as predictors of mortality.
Results
Non-surviving COVID-19 patients had significantly lower PhA and SPhA values (p < 0.001) and increased hydration (p < 0.001) compared to surviving patients. Patients in the lowest SPhA quartile had increased (p < 0.001) mortality and hospital stay, hyperhydration (p < 0.001), increased inflammation biomarkers [CRP (p < 0.001)], decreased nutritional parameters: body mass cell index [BCMI (p < 0.001) albumin (p < 0.001)], and reduced other biomarkers [D-dimer (p = 0.002)].
Multivariate analysis (Cox regression) revealed that PhA and hydration status, adjusted for age, sex, BMI, diabetes, hypertension, dyslipidaemia or heart disease, were associated (p < 0.001) with increased mortality. The hazard ratio was 2.48 (95% CI, 1.60–3.84, p < 0.001) for PhA and 1.12 (95% CI, 1.04–1.20, p = 0.003) for hydration percentage. PhA <3.95° was the cut-off for predicting mortality in acute COVID-19 with 93.8% sensitivity and 66.7% specificity. PhA offers greater sensitivity as a predictive prognostic test at admission, compared to the established analytical parameters of poor prognosis (CRP, lymphocytes, prealbumin).
Conclusions
Low PhA (<3.95°), independent of age, sex, BMI, and comorbidities, is a significant predictor of mortality risk in COVID-19. These findings suggest that the evaluation of body composition with single-frequency phase-sensitive BI measurements should be included in the routine clinical assessment of COVID-19 patients at hospital admission to identify patients at increased mortality risk.
Keywords: Bioelectrical impedance, COVID-19, Phase angle, Survival analysis, Mortality
1. Background
Severe acute respiratory syndrome coronavirus-2, SARS-CoV-2, resulted in the pandemic of COVID-19. It has a broad range of presentation, from asymptomatic disease to severe interstitial pneumonia and respiratory distress syndrome, which can lead to extensive pulmonary fibrosis and multi-organ failure [1]. Knowledge of factors linked with severity and outcome of the disease are limited. Current studies have included either few patients with severe outcomes [2] or have not compared patients with severe disease with those with less virulent disease [3,4], making it more complicated to identify and assess characteristics related with poor outcomes. Few large studies [1,5,6] have conducted multivariable analysis to identify the strongest risk factors for mortality including patient characteristics (age, pre-existing comorbidities, etc.) or standard clinical laboratory measurements (D-dimer, lymphopenia, ferritin, etc.).
Nutritional state of patients with COVID-19 at admission is not well characterized and thus its prognostic role is unknown [7]. The type of malnutrition suffered by these patients is an acute state with an increased need for protein, especially in critical patients [8,9], and it is unknown if increased protein in the early phase can attenuate muscle loss [10]. Therefore, it is important to evaluate body composition, as it is an acute state with an increased need for protein that may accelerate muscle loss.
Although dual x-ray absorptiometry (DXA) [11] is a conventional method to meet this need, it is impractical during the COVID-19 pandemic. Alternatively, bioelectrical impedance (BI) is a bedside, radiation-free, low-cost and suitable method for evaluating body composition that can be applied to COVID-19. BI uses whole-body measurements to classify and monitor hydration and cell mass independently of multiple regression equations [11,12]. Resistance (R) is the opposition of flow of low-level alternating current due to ionic fluids, and reactance (Xc) is the delay of current entry into cells related to cell membranes and cell interfaces [11,12]. Phase angle (PhA) describes the lag between voltage and current and is directly measured by phase-sensitive devices, using the geometric relationship between R and Xc. PhA characterizes fluid distribution between the extracellular and intracellular compartments (E/I). Recently, standardized PhA (SPhA) has gained interest because of the ability to compare an individual PhA value to normative data, stratified for age and sex, of the healthy population [12].
PhA is a general indicator of nutritional status, but not in patients with an increased inflammatory process as seen in SARS-CoV-2 infection [13,14], in which the hyperhydration contributes to a decline in the PhA independent of body composition [15]. The interpretation of PhA requires consideration of the impedance vector to identify expansión or constriction of total body water [11]. Thus, PhA is a biomarker for both malnutrition and inflammatory status, which can accompany these acute and serious disorders [11]. PhA is a unique predictor of mortality in diverse clinical conditions and a potentially useful screening tool for prognosis [11,16].
PhA and SPhA are BIA measurements that are novel options for practical assessment and clinical evaluation of impaired nutritional status and prognosis among hospitalized COVID-19 patients and could potentially contribute to enhanced patient care and clinical outcomes. A recent observational cross-sectional cohort study reported that a lower PhA increased the odds of severe COVID-19 and mortality during a 28-d period [17]. Thus, there is a need to determine the prognostic value of PhA and SPhA among COVID-19 patients during acute infection.
The aim of the present study was to ascertain whether PhA and SPhA have independent prognostic value for mortality risk in a population of patients admitted with COVID-19.
2. Patients and methods
2.1. Setting study
In this single-centre, longitudinal cohort study, we enrolled a sample of patients admitted consecutively to a hospital area care for COVID-19 in Infectious Disease Unit, in Virgen de la Victoria Hospital, measured between April 6th and April 17th, 2020 and we followed for 90 days outcomes. All patients were diagnosed with COVID-19 pneumonia according to World Health Organization interim guidance (WHO) [18] with SARS symptoms and tested by nasopharyngeal samples at admission using real-time reverse transcriptase–polymerase chain reaction assays. This study was approved the Ethics Committee of the Virgen de la Victoria. All patients included in our study met inclusion criteria (COVID-19 pneumonia, agreed to participate in the study by accepted informed consent), and none exclusion criteria (participation declined or inability of performing the measurement by BIA: ethnicity, extensive skin lesions, extravasation of fluids through the route and local hematomas, amputation…). A flow chart diagram shows the patients selection process for our study (Supplemental Fig. 1).
2.2. Phase angle and others parameters
We measured the PhA of the patients within the first 72 h after hospital admission. Whole body BI measurements were obtained with a 50 kHz, phase-sensitive impedance analyser (BIA 101 Whole Body Bioimpedance Vector Analyzer (AKERN, Italy)) that introduces 800 μA [19,20] using tetrapolar electrodes positioned on the right hand and foot. The body consists of complex circuits composed of resistive (R) and capacitive (Xc) elements that, when stimulated with an alternating current, experience a frequency-dependent delay in the current in relation to the flow of voltage. This phenomenon is measured as an out-of-phase or delay of current caused by cell membranes, and is the ratio between capacitive resistance (Xc) and resistance: Xc/R customarily is expressed in degrees as PhA,° = [arctan (Xc/R) × (180°/π)].
An individual SPhA value was determined from the sex- and age-matched reference population value by subtracting the reference PhA value from the observed patient PhA and then, dividing the result by the respective age- and sex-reference standard deviation (SD), SPhA = [(measured PhA – mean population reference PhA)/SD of the reference population PhA]: [21].
We used a standardized quality assurance protocol to reduce measurement variability. Daily assessment of technical accuracy of the BIA instrument used a precision circuit supplied by the BIA device manufacturer (Akern). All measured R and capacitance values were consistently ±1 Ω of the 385 Ohm reference value. We also determined the in vivo reproducibility of the BIA measurements and found coefficients of variation (CV) of 1–2% for R (resistance) and Xc (reactance).
Body weight and standing height were determined at admission and before BI measurement; weight measurements with a scale with 100g sensitivity; while height measurements were measured with a 2 mm sensitivity laser height rod. All BI measurements were obtained with the patient supine on a hospital bed. Because fluid shifts occur after moving from standing to recumbency and directly affect R and Z values, we waited five minutes in supine position before obtaining BI measurements, then established stabilization of BIA measurement values (±2 Ω for R and ±1 Ω for Xc).
Bioelectrical impedance vector analysis (BIVA) was performed [20] using the RXc graph to classify hydration status and body cell mass. Xc reflects the capacitance produced by cell membranes of soft tissues, and is positively related with body cell mass (BCM), while R is negatively related with the proportion body water. PhA is influenced by both Xc and R, and is positively related to BCM [20], and negatively related to the hydration percentage [total body water/fat free mass (TBW/FFM, %)] [23]. BIVA emphasizes the position of the impedance vector, derived from R and Xc values normalized by body height (H, m), on the R/Xc graph in relation to tolerance ellipses generated from sex-specific, healthy reference population (e.g., 50, 75 and 97% comparable to 1, 2, and SD) [20]. Individual and group vectors in different regions of the reference ellipses have specific body composition interpretations. Vector lengths of dehydrated individuals are elongated and lie toward the upper pole of the ellipse, whereas those with oedema are shortened and positioned in the lower pole. Additionally, individuals characterized by low soft tissue mass lie on the right side of the ellipse, and those with high soft tissue mass on the left side. BI measurements of patients were standardized for sex and age using data from healthy Italian adults [19,24].
Other body composition parameters in admitted COVID-19 patients included body mass index (BMI, kg/m2) with BI-derived estimates of body cell mass (BCM, kg) and hydration state (TBW/FFM, %). Euhydration is described as 72.7–74.3% with over-hydration exceeding 74.3% + 1 SD of euhydration and dehydration less than 72.7%, -1SD of euhydration [[25], [26], [27]].
2.3. Clinical and analytical variables
We determined the following clinical assessments: age, sex, any cardiovascular comorbidities (e.g., history of diabetes, hypertension, dyslipidaemia, obesity, heart disease, or pulmonary disease), and laboratory tests [lymphopenia (Sysmex XN-10), d-dimer (ng/mL), fibrinogen (mg/dL), international normalized ratio (INR) (Sysmex CS 2500), ferritin (ng/mL), prealbumin (mg/dL) (Atellica Siemens), albumin (g/dL), C-reactive protein (CRP, mg/L) (Dimension EXL 200 S) and CRP/prealbumin ratio].
2.4. Sample size calculation
We tested the hypothesis that PhA and SPhA were independent predictors of 90-d mortality in a multivariate model. We calculated the sample size using the findings of Stapel et al. [28], where the effect of PhA in the mortality showed a OR of 0.57 with a mortality rate of 20% in the low compared to the normal PhA (5%) group. Thus, an alpha error of 0.05, a power of 80% and a loss rate of 10%, a minimum of 66 patients were needed to attain sufficient power. To account lower actual mortality, we aimed to recruit 127 patients.
2.5. Statistical analysis
Statistical analyses of the data were primarily performed using the SPSS program (version 22.0.0 for Windows, SPSS Iberica, Spain). We used descriptive statistics to characterize our cohort of patients. Baseline characteristics were expressed as median (interquartile range) for continuous variables and as proportions for categorical variables. Furthermore, we categorized SPhA into quartiles (Q) as Q1 [lower 25th percentiles of SPhA (<-2)], Q2 [25th to 50th percentiles of SPhA (−1.9 to −0.8)], Q3 [50th to 75th percentiles of SPhA (−0.7 to 0.2)] and Q4 [more than 75th percentile of SPhA (≥0.3)]. We compared our quartiles of SPhA with either ANOVA test or Friedman test according to their distribution. Continuous variables were compared with Student's T-test or Mann–Whitney U test according to their distribution. Categorical variables were compared with the Chi squared (or Fisher's exact test). We also analysed the relationship using Pearson or Spearman correlations models according to normal distribution. In multivariate analysis, Cox proportional-hazards regression was used to assess the relationship between Raw BIA and mortality in COVID-19 patients. Hazard ratio (HR) and their 95% confidence intervals (CI) were calculated. We used two models; in the first model we analysed PhA measurements, in the second one we analysed degree of hydration (%). The HR for death was expressed per 1° decrease in PhA and 1% increase in hydration percentage. To prevent potential confounding factors, the results were adjusted for several covariates that were known as potential risk or protective factors for mortality: age (years, continuous); sex (male or female); BMI (kg/m2, continuous); history of diabetes mellitus (absence or diagnosis); high blood pressure (absence or diagnosis); dyslipidaemia (absence or diagnosis); heart disease (absence or diagnosis) and degree of hydration (%, continuous). We constructed three adjusted models: Model 1: adjusted for sex, age and BMI. Model 2: adjusted for sex, age, BMI, previous diagnosis of T2D, high blood pressure, dyslipidaemia or heart disease. Model 3: adjusted for sex, age, BMI, previous diagnosis of T2D, high blood pressure, dyslipidaemia or heart disease, and degree of hydration. Statistical significance was set at p < 0.05.
The Kaplan–Meier product-limit estimator at 90 d was used to calculate the cumulative probability of death, to estimate survival and to evaluate the difference among the SPhA quartiles. The Kaplan–Meier survival curves was compared using log-rank (Mantel–Cox) test. The time of origin was the admission day. The event was defined death and all cases were censored at their last observation. Differences were considered statistically significant with p < 0.05.
Evaluation of PhA diagnostic performance was based on the receiver operating characteristic (ROC) curve and the area under the curve (AUC). We estimated the accuracy of PhA using AUC by constructing a plot of sensitivity versus 1-specifity. ROC curves were used to determine the optimal cut-off values. These optimal cut-off points for each PhA measurement were determined by the point of convergence for greatest sensitivity and specificity.
Our research compares the value of PhA and SPhA with established indicators of prognosis in the COVID-19 patients (CRP, prealbumin, lymphocytes).
3. Results
A total of 127 patients was admitted in the area of hospital care for COVID-19 and enrolled in the present study. After 90 d, 111 patients (87.4%) had been discharged alive and 16 (12.6%) had died. COVID-19 study participants were predominantly males (59.1%), and their median [interquartile range (IQR)] age was 69 y (59–80 y). Of the total, 34.6% were classified as overweight (BMI 25–30 kg/m2) and 26% as obese (BMI >30 kg/m2). Presence of comorbidities was diabetes (29.1%), dyslipidaemia (40.9%), arterial hypertension (59.1%), cardiovascular disease (20.5%), lung disease (chronic obstructive pulmonary disease or asthma, 16.5%) and chronic renal failure (14.2%). Among the patients, 18.1% required care in intensive care unit (ICU) along their admission. The median length of hospital stay was 15 d (12–27 d) for general ward, and 47 d (25–60, p < 0.001) for ICU patients.
Table 1 shows the physical characteristics and blood biochemical measurements of the COVID-19 patients. At the time of BI measurement, the patients' temperature was normal, being under antipyretic treatment when indicated.
Table 1.
Physical characteristics and biochemical measurements of COVID-19 patients related to survival and mortality.
| COVID-19 Patients |
COVID-19 Survivors |
COVID-19 Non-survivors |
||
|---|---|---|---|---|
| Median (IQR) (n = 127) |
Median (IQR) (n = 111) |
Median (IQR) (n = 16) |
Pa | |
| Age (years) | 69 (59–80) | 68 (56–77) | 84 (70–88) | 0.001 |
| Male, n (%) | 75 (59.1) | 66 (59.5) | 9 (56.3) | 0.807 |
| BMI (kg/m2) | 28.2 (25.7–31.8) | 28.4 (25.8–32.4) | 26.0 (24.8–30.1) | 0.042 |
| PhA (°) | 4.4 (3.2–5.4) | 4.5 (3.5–5.5) | 2.8 (2.08–3.68) | <0.001 |
| SPhA | −0.8 (−2.0–0.2) | −0.7 (−1.8–0.3) | −2.95 (−3.6–−1.3) | <0.001 |
| R/H (Ω/m) | 302.5 (272.2–366.3) | 301.7 (272.2–363.5) | 334.6 (251.5–370.3) | 0.769 |
| Xc/H (Ω/m) | 24.7 (16.3–31.1) | 25.3 (18.9–32.4) | 15.0 (10.5–22.6) | 0.001 |
| BCM (kg) | 21.4 (16.3–27.9) | 23 (18.5–31.5) | 14.2 (10.2–18.0) | <0.001 |
| Hydration (%) | 73.8 (73.3–84.3) | 73.7 (73.2–82.1) | 85.2 (76.9–89.3) | 0.001 |
| Lymphocytes | 1520 (910–1960) | 1590 (980–2040) | 755 (543–1195) | <0.001 |
| CRP (mg/L) | 16.7 (5.0–59.9) | 14.3 (4.2–44.7) | 97.5 (24.4–199.6) | <0.001 |
| Fibrinogen (mg/dL) | 401.8 (307.5–514.5) | 384.2 (305–512.2) | 498⋅6 (399.9–790.4) | 0.032 |
| INR | 1.06 (1.01–1.14) | 1.06 (0.99–1.13) | 1⋅14 (1.05–1.53) | 0.004 |
| D-dimer (ng/mL) | 1211 (658.8–2860.5) | 1156 (583–2571) | 1758 (1074–3584) | 0.179 |
| Ferritin (ng/mL) | 482 (254.7–933.6) | 455.6 (247.8–917.3) | 486.5 (314.7–955.1) | 0.562 |
| Pre-albumin (mg/dL) | 18.8 (11.9–26.2) | 19 (14.35–27.9) | 8.3 (7–17.7) | 0.005 |
| CRP/Pre-albumin | 0.19 (0.02–0.59) | 0.09 (0.02–0.31) | 1.19 (0.24–3.22) | 0.002 |
P for comparison of non-survivors and survivors. IQR: interquartile range; PhA: phase angle; SPhA: standardized phase angle; R/H: Resistance/height; Xc/H: reactance/height; BMI body mass index; FFM: fat free mass; FM: fat mass; BCM: body cell mass; ECW: extracellular water; CRP: C-reactive protein; INR: International normalized ratio.
3.1. PhA and 90-d mortality
The wide distribution of individual impedance point vectors of the COVID-19 patients show a pattern of vector distribution in quadrants based on their nutritional and hydration characteristics (Fig. 1 ). The vertical axis of the tolerance ellipse represents the degree of hydration, where we observe that an important part of patients is in a state of hyperhydration. The horizontal axis shows the cell mass, where we can see patients with low cell mass associated with malnutrition. Patients who died are grouped in the lower right quadrant, corresponding to hyperhydration and low BCM. PhA and SPhA values among non-survivors were less (p < 0.001) than among surviving patients (Table 1). COVID-19 non-survivors differed (p < 0.001) in soft tissue hydration status and were hyperhydration compared to survivors (85.2 vs 73.7%).
Fig. 1.
Bioelectrical values of COVID-19 disease: non-survivors (n = 16) and survivors patients (n = 111). R: resistance (Ohm); Xc: reactance (Ohm); H: height (m); (R/H) and (Xc/H): R/H and Xc/H standardized for sex and age using bioelectrical Italian standards. Red point: non-survivor patients; Black point: survivor patients. The bioelectrical impedance vector distribution analysis shows a situation of inflammation and cellular injury associated with COVID-19. The lower right quadrant encompasses patients with decreased cell mass and hyperhydration, most of the deceased patients.
Stratification of SPhA values reveals greater mortality in quartiles with low SPhA (quartile 1 vs 4; 32.4 vs 0%, p < 0.001) and increased hospital stay (Q1 vs Q4, 23 (11–35) vs. 13 (7–17) d, p = 0.018) (Table 2 ). The lowest SPhA quartile had significantly increased CRP and D-dimer, and decreased BCMI compared to the highest quartile (Q1 vs Q4).
Table 2.
Characteristics of COVID-19 patients according to quartiles of standardized phase angle (SPhA).
| Variables | (Q1) <-2 |
(Q2) −1.9–−0.8 |
(Q3) −0.7–0.2 |
(Q4) >0.3 |
p |
|---|---|---|---|---|---|
| (n = 34) |
(n = 31) |
(n = 34) |
(n = 28) |
||
| Median (IQR) | Median (IQR) | Median (IQR) | Median (IQR) | ||
| Mortality ratio (%) | 11/16 (32.4) | 2/16 (6.5) | 3/16 (8.8) | 0/10 (0) | 0.001 |
| ICU admission ratio (%) | 11/23 (47.8) | 8/23 (34.8) | 2/23 (8.7) | 2/23 (8.7) | 0.009 |
| Hospital stay (d) | 23 (11–35) | 19 (13–30) | 13 (9–23) | 13 (7–17) | 0.018 |
| Age (y) | 70 (61–85) | 73 (55–85) | 69 (58–81) | 63 (56–73) | 0.424 |
| Male sex n (%) | 11 (32.4) | 13 (41.9) | 14 (41.2) | 14 (50.0) | 0.572 |
| R/H (Ω/m) | 291 (239–339) | 303 (272–359) | 309 (275–363) | 349 (285–389) | 0.068 |
| Xc/H (Ω/m) | 13 (10–16) | 21 (17–25) | 28 (25–32) | 37 (31–43) | <0.001 |
| PhA (°) | 2.65 (2–3.3) | 3.8 (3.3–4.4) | 4.8 (4.4–5.7) | 6.1 (5.4–7) | <0.001 |
| Weight (kg) | 72.5 (63.8–75.8) | 78 (70–90) | 80 (69.8–90.5) | 71.5 (60.5–80.0) | 0.054 |
| BMI (kg/m2) | 24.5 (23.3–27.4) | 26.9 (23.4–31.7) | 28.5 (24.8–31.4) | 26.6 (23.5–29.6) | 0.029 |
| Hydration (%) | 87.3 (84.2–90) | 77.1 (73.7–82.2) | 73.6 (73.3–73.9) | 73.1 (72.6–73.3) | <0.001 |
| BCMI (kg) | 4.9 (3.7–6.9) | 7.3 (6.2–8.5) | 8.9 (7.6–10.9) | 9.9 (8.4–12.4) | <0.001 |
| Lymphocytes | 1045 (743–1685) | 1480 (950–1830) | 1465 (850–2050) | 1800 (1513–2318) | 0.003 |
| CRP (mg/L) | 32.6 (14.4–158.3) | 12.5 (4.2–38.6) | 14.5 (5.5–59.8) | 10.3 (4–46) | 0.017 |
| Albumin (g/dL) | 2.3 (2.2–2.7) | 2.7 (2.3–3.1) | 2.9 (2.5–3.1) | 3.1 (2.8–3.2) | <0.001 |
| Pre-albumin (mg/dL) | 12.7 (8.0–24.1) | 19.1 (15–41.7) | 18.2 (9.4–27.5) | 21.6 (18–24.3) | 0.058 |
| INR | 1.09 (1.02–1.16) | 1.08 (0.99–1.15) | 1.06 (1.00–1.13) | 1.04 (0.98–1.08) | 0.174 |
| D-dimer (ng/mL) | 1802 (1050–3331) | 2095 (860–4190) | 959 (480.8–3090) | 835 (483–1309) | 0.002 |
| Ferritin (ng/mL) | 482 (255–764) | 373 (240–729) | 686 (263–1121) | 447 (277–1109) | 0.508 |
| Fibrinogen (mg/dL) | 455 (307–530) | 352 (283–483) | 442 (342–515) | 425 (302–580) | 0.402 |
IQR: interquartile range; R/H: Resistance/height; Xc/H: reactance/height; PhA: phase angle; BMI body mass index; BCMI: body cell mass index; CRP: C-reactive protein; INR: International normalized ratio.
The 25th, 50th and 75th percentiles of SPhA were used to stratify the patients with acute COVID-19 into 4 groups (Q1, Q2, Q3, Q4) and made the Kaplan–Meier plot (Fig. 2 ). Kaplan–Meier product-limit estimator showed that lower SPhA (Q1) was significantly linked with higher mortality rates (log rank test, p < 0.001) (Fig. 2).
Fig. 2.
Kaplan–Meier analysis for the cumulative percentage of surviving patients at 90-days according to SPhA quartiles. The 25th, 50th and 75th percentiles of SPhA were used as the cut-off point to divide the patients with acute COVID-19 disease into 4 groups (Q1, Q2, Q3, Q4). A significantly lower survival rate was observed in patients with lower SPhA values (log rank test, p < 0.01). Mortality was mainly concentrated in Q1.
At the end of follow-up period (90 d), 16 deaths occurred among the study patients. Median of survival time in Q1 was 71.2 d (58.9–83.5d), 93.1 d (86.6–99.7d) for Q2 and 91.2 d (84.9–97.5d) for Q3 quartile. In Q4 quartile, there were not recorded deaths.
3.2. Optimal PhA cut-off value and 90-d mortality prediction
Using ROC curves, we determined the PhA value cut-off point for predicting mortality. AUC was 0⋅839 and the 3.95° PhA value was the most sensitive (93.8%) and specific (66.7%) factor for mortality risk prognosis in acute SARS-CoV-2 infection. These results are adjusted by sex and age. Thus, SPhA may be useful as global cut-off point for all patients screened at admission. For SPhA, AUC was 0.779 and −1.85 SPhA value was the most sensitive (75%) and specific (75.7%) factor for mortality risk prognostic in acute SARS-CoV-2 infection. We compared these data with the cut-off points of mortality prediction using ROC curves for the analytical parameters established as prognostic factors (Table 3 , Fig. 3 ). Prealbumin less than 13.55 mg/dL, lymphocytes less than 1075 (106/L) and CRP 62.75 mg/L were the cut-off points for predicting mortality. The PhA showed the highest sensitivity with respect to the rest of the parameters in predicting mortality at 90 days.
Table 3.
Analysis of the prognostic factors of mortality in COVID-19.
| Variables | AUC | Cut-off point | Sensitivity | Specificity | p |
|---|---|---|---|---|---|
| PhA | 0,839 | 3.95 | 93.8% | 66.7% | 0,001 |
| SPhA | 0,779 | −1.85 | 75% | 75.7% | 0,005 |
| Prealbumin | 0,769 | 13.55 | 72.7% | 77% | 0,005 |
| Lymphocytes | 0,801 | 1075 | 75% | 73.9% | 0,000 |
| CRP | 0,783 | 62.75 | 68.8% | 82.7% | 0,003 |
PhA: phase angle; SPhA: standardized phase angle; AUC: area under the curve; CRP: C-reactive protein.
Evaluation of prognosis factors of mortality in COVID-19 based on the area under the curve (AUC) of the receiver operating characteristic (ROC) curve, and sensitivity, specifity values to determine the optimal cut-off values.
Fig. 3.
Comparative analysis of ROC Curve of BIA measurements (PhA and SPhA) with established prognosis factor in COVID-19 patients. Comparative analysis of the receiver operating characteristic curves of PhA, SPhA and established indicators (CRP, prealbumin and lymphocytes) for prediction of mortality in patients with COVID-19 (n = 127).
PhA and SPhA showed a negative correlation with hospital stay in survivors of COVID-19, PhA (r = −0.313, p = 0.001) and SPhA (r = −0.311, p = 0.001). Furthermore, the PhA correlates with hydration percentage (r = −0.838, p < 0.001).
We used an 8-component model multivariate analysis (by Cox regression) to evaluate the utility of the bioelectrical parameters as prognostic indicators of mortality in COVID-19. We found that a decreased PhA value was significantly associated with a higher mortality hazards ratio (HR 2.480, 95% CI 1.604–3.835, p < 0.001). This trend was also maintained in the adjusted models by the confounding variables (Table 4 ). Likewise, the hydration status was associated with an increase of 11.5% in the hazard ratio of the crude model [HR: 1.115 (1.039–1.197), p = 0.003], with this relationship maintained in the adjusted models (Table 4).
Table 4.
Multivariate Cox regression for PhA and hydration status as predictors of mortality in COVID-19 patients.
| Phase angle (°) |
Hydration (%) |
|||
|---|---|---|---|---|
| HR (CI) | p | HR (CI) | p | |
| Crude model | 2.480 (1.604–3.835) | <0.001 | 1.115 (1.039–1.197) | 0.003 |
| Model 1 | 2.281 (1.365–3.813) | 0.002 | 1.097 (1.015–1.180) | 0.014 |
| Model 2 | 2.507 (1.455–4.318) | 0.001 | 1.112 (1.024–1.209) | 0.012 |
| Model 3 | 3.912 (1.322–11.572) | 0.014 | NA | |
The Hazard ratio (HR) for death was expressed per 1° decrease in PhA and 1% increase in hydration percentage, for a univariate model and sequential adjustment models. Dependent variable: survivors (0) vs. non-survivors (1). Cox regression was expressed by HR and 95% confidence interval (CI). NA (not applicable).
Model 1: adjusted for sex, age and BMI.
Model 2: additionally adjusted for diagnosis previous of type 2 diabetes, high blood pressure, dyslipidaemia and heart disease.
Model 3: additionally adjusted for hydration status.
4. Discussion
The aim of this study was to assess whether raw BIA measurements, PhA and SPhA, and estimated hydration could predict mortality in a cohort of patients with COVID-19. After adjusting for sex, age, BMI and comorbidities, we determined that PhA, SPhA and hydration were significant predictors of 90-d mortality in these patients.
The mean PhA found in our sample was 4.4° (3.2–5.4°), showing a situation of inflammation and cell injury associated with SARS-CoV-2 infection. In literature, there are few data on PhA in acute respiratory pathology and infectious processes [35], usually described in chronic pathology (cancer, cirrhosis, chronic obstructive pulmonary disease (COPD)) [30,31]. Shah et al. has described that the PhA in human immunodeficiency virus (HIV) subjects with acute coinfection by tuberculosis (TB) was 5.42° + 1.01 in men and 5.35° + 1.27 in women (539 HIV patients with and without TB coinfection) [38]. PhA 4.4° in men and PhA 4.3° in women in our series reflected a decline in the clinical situation compared with acute TB infection.
Our observations are consistent with the initial finding of Moonen et al. [17], who reported a direct relationship between mortality and PhA in critical patients with COVID-19 but did not determine the cut-off point of PhA as a predictor of mortality. Our results extend this observation and overcome the limitations of limited sample size and short follow-up (28 d) by developing survival curves and PhA cut-off points to predict mortality [17]. Importantly, we demonstrate that a cut-off PhA value (by ROC curve of survival) less than 3.95° is more sensitive prognostic factor in predicting mortality in COVID-19 patients than standard biochemical measurements of inflammation (Table 3, Fig. 3).
The most used measures of association in literature to determine the PhA-mortality correlation were relative risk (RR) and hazard ratio (HR). Our data analysis showed a higher risk for mortality in the crude Cox regression model. Beyond these data, the survival analysis revealed 2.48 times higher hazards ratio of mortality for a decrease in 1° in PhA value. Our results confirm the initial report of Moonen et al. [17] that a low PhA was associated with COVID-19 severity in the composite score (OR 0.299, p = 0.046) [17]. It is important to highlight that our results, on the general COVID-19 population and not limited only to critical patients, may have greater applicability in clinical practice.
PhA results in our serie are similar to other findings in different pathologies. In patients with acute decompensated heart failure, a PhA <4.8° had a RR 2.67 (95% CI 1.21–5.89, p = 0.015) of increased mortality [29]. PhA discriminated (p < 0.01) surviving (4.1° ± 1.2) and non-surviving (2.9° ± 0.8) individuals admitted to ICU [34]. We also found similar differences between survivor [4.5° (3.5–5.5°)] and non-survivor patients [2.8° (2.08–3.68°)], (p < 0.001). For patients with advanced colorectal cancer, the RR value mortality was 10.75 (95% CI 1.92–60.24%, p = 0.007) in individuals with PhA ≤ 5.57° [31]. In some kidney disease studies, 1° increment in PhA predicted a significantly increased probability of survival. One of which showed HR 0.390 (05% CI 0.267–0.570%) [32], which is similar to our COVID-19 patients. However, we believe the relationship between PhA and mortality risk must be analysed from the “PhA decrease perspective”, as this has a direct influence among individuals admitted to on mortality.
The incorporation of the SPhA is an added value to our data, because it allows comparison for any age range or sex. Our patients admitted by COVID-19 can be assumed to have a decrease in 0.8 point in SPhA compared to the general population (reference value 0). The SPhA analysis translates direct information on the population reference pattern [19]. In spite of not being easy to translate these results, the decrease in SPhA implies a poor prognosis of the COVID-19, linked to a damaged cellular health, as it is demonstrated in other diseases (oncology patients, general ICU patients) [16,28]. We used the quartiles of SPhA in this cohort as the cut-off point to separate patients into four groups, which showed that lower SPhA quartiles have a higher mortality risk. Patients in Q1 of SPhA showed a relative risk of mortality of 32.4% for a mean PhA of 2.65°, while in Q2 the mortality was 6.5% for a mean PhA of 3.8°, Q3 where the mortality was of 8.8% for a mean PhA of 4.8° and in Q4 mortality was 0% for a mean PhA of 6.1°. These findings indicate that lower values of PhA and SPhA in COVID-19 admitted patients indicate a poorer outcome.
Most reports established PhA cut-off points to relate them to mortality through the analysis of survival using a ROC curve. The predictive value of the SPhA depends on the choice of cut-off points, which should be developed for the specific clinical population [34]. The present study found that patients with a SPhA < −1.85 had a lower probability to survive following the 90-d follow-up, with a high sensitivity (75%) and specificity (75%) predictor factor. Consequently, our results confirm a strong association between SPhA and survival.
Furthermore, we established an association between SPhA and mortality using Kaplan–Meier analysis. The worst survival in COVID-19 patients was found in the lowest quartile of SPhA; and its prognostic significance was proven in presence of other clinical indicators of a poor prognostic such as sex and age. In the graph of survival curve (Kaplan–Meier), a SPhA < −2 (Q1 SPhA) grouped the majority of decease patients, as significant predictor of mortality. It has been stablished different cut-points for mortality distributions in several pathology associated to PhA and SPhA, Paiva et al. descripted SPhA cut-point for mortality was −1.65 in cancer patients [25].
Hyperhydration is emerging as a factor associated with mortality in acute COVID-19 [33]. In the Cox regression multivariate model, the increase in 1% of hydration percentage raised the hazards ratio of mortality by 1.12, which demonstrates that hyperhydration is harmful for COVID-19 patients.
In the mortality analysis, it is important to highlight the profile of deceased patients. It shows a significant decrease in PhA 2.8° and SPhA −2.95, indicating a deteriorating nutritional status, a higher state of hydration (ECW and hydration percentage) and pro-inflammatory and immune response (CRP, lymphocytes) [3]. Bioelectric results obtained in patients who had poor prognosis and eventually died gives us information about the presence of cell damage (decrease SPhA less than 0) and hyperhydration (hydration percentage more than 74.3%, +1 standard deviation of euhydratation) [27], which may be useful as a prognostic factor [20].
The average hospital length of stay in this study is 2 wk, increasing to more than a month and a half in patients requiring ICU care. These descriptive results are similar to those published on large series of COVID-19 [2,3].
5. Strengths and limitations
To our knowledge, this study is the first to investigate the role of PhA as an indicator of prognosis in COVID-19 and to compare it with standard clinical biomarkers of inflammation. Although the total number of deaths in our study was relatively low, it was consistent with the results of other COVID-19 cohort studies [4].
Some limitations of our study are caused by the limited number of patients, the location being only a single hospital and different stage of the disease at the time of the measure. However, our sample data present similarities to those described in the large series [[2], [3], [4]].
The addition of serial BIA measurements during the hospitalization stay concurrent with medical treatment could be a useful to determine notable effects of the different treatments among other variables, on clinical course and mortality. Also, PhA could be monitored for 3 or 6 months after discharge. These data open the possibility of establishing multiple modes of therapeutic intervention aimed at improving the management and prognosis of patients with COVID-19. The fact that raw bioelectric parameters do not depend on the interpretation of equations facilitates their direct application in the usual clinical practice for these COVID-19 patients, and therefore offers the possibility of assessing their prognostic situation through the general, nutritional, hydration and cellular health aspects.
6. Conclusion
Overall, PhA as a raw BIA measurement may play a role in assessing mortality risk in COVID-19, independently of age, sex, BMI, and comorbidities. PhA decrease was associated with an important increment of mortality risk. The change of hydration status is also a significant prognostic factor. The SPhA quartiles distribution is clearly related with the median of survival time in different SPhA.
PhA offers greater sensitivity as a predictive prognostic test at admission, compared to the established analytical parameters of poor prognosis (CRP, lymphocytes, prealbumin).
Eventual non-survivors are more likely to have low PhA, higher hydration percentage, severe clinical situation and analytical disturbance.
Indeed, very low values of PhA (<3.95°) and SPhA (<-1.85) likely indicate a very high risk of poor prognosis status and mortality.
Given its prognostic utility, these findings suggest that the evaluation of body composition through raw BIA variables could be included in the clinical routine of COVID-19 patients, especially in those in hospital admission, in order to identify potential patients at risk. Further studies should be conducted to confirm these findings.
Funding statement
I.C.-P. was the recipient of a postdoctoral grant (Río Hortega CM 17/00169) and is now the recipient of a postdoctoral grant (Juan Rodes JR 19/00054) from the Instituto de Salud Carlos III and co-funded by Fondo Europeo de Desarrollo Regional-FEDER.
Author contributions
J.M.G.-A., I.C.-P and I.M.V.-A conceived the study and drafted the manuscript. D.B.-G, A.T. and H.L. contributed to drafting sections of the manuscript. I.C.-P did data analyses. F.J.T and H.L. participated in the study design and critically revised the article. All authors contributed to the interpretation of data, read and approved the final manuscript.
Conflict of interest
The authors declare that they have no conflict of interest.
Acknowledgments
We thank Marina Garcia-Almeida for English language editing.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.clnu.2021.02.017.
Appendix A. Supplementary data
The following is the Supplementary data to this article:
References
- 1.Zhou F., Yu T., Du R., Fan G., Liu Y., Liu Z., et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395:1054–1062. doi: 10.1016/S0140-6736(20)30566-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Wu Z., McGoogan J.M. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese center for disease control and prevention. JAMA. 2020;323:1239–1242. doi: 10.1001/jama.2020.2648. [DOI] [PubMed] [Google Scholar]
- 3.Petrilli C.M., Jones S.A., Yang J., Rajagopalan H., O'Donnell L.F., Chernyak Y., et al. Factors associated with hospitalization and critical illness among 4,103 patients with COVID-19 disease in New York City. MedRxiv. 2020:2020. doi: 10.1101/2020.04.08.20057794. 04.08.20057794. [DOI] [Google Scholar]
- 4.Khalil K., Agbontaen K., McNally D., Love A., Mandalia S., Banya W., et al. Clinical characteristics and 28-day mortality of medical patients admitted with COVID-19 to a central london teaching hospital. J Infect. 2020 doi: 10.1016/j.jinf.2020.06.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Wang D., Hu B., Hu C., Zhu F., Liu X., Zhang J., et al. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus–infected pneumonia in Wuhan, China. JAMA. 2020;323:1061–1069. doi: 10.1001/jama.2020.1585. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Yang X., Yu Y., Xu J., Shu H., Xia J., Liu H., et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med. 2020 doi: 10.1016/S2213-2600(20)30079-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Barazzoni R., Bischoff S.C., Breda J., Wickramasinghe K., Krznaric Z., Nitzan D., et al. ESPEN expert statements and practical guidance for nutritional management of individuals with SARS-CoV-2 infection. Clin Nutr. 2020;39:1631–1638. doi: 10.1016/j.clnu.2020.03.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Martindale R., Patel J.J. 2020. Nutrition therapy in the patient with COVID-19 disease requiring ICU care; p. 8. [Google Scholar]
- 9.Singer P., Blaser A.R., Berger M.M., Alhazzani W., Calder P.C., Casaer M.P., et al. ESPEN guideline on clinical nutrition in the intensive care unit. Clin Nutr Edinb Scotl. 2019;38:48–79. doi: 10.1016/j.clnu.2018.08.037. [DOI] [PubMed] [Google Scholar]
- 10.van Gassel R.J.J., Baggerman M.R., van de Poll M.C.G. Metabolic aspects of muscle wasting during critical illness. Curr Opin Clin Nutr Metab Care. 2020;23:96–101. doi: 10.1097/MCO.0000000000000628. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Lukaski H.C., Kyle U.G., Kondrup J. Assessment of adult malnutrition and prognosis with bioelectrical impedance analysis: phase angle and impedance ratio. Curr Opin Clin Nutr Metab Care. 2017;20:330–339. doi: 10.1097/MCO.0000000000000387. [DOI] [PubMed] [Google Scholar]
- 12.Lukaski H.C., Vega Diaz N., Talluri A., Nescolarde L. Classification of hydration in clinical conditions: indirect and direct approaches using bioimpedance. Nutrients. 2019;11 doi: 10.3390/nu11040809. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Merad M., Martin J.C. Pathological inflammation in patients with COVID-19: a key role for monocytes and macrophages. Nat Rev Immunol. 2020;20:355–362. doi: 10.1038/s41577-020-0331-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Ruan Q., Yang K., Wang W., Jiang L., Song J. Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China. Intensive Care Med. 2020;46:846–848. doi: 10.1007/s00134-020-05991-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Beberashvili I., Azar A., Sinuani I., Shapiro G., Feldman L., Stav K., et al. Bioimpedance phase angle predicts muscle function, quality of life and clinical outcome in maintenance hemodialysis patients. Eur J Clin Nutr. 2014;68:683–689. doi: 10.1038/ejcn.2014.67. [DOI] [PubMed] [Google Scholar]
- 16.Garlini L.M., Alves F.D., Ceretta L.B., Perry I.S., Souza G.C., Clausell N.O. Phase angle and mortality: a systematic review. Eur J Clin Nutr. 2019;73:495–508. doi: 10.1038/s41430-018-0159-1. [DOI] [PubMed] [Google Scholar]
- 17.Moonen H.P.F.X., van Zanten F.J.L., Driessen L., de Smet V., Slingerland-Boot R., Mensink M., et al. Association of bioelectric impedance analysis body composition and disease severity in COVID-19 hospital ward and ICU patients: the BIAC-19 study. Clin Nutr Edinb Scotl. 2020 doi: 10.1016/j.clnu.2020.10.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.COVID-19: clinical/therapeutic staging proposal and treatment. REBEL EM - Emerg Med Blog; 2020. https://rebelem.com/covid-19-clinical-therapeutic-staging-proposal-and-treatment/ [Google Scholar]
- 19.Piccoli A., Nigrelli S., Caberlotto A., Bottazzo S., Rossi B., Pillon L., et al. Bivariate normal values of the bioelectrical impedance vector in adult and elderly populations. Am J Clin Nutr. 1995;61:269–270. doi: 10.1093/ajcn/61.2.269. [DOI] [PubMed] [Google Scholar]
- 20.Piccoli A., Rossi B., Pillon L., Bucciante G. A new method for monitoring body fluid variation by bioimpedance analysis: the RXc graph. Kidney Int. 1994;46:534–539. doi: 10.1038/ki.1994.305. [DOI] [PubMed] [Google Scholar]
- 21.Cardinal T.R., Wazlawik E., Bastos J.L., Nakazora L.M., Scheunemann L. Standardized phase angle indicates nutritional status in hospitalized preoperative patients. Nutr Res N Y N. 2010;30:594–600. doi: 10.1016/j.nutres.2010.08.009. [DOI] [PubMed] [Google Scholar]
- 23.Chumlea W.C., Schubert C.M., Sun S.S., Demerath E., Towne B., Siervogel R.M. A review of body water status and the effects of age and body fatness in children and adults. J Nutr Health Aging. 2007;11:111–118. [PubMed] [Google Scholar]
- 24.De Palo T., Messina G., Edefonti A., Perfumo F., Pisanello L., Peruzzi L., et al. Normal values of the bioelectrical impedance vector in childhood and puberty. Nutr Burbank Los Angel Cty Calif. 2000;16:417–424. doi: 10.1016/s0899-9007(00)00269-0. [DOI] [PubMed] [Google Scholar]
- 25.Paiva S.I., Borges L.R., Halpern-Silveira D., Assunção M.C.F., Barros A.J.D., Gonzalez M.C. Standardized phase angle from bioelectrical impedance analysis as prognostic factor for survival in patients with cancer. Support Care Cancer Off J Multinatl Assoc Support Care Cancer. 2010;19:187–192. doi: 10.1007/s00520-009-0798-9. [DOI] [PubMed] [Google Scholar]
- 26.Schutz Y., Kyle U.U.G., Pichard C. Fat-free mass index and fat mass index percentiles in Caucasians aged 18-98 y. Int J Obes Relat Metab Disord J Int Assoc Study Obes. 2002;26:953–960. doi: 10.1038/sj.ijo.0802037. [DOI] [PubMed] [Google Scholar]
- 27.Moore F.D., Boyden C.M. Body cell mass and limits of hydration of the fat-free body: their relation to estimated skeletal weight. Ann N Y Acad Sci. 1963;110:62–71. doi: 10.1111/j.1749-6632.1963.tb17072.x. [DOI] [PubMed] [Google Scholar]
- 28.Stapel S.N., Looijaard W.G.P.M., Dekker I.M., Girbes A.R.J., Weijs P.J.M., Oudemans-van Straaten H.M. Bioelectrical impedance analysis-derived phase angle at admission as a predictor of 90-day mortality in intensive care patients. Eur J Clin Nutr. 2018;72:1019–1025. doi: 10.1038/s41430-018-0167-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Marini E., Buffa R., Contreras M., Magris M., Hidalgo G., Sanchez W., et al. Effect of influenza-induced fever on human bioimpedance values. PloS One. 2015;10 doi: 10.1371/journal.pone.0125301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.de Blasio F., Scalfi L., Di Gregorio A., Alicante P., Bianco A., Tantucci C., et al. Raw bioelectrical impedance analysis variables are independent predictors of early all-cause mortality in patients with COPD. Chest. 2019;155:1148–1157. doi: 10.1016/j.chest.2019.01.001. [DOI] [PubMed] [Google Scholar]
- 31.Norman K., Wirth R., Neubauer M., Eckardt R., Stobäus N. The bioimpedance phase angle predicts low muscle strength, impaired quality of life, and increased mortality in old patients with cancer. J Am Med Dir Assoc. 2015;16(173):e17–e22. doi: 10.1016/j.jamda.2014.10.024. [DOI] [PubMed] [Google Scholar]
- 32.Shah S., Whalen C., Kotler D.P., Mayanja H., Namale A., Melikian G., et al. Severity of human immunodeficiency virus infection is associated with decreased phase angle, fat mass and body cell mass in adults with pulmonary tuberculosis infection in Uganda. J Nutr. 2001;131:2843–2847. doi: 10.1093/jn/131.11.2843. [DOI] [PubMed] [Google Scholar]
- 33.Koh K.-H., Wong H.-S., Go K.-W., Morad Z. Normalized bioimpedance indices are better predictors of outcome in peritoneal dialysis patients. Perit Dial Int J Int Soc Perit Dial. 2011;31:574–582. doi: 10.3747/pdi.2009.00140. [DOI] [PubMed] [Google Scholar]
- 34.Lee Y., Kwon O., Shin C.S., Lee S.M. Use of bioelectrical impedance analysis for the assessment of nutritional status in critically ill patients. Clin Nutr Res. 2015;4:32–40. doi: 10.7762/cnr.2015.4.1.32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Gupta D., Lammersfeld C.A., Vashi P.G., King J., Dahlk S.L., Grutsch J.F., et al. Bioelectrical impedance phase angle in clinical practice: implications for prognosis in stage IIIB and IV non-small cell lung cancer. BMC Canc. 2009;9:37. doi: 10.1186/1471-2407-9-37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Koratala A., Ronco C., Kazory A. Need for objective assessment of volume status in critically ill patients with COVID-19: the tri-POCUS approach. Cardiorenal Med. 2020:1–8. doi: 10.1159/000508544. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.