Abstract
Background
Coronavirus disease-2019 (COVID-19) is associated with a high risk of acute kidney injury (AKI), often requiring renal replacement therapy (RRT). Serum Cystatin C (sCysC) and serum Neutrophil Gelatinase-Associated Lipocalin (sNGAL) are emerging biomarkers for kidney injury, and were suggested to be superior to serum creatinine (sCr) in several clinical settings. Moreover, elevated sCysC is associated with disease severity and mortality in COVID-19. We aimed to assess the utility of sCysC and sNGAL for predicting COVID-19-associated AKI, need for RRT, and need for intensive care unit (ICU) admission, when measured at presentation to the emergency department (ED).
Methods
Patients presenting to the ED with laboratory-confirmed COVID-19 were included. The primary outcome was development of COVID-19-associated AKI, while the secondary outcomes were need for RRT and ICU admission.
Results
Among 52 COVID-19 patients, 22 (42.3%) developed AKI with 8/22 (36.4%) requiring RRT. Both sCr and sCysC demonstrated excellent performance for predicting AKI (AUC, 0.86 and 0.87, respectively) and need for RRT (AUC, 0.94 and 0.95, respectively). sNGAL displayed acceptable performance for predicting AKI (AUC, 0.81) and need for RRT (AUC, 0.87).
Conclusions
SCr and sCysC measured at ED presentation are both highly accurate predictors of AKI and need for RRT, whereas sNGAL demonstrated adequate diagnostic performance. While sCyC was previously shown to be superior to sCr as a diagnostic biomarker of kidney injury in certain etiologies, our findings demonstrate that sCr is comparable to sCyC in the context of predicting COVID-19-associated AKI. Given the high sensitivity of these biomarkers for predicting the need for RRT, and as sCysC is associated with mortality in COVID-19 patients, we recommend their measurement for enabling risk stratification and early intervention.
Keywords: SARS-CoV-2, COVID-19, Biomarkers, Acute kidney injury, Diagnosis, Laboratory medicine
1. Introduction
A substantial body of evidence now attests that Coronavirus disease 2019 (COVID-19) is associated with a high risk of developing acute kidney injury (AKI) [1]. A recent study involving 3993 hospitalized patients with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection in New York City, found a COVID-19-associated AKI incidence of 46%, which also conferred an enhanced risk of mortality [2]. Notably, 35% of survivors had not recovered to their baseline renal function at discharge [2]. In another study across intensive care units (ICUs) at 67 hospitals in the United States, one in five ICU patients developed AKI requiring renal replacement therapy (RRT) [3]. Among these patients, 63% died, whilst one in three survivors remained RRT-dependent upon discharge, and one in six survivors remained RRT dependent 60 days post-ICU admission [3]. In a recent meta-analysis of 15 studies, we reported that AKI development was associated with ∼19 fold (95% confidence interval (95%CI), 9–28) increased odds of severe COVID-19 and nearly ∼24 fold (95%CI, 19–30) increased odds of in-hospital death [4].
Given the high morbidity and mortality associated with development of AKI in COVID-19 patients, especially among those needing RRT, special attention is needed to identify subjects at higher risk of progression to AKI early in the course of disease, thus enabling targeted monitoring and earlier, more aggressive intervention. While serum creatinine (SCr) is the standard by which renal function is assessed and AKI is diagnosed, a single value of SCr is unable to differentiate pre-renal azotemia from intrinsic renal injury or chronic kidney disease, all of which have critically distinct therapeutic managements [5]. Moreover, SCr has a long-half life, and its variations cannot reliably mirror sudden renal function changes until a new steady state is reached [5]. SCr is also impacted by a number of patient variables, such as age, sex, muscle mass and metabolism, fluid volume status, and diet [5]. Two novel molecules have grown in popularity and utility for the purposes of early diagnosis and monitoring of AKI [5], [6]. Cystatin C is a biomarker of renal function typically used to estimate glomerular filtration rate (GFR), as it is less reliant than SCr on muscle mass and dietary intake for a steady state [6]. Neutrophil Gelatinase-Associated Lipocalin (NGAL), is a biomarker of (especially acute) renal damage, released from epithelial cells of injured kidneys and reaching plasma via tubular backleak or being excreted into urine in the distal nephron [5], [6]. Recently, serum Cystatin C levels have been shown to correlate with disease severity and mortality in COVID-19 [1]. Therefore, this investigation was aimed at exploring the role of these serum biomarkers measured at emergency department (ED) presentation for predicting emergence of COVID-19-associated AKI, need for RRT and ICU admission. We also explored the relationship of Cystatin C and NGAL with biomarkers of hemodynamics, thromboinflammation, and renal function, to help elucidate potential mechanisms of COVID-19-associated AKI.
2. Methods
2.1. Study design
This prospective, observational investigation was conducted at the Cincinnati ED COVID-19 Laboratory Cohort. Patients’ enrollment took place between April-May 2020 at the ED of University of Cincinnati Medical Center. Adults (≥18 years) presenting with COVID-19 symptoms and undergoing clinically routine blood draw were considered eligible for enrollment. Only patients with a positive result on standard of care nucleic acid amplification test (NAAT) for SARS-CoV-2 on nasopharyngeal swabs were finally enrolled, whilst patients <18 years of age at time of ED visit, negative result of NAAT, or known prisoners were excluded. This investigation was reviewed and approved by the Institutional Review Board (IRB) of the University of Cincinnati. The study was deemed no greater than minimal risk and thus performed under a waiver of informed consent. This study was performed in compliance with the Declaration of Helsinki and local and national regulations. Study reporting was in accordance to the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) guidelines (Supplement 1).
2.2. Sample collection and processing
All blood samples were collected as part of routine blood draw during ED visit. Following collection, samples were centrifuged at 2000 g for 15 min and subsequently frozen at −80 °C until analysis. All laboratory tests were performed at the Clinical Nephrology Lab of the Cincinnati Children’s Hospital Medical Center (CCHMC) or the central laboratory of the UCMC. Serum Cystatin C (sCysC) was measured using a Siemens N Latex Cystatin C assay (REF: OQNM19) on a BN II nephelometric analyzer (Siemens Medical Solutions USA, Inc., Malvern, PA, USA). Serum NGAL (sNGAL) was assayed with The NGAL Test (BioPorto Diagnostics A/S, Hellerup, Denmark), a particle-enhanced turbidimetric immunoassay, also using a BN II System.
2.3. Measurement of NGAL and Cystatin C
sCysC and sNGAL were measured as previously described [7]. Briefly, for sCysC, polystyrene particles coated with specific anti-human CysC antibodies are aggregated after being mixed with specimens containing CysC. The generation of these aggregates scatter a light beam passed through the sample, whose intensity is directly proportional to CysC concentration in the test sample. The total imprecision of the assay ranges from 1.5 to 3.2%, the measuring range is between 0.08 and 42.91 mg/L.
For sNGAL, serum specimens are mixed with the reaction buffer containing a suspension of polystyrene microparticles coated with mouse monoclonal antibodies to NGAL. The presence of NGAL in the test sample triggers immunoparticles aggregation, which is then quantified by the amount of light scattered, whose intensity is directly proportional to NGAL concentration in the test sample. The total imprecision of the assay ranges from 2.5 to 3.6%, the measuring range is between 25 and 5000 ng/mL. sNGAL measurement was preferred over urine, to exclude potential interference from leukocyturia [8]. The same instrument, lot of reagents and calibration curve were used for all measurements of sCysC and sNGAL.
2.4. Measurement of other biomarkers
SCr was assessed using a kinetic alkaline picrate (modified Jaffe) method using either a Beckman Coulter AU480 Chemistry Analyzer (Brea, California, USA) or a Beckman Coulter AU5822 Chemistry Analyzer (Brea, California, USA). Plasma concentrations of interleukin (IL)-6, IL-8, and IL-10, and tumor necrosis factor-α (TNF-α) were measured using Meso Scale Discovery (MSD) U-Plex assay (Rockville, Maryland, USA). Plasma concentrations of fibrinogen, ferritin, myoglobin, haptoglobin, and C-reactive protein were determined with a BN II System. Lactate dehydrogenase (LDH) was assayed on a Dimension RxL Max Integrated Chemistry System (Siemens Medical Solutions USA, Inc, Malvern, PA, USA), while procalcitonin and direct renin concentrations were measured with chemiluminescent immunoassays (CLIA) on the Diasorin Liaison XL (DiaSorin S.p.A. Saluggia, Italy). All laboratory tests were performed according to manufacturers’ instructions and recommendations.
2.5. Data collection
Data collection was executed by two practicing ED physicians, with select records cross-checked for accuracy. Patient demographics, past medical history, and clinical variables (vital signs, laboratory data, clinical course, outcomes) were retrieved from electronic medical records (EMR) and recorded into a REDCap (Research Electronic Data Capture) database. Clinical data on hospitalized patients was recorded through discharge, whilst data on clinical course of patients discharged at index ED visit were monitored for the ensuing 30 days.
2.6. Outcomes
The primary outcome of interest was development of acute kidney injury (AKI), defined by Kidney Disease Improving Global Outcomes (KDIGO) according to sCr criteria [9]). Baseline sCr values were retrieved from the medical records by the research physicians using their best medical judgment. If available, pre-COVID-19 sCr values were retrieved from medical records for assessment of baseline sCr. When a pre-COVID-19 sCr was not available in the medical record, the lowest sCr was employed as a baseline sCr for AKI assessment, either during hospitalization for patients admitted from the ED or within 30 days for patients discharged from the ED. Secondary outcomes included severe AKI [defined as KDIGO Stage 2 + 3 (Supplemental Table S2)], need for dialysis, and need for ICU admission. Elevations in sCysC (age <45 years, > 0.95 mg/L; age >45 years, >1.20 mg/L) [10] and sNGAL (>181.0 ng/L) [11] were in accordance with previously established reference ranges and cut-off values.
2.7. Statistical analysis
Categorical data were tabulated as absolute number (n) and relative frequency (%), whilst continuous variables were reported as median and interquartile range (IQR). Based on expected accounts, categorical variables were compared using the chi-squared test (χ2) or Fisher’s exact test, as appropriate. Continuous variables were compared using the Mann-Whitney U test. Spearman’s correlation coefficient was used to estimate correlation between continuous variables. ROC curves were used to evaluate the predictive value of specific factors. AUCs and respective confidence intervals were estimated, and optimal cutoffs were chosen using the Youden index method [12]. Considering the inevitability of multicollinearity between laboratory values of interest, a partial least squares discriminant analysis was performed to determine factors (demographic, comorbidity, and laboratory) predictive of AKI in COVID-19 patients. As this investigation represents a pilot, hypothesis-generating study, we did not adjust for multiple comparisons. Statistical analysis was performed with R (version 4.0.2, R Foundation for Statistical Computing, Vienna, Austria), with p < 0.05 considered statistically significant.
3. Results
3.1. Patient demographics and clinical course
A total of 52 patients with positive RT-PCR result for SARS-CoV-2 were enrolled in this study. The demographics, medical history, and baseline laboratory values of the cohort are presented in Table 1 . No patient was vaccinated at the time of enrollment or during the course of the study. With respect to pharmacologic therapy, 13.5% received hydroxychloroquine, 9.6% received remdesivir, 5.8% received azithromycin, 3.8% received tocilizumab, 3.8% received corticosteroids, and 1.9% received convalescent plasma.
Table 1.
Variable | All Patients (n = 52) | Acute Kidney Injury |
||
---|---|---|---|---|
No (n = 30) | Yes (n = 22) | p-value | ||
Age (years): median (IQR) | 51 (39–66) | 45 (37–51.8) | 66 (59.3–71) | <0.001 |
Sex (male): n (%) | 31 (59.6%) | 17 (56.7%) | 14 (63.6%) | 0.826 |
Race: n (%) | ||||
Black | 21 (40.4%) | 10 (33.3%) | 11 (50%) | 0.007 |
Hispanic | 19 (36.5%) | 16 (53.5%) | 3 (13.6%) | |
White | 9 (17.3%) | 2 (6.7%) | 7 (31.8%) | |
Other | 3 (5.8%) | 2 (6.7%) | 1 (4.5%) | |
Comorbidities: n (%) | ||||
Coronary Artery Disease | 8 (15.4%) | 2 (6.7%) | 6 (27.3%) | 0.058 |
Heart Failure | 9 (17.3%) | 1 (3.3%) | 8 (36.4%) | 0.003 |
Hypertension | 26 (50%) | 11 (36.7%) | 15 (68.2%) | 0.049 |
Hyperlipidemia | 15 (28.8%) | 7 (23.3%) | 8 (36.4%) | 0.478 |
Diabetes | 21 (40.4%) | 9 (30%) | 12 (54.5%) | 0.161 |
Chronic Obstructive Pulmonary Disease | 8 (15.4%) | 2 (6.7%) | 6 (27.3%) | 0.058 |
Asthma | 8 (15.4%) | 5 (16.7%) | 3 (13.6%) | 1 |
Chronic Kidney Disease | 6 (11.5%) | 0 | 6 (27.3%) | 0.004 |
Chronic Liver Disease | 7 (13.5%) | 2 (6.7%) | 5 (22.7%) | 0.119 |
Cerebrovascular Disease | 1 (1.9%) | 0 | 7 (31.8%) | 0.001 |
Cancer | 4 (7.7%) | 0 | 4 (18.2%) | 0.027 |
Obesity | 18 (34.6%) | 12 (40%) | 6 (27.3%) | 0.202 |
Acquired Immunodeficiency (HIV, Transplant) | 3 (5.8%) | 1 (3.3%) | 2 (9.1%) | 0.749 |
Autoimmune Disease | 2 (3.8%) | 1 (3.3%) | 1 (4.5%) | 1 |
Current Smoker | 12 (23.1%) | 4 (13.3%) | 7 (31.8%) | 0.494 |
Former Smoker | 11 (21.2%) | 5 (16.7%) | 6 (27.3%) | 0.169 |
Labs at ED visit: median (IQR) | ||||
White Blood Cell Count (x103/mm3) | 6.8 (5.3–9.5) | 6.2 (4.9–9.2) | 7.1 (5.5–9.6) | 0.384 |
Absolute Neutrophil Count (x103/mm3) | 4.8 (3.6–7.1) | 4.4 (3.2–6.6) | 4.8 (4.1–7.8) | 0.146 |
Absolute Lymphocyte Count (x103/mm3) | 0.94 (0.60–1.38) | 0.98 (0.59–1.54) | 0.91 (0.66–1.22) | 1 |
Neutrophil to Lymphocyte Ratio | 5.50 (3.31–7.24) | 5.74 (3.27–6.65) | 5.32 (3.65–9.95) | 0.479 |
Platelet Count (x103/mm3) | 208.5 (163.8–253) | 208.5 (182.5–290.5) | 209 (141–234) | 0.196 |
Hemoglobin (g/dL) | 13.4 (11.5–14.4) | 14.1 (12.9–15.2) | 12.3 (10.6–13.7) | 0.003 |
C-reactive protein (mg/dL) | 4.8 (1–12.1) | 3.6 (0.7–10.5) | 6.8 (3.6–12.3) | 0.074 |
Procalcitonin (ng/mL) | 0.11 (0.05–0.36) | 0.06 (0.05–0.13) | 0.42 (0.15–1.25) | <0.001 |
Ferritin (ug/L) | 380 (123–1310) | 227 (109–892) | 830 (276.5–1472.5) | 0.014 |
Lactate Dehydrogenase (U/L) | 314 (254.5–449.5) | 290 (234–366) | 412.5 (301–490) | 0.005 |
Aspartate Transaminase (U/L) | 52.5 (34.5–71.5) | 37 (31.5–72) | 53 (43–62) | 0.880 |
Alanine Aminotransferase (U/L) | 31.5 (17.7–53.2) | 47 (27–70) | 25 (10–36) | 0.047 |
Total Bilirubin (mg/dL) | 0.6 (0.4–0.8) | 0.6 (0.4–0.7) | 0.6 (0.5–0.8) | 0.391 |
Albumin (g/dL) | 3.7 (3.5–4.0) | 3.8 (3.6–4.1) | 3.6 (3.3–3.9) | 0.128 |
Fibrinogen (g/L) | 5.4 (3.7–6.9) | 4.5 (2.9–6.6) | 5.6 (4.9–7.9) | 0.050 |
Haptoglobin (mg/dL) | 260 (167.5–332.5) | 248 (159–330) | 269 (210–352.5) | 0.575 |
Myoglobin (ug/L) | 33.2 (17.9–185.5) | 18.4 (15.1–31) | 185.5 (41.7–385) | <0.001 |
D-dimer (ug/mL FEU) | 1.2 (0.7–1.8) | 1.2 (0.5–2.00) | 1.3 (0.9–1.7) | 0.550 |
TNF-α (pg/mL) | 2.8 (2.1–5.9) | 2.5 (1.7–3.7) | 4.8 (3.0–11.6) | 0.003 |
IL-6 (pg/mL) | 15.8 (5.2–36.6) | 9.7 (2.3–16.9) | 37.7 (15.9–49.1) | <0.001 |
IL-8 (pg/mL) | 13.5 (8.7–27.3) | 11.4 (5.4–15.9) | 26.4 (12.6–50.7) | <0,001 |
IL-10 (pg/mL) | 1.6 (0.6–3.3) | 0.9 (0.5–2.0) | 3.7 (1.1–6.0) | 0.004 |
Creatinine (mg/dL) | 0.96 (0.73–1.40) | 0.76 (0.71–0.92) | 1.77 (1.03–6.01) | <0.001 |
BUN (mg/dL) | 13.5 (10–25.2) | 11 (8–13) | 28.5 (15–61.2) | <0.001 |
BUN to Creatinine | 13.4 (10.9–18.3) | 12.9 (10.8–15.2) | 16.1 (12.2–20.2) | 0.099 |
Direct Renin (pg/mL) | 12.01 (5.76–57.29) | 11.23 (6.00–32.92) | 14.93 (5.30–104.70) | 0.324 |
Days Since Symptom Onset: median (IQR) | 7 (4–9.7) | 7 (4–10) | 7 (2–9) | 0.492 |
A total of 22 patients (42.3%) developed AKI over the course of COVID-19. Among these subjects, 12/22 (54.5%) developed severe AKI (KDIGO 2 + 3) and 8/22 (36.4%) required RRT. A total of 14/52 (26.9%) patients required ICU admission, 6/14 (42.8%) of whom developed severe AKI and 3/14 (21.4%) needed RRT. AKI was significantly more frequent in patients who required ICU admission (p = 0.002). Three patients with severe AKI died. The median age of patients developing AKI was 21 years older than those who did not (66 vs 45, respectively, p < 0.001). No difference was found with respect to sex. AKI was more frequently observed in white patients (77.8%), and least frequently in Hispanic subjects (15.8%). Patients who developed AKI had higher frequency of comorbid heart failure (p = 0.003), hypertension (p = 0.049), chronic kidney disease (p = 0.004), cerebrovascular disease (p = 0.001), and cancer (p = 0.027). With respect to baseline laboratory values, no differences between patients with and without AKI were observed for blood cell counts, despite that hemoglobin trended lower in patients with AKI (p = 0.003). With respect to inflammatory biomarkers, LDH (p = 0.005), ferritin (p = 0.014), myoglobin (p < 0.001), IL-6 (p < 0.001), IL-8 (p < 0.001), IL-10 (p = 0.004), and TNF-α (p = 0.003) were significantly higher at index ED visit in patients who developed AKI. Procalcitonin (PCT) was also higher in patients who developed AKI (<0.001).
3.2. NGAL and Cystatin C in COVID-19
Both sNGAL (p < 0.001) and sCysC (p = 0.002) levels were significantly elevated in patients who developed AKI during their COVID-19 course. The trends of sNGAL, sCysC, and ED SCr are shown in Table 3 and Fig. 1 . Both sCysC and ED SCr exhibited a significantly increasing trend with each subsequent increase in severity of renal injury. For sNGAL, while significantly elevated in patients with AKI, no difference was observed in patients who developed mild (KDIGO 1) as opposed to severe AKI (p = 0.381). However, sNGAL levels were significantly increased among patients with severe AKI who required RRT compared to those who did not (p < 0.01). Interestingly, 10/30 (33.3%) patients without AKI according to KDIGO criteria had elevated sCysC above the upper limit of the reference range, while this was observed in 2 patients (6.7%) for sNGAL. Neither sCysC (p = 0.093) or sNGAL (p = 0.075) were significantly elevated in patients who required ICU admission compared to those who did not.
Table 3.
Correlations with serum Cystatin C | ||
---|---|---|
Variable | Spearman's Correlation Coefficient | p-value |
C-reactive protein (mg/dL) | 0.033 | 0.826 |
Ferritin (ug/L) | 0.279 | 0.058 |
IL-6 (pg/mL) | 0.280 | 0.059 |
IL-10 (pg/mL) | 0.248 | 0.096 |
TNF-α (pg/mL) | 0.669 | <0.001 |
Neutrophil-to-Lymphocyte Ratio | 0.042 | 0.784 |
Platelets (x103/mm3) | 0.094 | 0.539 |
Procalcitonin (ng/mL) | 0.529 | <0.001 |
Fibrinogen (g/L) | 0.178 | 0.278 |
Myoglobin (ug/L) | 0.742 | <0.001 |
Lactate Dehydrogenase (U/L) | 0.230 | 0.119 |
ED Creatinine (mg/dL) | 0.825 | <0.001 |
Peak Creatinine (mg/dL) | 0.885 | <0.001 |
BUN (mg/dL) | 0.810 | <0.001 |
BUN to Creatinine | 0.140 | 0.354 |
Direct Renin (pg/mL) | 0.045 | 0.763 |
Correlations with serum NGAL | ||
C-reactive protein (mg/dL) | 0.352 | 0.012 |
Ferritin (ug/L) | 0.274 | 0.054 |
IL-6 (pg/mL) | 0.427 | 0.002 |
IL-10 (pg/mL) | 0.315 | 0.027 |
TNF-α (pg/mL) | 0.664 | <0.001 |
Neutrophil-to-Lymphocyte Ratio | 0.306 | 0.037 |
Platelets (x103/mm3) | 0.102 | 0.490 |
Procalcitonin (ng/mL) | 0.596 | <0.001 |
Fibrinogen (g/L) | 0.239 | 0.138 |
Myoglobin (ug/L) | 0.466 | <0.001 |
Lactate Dehydrogenase (U/L) | 0.214 | 0.135 |
ED Creatinine (mg/dL) | 0.550 | <0.001 |
Peak Creatinine (mg/dL) | 0.659 | <0.001 |
BUN (mg/dL) | 0.562 | <0.001 |
BUN to Creatinine | 0.102 | 0.484 |
Direct Renin (pg/mL) | 0.088 | 0.541 |
Cystatin C (mg/L) | 0.603 | <0.001 |
The correlations of sCysC and sNGAL with other inflammatory and renal biomarkers are shown in Table 3. Both sCysC and sNGAL were significantly inter-correlated (r = 0.603, p < 0.001), and were both also positively correlated with myoglobin, BUN, initial ED SCr and peak SCr during disease course. NGAL was positively correlated with neutrophil-to-lymphocyte ratio, CRP, LDH, IL-6, IL-8, IL-10, TNF-α, and IL-10. On the other hand, Cystatin C was not correlated with most inflammatory biomarkers, except TNF-α and procalcitonin. Moreover, cystatin C was not correlated with direct renin level.
3.3. Diagnostic performance of NGAL, Cystatin C, and ED Serum Creatinine
The diagnostic performance of sCysC and sNGAL for predicting AKI, need for RRT, and need for ICU admission are presented in Table 2 . Receiver operating characteristics (ROC) curves are presented in Fig. 2 . sCysC at ED presentation displayed excellent performance for predicting need for RRT during the course of disease (AUC, 0.95; 95% CI, 0.89–1.00), as well as for predicting AKI diagnosed with KDIGO criteria (AUC, 0.87; 95% CI, 0.77–0.98). sNGAL also displayed acceptable performance for predicting AKI (AUC, 0.81; 95% CI, 0.68–0.95) and need for RRT (AUC, 0.87; 95% CI, 0.75–1.0). However, both biomarkers demonstrated only modest performance for predicting need for ICU admission (both AUC, ∼0.66). The ED SCr performed similar to sCysC for predicting AKI and need for RRT, however, SCr displayed higher specificity for these outcomes, while sCysC displayed higher sensitivity. We have further assessed the Positive Predictive Values (PPV) and Negative Predictive Values (NPV) for each of these markers for development of AKI or need for RRT (Table 4 ). sCysC showed the highest PPV for development of AKI (PPV = 0.93, NPV = 0.818) compared to sNGAL (PPV = 0.875, NPV = 0.771). Interestingly, CysC had the lowest PPV for need for RRT, however with the highest NPV among the three markers (PPV = 0.53, NPV = 1). ED SCr showed the highest PPV for need for RRT while maintaining high NPV (PPV = 0.778, NPV = 0.976).
Table 2.
Variable | No AKI | Non-Severe AKI (KDIGO 1) | Severe AKI, no RRT (KDIGO 2 + 3) | Need for RRT |
---|---|---|---|---|
sCysC (mg/L) | 0.82 (0.74–0.99) | 1.63 (0.92–2.0) | 2.97 (0.96–5.92) | 4.83 (4.19–7.17) |
sNGAL (ng/mL) | 57.6 (49.1–95) | 123.5 (80.9–166.2) | 98.7 (58.5–169) | 392 (167.8–598.8) |
Creatinine (mg/dL) | 0.76 (0.71–0.92) | 1.2 (1.03–1.66) | 2.44 (0.87–4.51) | 6.94 (4.72–7.76) |
* Values presented as median (IQR). RRT, renal replacement therapy.
Table 4.
Serum Cystatin C (mg/L) | ||||||
---|---|---|---|---|---|---|
Optimal Cut-Off | Sensitivity | Specificity | AUC (95%CI) | PPV | NPV | |
AKI | 1.27 | 0.70 | 0.96 | 0.87 (0.77–0.98) | 0.933 | 0.818 |
Need for RRT | 3.22 | 1.00 | 0.83 | 0.94 (0.88–1.00) | 0.533 | 1 |
Need for ICU Admission | 0.98 | 0.73 | 0.61 | 0.65 (0.47–0.83) | N/A | N/A |
Serum NGAL (ng/L) | ||||||
AKI | 120 | 0.64 | 0.93 | 0.81 (0.68–0.95) | 0.875 | 0.771 |
Need for RRT | 190 | 0.75 | 0.93 | 0.87 (0.75–1.00) | 0.667 | 0.952 |
Need for ICU Admission | 135 | 0.56 | 0.89 | 0.66 (0.47–0.84) | N/A | N/A |
ED Serum Creatinine (mg/dL) | ||||||
AKI | 1.36 | 0.63 | 1.00 | 0.86 (0.75–0.97) | 1 | 0.778 |
Need for RRT | 3.14 | 0.87 | 0.95 | 0.94 (0.84–1.00) | 0.778 | 0.976 |
Need for ICU Admission | 0.62 | 1.00 | 0.12 | 0.44 (0.25–0.63) | N/A | N/A |
AKI, Acute Kidney Injury; AUC, Area Under the Curve; ED, Emergency Department; ICU, Intensive Care Unit; RRT, Renal Replacement Therapy.
3.4. Sparse partial least squares discriminant analysis for AKI development
PLS discriminant analysis were used to identify factors (excluding SCr) at ED presentation predictive for development of AKI in COVID-19 patients according to KDIGO criteria. Five-fold cross-validation procedures repeated 10 times over determined the optimal model to include one component made up of five variables, when deliberately leaving out sCr. The variables selected, in order of magnitude of contribution, were patient’s age, Cystatin C, Blood Urea Nitrogen, IL-8 and NGAL values. Allocation performance was very satisfactory, with an AUC as high as 0.942. Twenty out of 22 patients who had AKI (91%) were correctly assigned, and 27 out of 30 patients without AKI (90%) were correctly assigned, estimating the overall error rate at a modest 9.6%.
When the same analysis was conducted including ED SCr in the pool of available covariates, five-fold cross-validation procedures repeated 10 times over determined the optimal model to include one component made up of 4 variables. The variables selected, in order of magnitude of contribution, were patient’s age, Cystatin C, Blood Urea Nitrogen and Serum Creatinine. This model had similar performance to the prior, correctly assigning 20 out of 22 (90.9%) patients with AKI and 28 out of 20 (93.3%) patients without AKI, estimating the overall error rate at 7.7%. Variable contribution to the first component in each analysis is available in Fig. 3 .
4. Discussion
In this single center, prospective investigation, we sought to evaluate whether serum Cystatin C (sCysC) and serum NGAL (sNGAL) levels could be utilized as early biomarkers for the prediction of COVID-19-associated AKI, and if so, whether they show superiority to serum Creatinine (sCr) for this purpose. We found that sCysC and sNGAL measured at index ED visit indeed were strong predictors of AKI in patients with COVID-19, although overall they did not show superiority to sCr. sCysC displayed a stepwise increase with each increase in severity of renal injury, which mirrored that of SCr. Although sNGAL displayed the ability to differentiate patients with and without AKI, this biomarker appeared to lack the efficiency to distinguish patients with and without severe AKI, with exception of those requiring RRT. The diagnostic performance of sCysC was superior to that of sNGAL for both AKI and need for RRT, but comparable to that of SCr. Nonetheless, neither biomarker displayed good performance for predicting need for ICU admission or were significantly elevated at the index ED visit in patients who progressed to requiring ICU support. This evidence would hence suggest that these two biomarkers may play an important role in specifically identifying patients at higher risk of developing AKI, or even subacute kidney injury, independent and notwithstanding progression and severity of COVID-19. Importantly, given the high rate of mortality associated with RRT in COVID-19 AKI, which is reported to be as high as 63% [3], and the growing evidence showing sCysC as an independent predictor of mortality in COVID-19 [13], the diagnostic performance of sCysC (0.94 AUC), suggests that the measurement of this biomarker at initial presentation may enable early monitoring and more aggressive intervention in patients at high risk of mortality. While sCysC provided a higher sensitivity than sCr (100% vs. 87%) for predicting the need for RRT, SCr displayed an equal AUC, and higher specificity (95% vs. 83% for RRT, and 100% vs. 89% for AKI).
Of note, among patients without COVID-19 AKI diagnosed per KDIGO SCr criteria, nearly one-third had a sCysC measurement at index ED visit above the upper limit of the reference range, while only ∼7% had elevated sNGAL. While there are multitude of co-morbidities that may be associated with upregulated Cystatin C, the high frequency of elevation observed in this study raises the potential of subclinical AKI in many patients with otherwise mild or moderate COVID-19. Given that KDIGO SCr criteria was used to define AKI in this study, these findings may not be surprising, since altered GFR resulting in rising SCr may only become apparent when >50% of the renal mass is compromised, as may occur in patients with relatively mild infection [14]. Nonetheless, subclinical AKI may be associated with poor short-term outcome and long term sequelae [14]. Both NGAL and Cystatin C have been demonstrated to be earlier markers of both AKI and subclinical AKI compared to SCr in several etiologies [5], [15], and their role in identifying subclinical AKI in COVID-19 should be the subject of future prospective studies.
In addition to their diagnostic value, sNGAL and sCysC measurement may help shed some light on the yet to be elucidated pathophysiology of COVID-19 AKI. A wide range of mechanisms appear to contribute to COVID-19 AKI, encompassing direct cellular injury, thrombo-inflammation, overactivation of complement, dysregulation of the renin–angiotensin–aldosterone system, collapsing glomerulopathy, and endothelial injury, as well as factors related to critical illness, including hemodynamic alterations, right heart failure, sepsis, administration of nephrotoxic medications, and severe hypoxemia [1], [4]. To-date, histopathologic studies have demonstrated a wide range of pathologic findings, including acute tubular necrosis, with lymphocyte and macrophage infiltration being a prominent feature, as well as peritubular erythrocyte aggregation and glomerular fibrin thrombi with ischemic collapse [16], [17], [18]. Interestingly, Angiotensin Converting Enzyme 2 (ACE2), the human host receptor for SARS-CoV-2, is more highly expressed in the kidney than in the lung, especially on podocytes and proximal straight tubule cells [19], [20], [21]. While some evidence suggests viral tropism and direct infection of kidney parenchyma via SARS-CoV-2 particles identified in electron microscopy or via viral RNA and/or proteins detected via molecular methods [[16], [22], [23]], other studies have shown conflicting findings [18]. However, recently Braun et al. were able to extract replication-competent SARS-CoV-2 from renal tissue on autopsy [24]. Most recently, Omer et al. demonstrated that while infection of kidney cells with SARS-CoV-2 occurs, it usually does not result in a cytopathic effect [25]. Overall, notwithstanding renal viral tropism, it seems more likely that the etiology of COVID-19 AKI is a multifactorial.
To better understand potential mechanisms, we correlated sNGAL and sCysC with a variety of other renal and inflammatory biomarkers in our cohort. NGAL, a 178 amino acid long glycoprotein, is an early marker of tubular damage, and thus may have high utility in COVID-19 AKI, which has been shown to be characterized by a specific proximal tubular dysfunction [26]. We found that sNGAL was significantly correlated with the majority of inflammatory biomarkers measured in this study. This is not surprising given that NGAL can also play a role as a pro-inflammatory molecule, being elevated even in absence of AKI [27]. However, it appears that systemic inflammation may in part contribute to COVID-19 pathophysiology via thromboinflammation and other immune-mediated pathways, as noted in the PLS discriminant analysis where IL-8 was identified as one of the five most important factors for predicting AKI development. sNGAL values were also correlated with IL-10, which is thought to produce renal protective effects through induction of NGAL expression [28]. Although a hyperinflammatory state may have at least partially blunted the ability of NGAL to differentiate between severe and non-severe AKI, sNGAL (like sCysC) was highly correlated with BUN, ED SCr and peak SCr during COVID-19 course. Overall, the presence of elevated NGAL at ED presentation suggests that tubular injury occurs relatively early in disease process, and substantially contributes to COVID-19 AKI, regardless of the driving pathophysiologic mechanism.
sCysC, which had better diagnostic performance compared to NGAL for AKI and need for RRT, did not correlate with the majority of inflammatory biomarkers, nor with direct renin levels, suggesting that at least early in the disease course, prerenal (functional) mechanisms may have a minimal role in AKI development. Moreover, no correlation was observed with LDH, a prominent marker of severe COVID-19 [29], despite significant observations of low ADAMTS13 activity suggestive of a secondary microangiopathy phenomenon in patients with COVID-19 [30]. Of note, both sNGAL and sCysC where highly correlated with myoglobin (both p < 0.001), thus suggesting that rhabdomyolysis may also play an important role in the risk of developing COVID-19 AKI. High myoglobin at admission has been noted to be associated with severe COVID-19 [31], and the role of this muscle protein should be further investigated in future studies on COVID-19 AKI. AKI is noted to contribute to immune dysfunction through a variety of mechanisms, including neutrophil and thrombocyte impairment, as well as increased production and reduced clearance of cytokines [32], which may in part explain the high morbidity and mortality observed in patients with COVID-19 AKI.
In our PLS discriminant analysis, age was determined to be the most important contributor to AKI development. This is consistent with the abundance of literature demonstrating that age is the single most important contributing factor to COVID-19 severity [33], as well as literature showing that the AKI risk with severe illness typically increases with advancing age [34]. After age, we found elevated sCysC to be the second most profound factor impacting progression to AKI, in line with its diagnostic performance. The question remains, however, what is its utility as a biomarker in the early presentation of patients with COVID-19, compared to sCr. On the one hand, sCysC is a strong predictor not only of disease severity and mortality in COVID-19, but (as shown herein) also for COVID-19-associated AKI, to a degree comparable to that of sCr. On the other hand, a single run of sCysC would cost around 20-fold more than sCr in our local setting (i.e., $3.00 USD vs. $0.15 USD) [35]. While our main conclusion remains that sCr is as good as cCysC as a biomarker for COVID-19-associated AKI and need for RRT, the value of a single measurement of sCysC at initial ED presentation could enable patient risk stratification and early prophylactic interventions, thus likely marginalizing the cost of the test.
This study was limited by a relatively small sample size and the results should be confirmed in larger diverse cohorts. The study design did not include the collection of urine samples, nor was urine collection part of standard care in the ED. However, future investigations measuring urine NGAL as opposed to sNGAL, along with other novel renal biomarkers, would be required to further elucidate the mechanisms of COVID-19 AKI. Lastly, when a pre-COVID-19 SCr value was unavailable, the lowest sCr during hospitalization or within 30 days of ED visit was employed as a baseline sCr for AKI assessment. Thus, it is possible that incidents of AKI were missed in patients who presented with mild AKI and whose SCr values resolved during illness course. As such, the incidence of AKI may have been underrepresented, which in part, may potentially explain the observation of elevated sCysC in patients without a final diagnosis of AKI.
5. Conclusion
sCysC measured at ED presentation has an excellent overall performance as an early predictor of AKI and need for RRT in patients with COVID-19, however does not surpass that of sCr, whilst sNGAL demonstrates only adequate diagnostic performance. Elevated sNGAL suggests that acute tubular injury occurs relatively early in the course of disease and is correlated with inflammation. Given the predictive value of sCysC not only for COVID-19-associated AKI, but also for disease severity and mortality, we suggest considering its measurement upon presentation, to enable patient risk stratification and early intervention.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Funding
This study was funded by the University of Cincinnati College of Medicine Special Coronavirus (COVID-19) Research Pilot Grant Program.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.clinbiochem.2022.01.006.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
References
- 1.Nadim M.K., Forni L.G., Mehta R.L., Connor M.J., Liu K.D., Ostermann M., et al. COVID-19-associated acute kidney injury: consensus report of the 25th Acute Disease Quality Initiative (ADQI) Workgroup. Nature Reviews Nephrology. 2020;15:1–18. doi: 10.1038/s41581-020-00356-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.L. Chan K. Chaudhary A. Saha K. Chauhan A. Vaid S. Zhao et al. AKI in Hospitalized Patients with COVID-19 JASN [Internet]. 2020 Sep 3 [cited 2020 Nov 11]; Available from: https://jasn.asnjournals.org/content/early/2020/09/02/ASN.2020050615.
- 3.S. Gupta S.G. Coca L. Chan M.L. Melamed S.K. Brenner S.S. Hayek et al. AKI Treated with Renal Replacement Therapy in Critically Ill Patients with COVID-19 JASN [Internet]. 2020 Oct 16 [cited 2020 Nov 11]; Available from: https://jasn.asnjournals.org/content/early/2020/10/15/ASN.2020060897. [DOI] [PMC free article] [PubMed]
- 4.Cheruiyot I, Henry B, Lippi G, Kipkorir V, Ngure B, Munguti J, et al. Acute Kidney Injury is Associated with Worse Prognosis In COVID-19 Patients: A Systematic Review and Meta-analysis. 1. 2020 Jul 16;91:ahead of print-ahead of print. [DOI] [PMC free article] [PubMed]
- 5.Devarajan P. NGssion serum cystatin C level be an ey injury in the emergency room. Biomark Med. 2014;8:217–219. doi: 10.2217/bmm.13.149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.M. Ostermann A. Zarbock S. Goldstein K. Kashani E. Macedo R. Murugan M. Bell L. Forni L. Guzzi M. Joannidis S.L. Kane-Gill M. Legrand R. Mehta P.T. Murray P. Pickkers M. Plebani J. Prowle Z. Ricci T. Rimmelé M. Rosner A.D. Shaw J.A. Kellum C. Ronco Recommendations on Acute Kidney Injury Biomarkers From the Acute Disease Quality Initiative Consensus Conference: A Consensus Statement JAMA Netw Open. 3 10 2020 e2019209 10.1001/jamanetworkopen.2020.19209. [DOI] [PubMed]
- 7.He Y., Deng Y., Zhuang K., Li S., Xi J., Chen J. Predictive value of cystatin C and neutrophil gelatinase-associated lipocalin in contrast-induced nephropathy: A meta-analysis. PLOS ONE. 2020;15 doi: 10.1371/journal.pone.0230934. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lippi G., Plebani M. Neutrophil gelatinase-associated lipocalin (NGAL): the laboratory perspective. Clin Chem Lab Med. 2012 Jun 27:1–5. doi: 10.1515/cclm-2012-0344. [DOI] [PubMed] [Google Scholar]
- 9.KDIGO AKI Working Group KDIGO clinical practice guideline for acute kidney injury. Kidney International Suppl. 2012;2:1. [Google Scholar]
- 10.Ognibene A., Mannucci E., Caldini A., Terreni A., Brogi M., Bardini G., Sposato I., Mosconi V., Salvadori B., Rotella C.M., Messeri G. Cystatin C reference values and aging. Clinical Biochemistry. 2006;39:658–661. doi: 10.1016/j.clinbiochem.2006.03.017. [DOI] [PubMed] [Google Scholar]
- 11.Negrin L.L., Hahn R., Heinz T., Hajdu S. Diagnostic Utility of Serum Neutrophil Gelatinase-Associated Lipocalin in Polytraumatized Patients Suffering Acute Kidney Injury: A Prospective Study. BioMed Research International. 2018;2018:1–11. doi: 10.1155/2018/2687584. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Fluss R., Faraggi D., Reiser B. Estimation of the Youden Index and its Associated Cutoff Point. Biometrical Journal. 2005;47:458–472. doi: 10.1002/bimj.200410135. [DOI] [PubMed] [Google Scholar]
- 13.Zinellu A., Mangoni A.A., Cystatin C., Newman D.J. COVID-19 severity and mortality: a systematic review and meta-analysis. J Nephrol. 2021 doi: 10.1007/s40620-021-01139-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Ronco C., Kellum J.A., Haase M. Subclinical AKI is still AKI. Crit Care. 2012;16:313. doi: 10.1186/cc11240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Gaygısız Ü., Aydoğdu M., Badoğlu M., Boyacı N., Güllü Z., Gürsel G. Can admission serum cystatin C level be an early marker subclinical acute kidney injury in critical care patients? Scand J Clin Lab Invest. 2016;76:143–150. doi: 10.3109/00365513.2015.1126854. [DOI] [PubMed] [Google Scholar]
- 16.Su H., Yang M., Wan C., et al. Renal histopathological analysis of 26 postmortem findings of patients with COVID-19 in China. Kidney International. 2020;98:219–227. doi: 10.1016/j.kint.2020.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Santoriello D., Khairallah P., Bomback A.S., Xu K., Kudose S., Batal I., Barasch J., Radhakrishnan J., D’Agati V., Markowitz G. Postmortem Kidney Pathology Findings in Patients with COVID-19. JASN. 2020;31:2158–2167. doi: 10.1681/ASN.2020050744. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Diao B, Wang C, Wang R, Feng Z, Tan Y, Wang H, et al. Human Kidney is a Target for Novel Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Infection. medRxiv. 2020 Apr 10;2020.03.04.20031120.
- 19.Ye M., Wysocki J., William J., et al. Glomerular Localization and Expression of Angiotensin-Converting Enzyme 2 and Angiotensin-Converting Enzyme: Implications for Albuminuria in Diabetes. JASN. 2006;17:3067–3075. doi: 10.1681/ASN.200605042320. [DOI] [PubMed] [Google Scholar]
- 20.Batlle D., Soler M.J., Sparks M.A., et al. Acute Kidney Injury in COVID-19: Emerging Evidence of a Distinct Pathophysiology. JASN. 2020;31:1380–1383. doi: 10.1681/ASN.202004041921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Pan X.-w., Xu D.a., Zhang H., et al. Identification of a potential mechanism of acute kidney injury during the COVID-19 outbreak: a study based on single-cell transcriptome analysis. Intensive Care Med. 2020:1–3. doi: 10.1007/s00134-020-06026-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Puelles V.G., Lütgehetmann M., Lindenmeyer M.T., et al. Multiorgan and Renal Tropism of SARS-CoV-2. New England Journal of Medicine. 2020;383:590–592. doi: 10.1056/NEJMc201140023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Bradley B.T., Maioli H., Johnston R., et al. Histopathology and ultrastructural findings of fatal COVID-19 infections in Washington State: a case series. Lancet. 2020;396:320–332. doi: 10.1016/S0140-6736(20)31305-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Braun F., Lütgehetmann M., Pfefferle S., et al. SARS-CoV-2 renal tropism associates with acute kidney injury. Lancet. 2020;396:597–598. doi: 10.1016/S0140-6736(20)31759-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Omer D., Pleniceanu O., Gnatek Y., Namestnikov M., Cohen-Zontag O., Goldberg S., Friedman Y.E., Friedman N., Mandelboim M., Vitner E.B., et al. Human Kidney Spheroids and Monolayers Provide Insights into SARS-CoV-2 Renal Interactions. J Am Soc Nephrol. 2021;32(9):2242–2254. doi: 10.1681/ASN.2020111546. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Werion A., Belkhir L., Perrot M., et al. SARS-CoV-2 causes a specific dysfunction of the kidney proximal tubule. Kidney Int. 2020;98:1296–1307. doi: 10.1016/j.kint.2020.07.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Choi J.W., Fujii T., Fujii N. Significance of Neutrophil Gelatinase-Associated Lipocalin Level-to-Serum Creatinine Ratio for Assessing Severity of Inflammation in Patients with Renal Dysfunction. Biomed Res Int [Internet]. 2015;2015:1–7. doi: 10.1155/2015/791926. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Jung M., Sola A., Hughes J., et al. Infusion of IL-10-expressing cells protects against renal ischemia through induction of lipocalin-2. Kidney Int. 2012;81:969–982. doi: 10.1038/ki.2011.446. [DOI] [PubMed] [Google Scholar]
- 29.Henry B.M., Aggarwal G., Wong J., et al. Lactate dehydrogenase levels predict coronavirus disease 2019 (COVID-19) severity and mortality: A pooled analysis. Am J Emerg Med. 2020;38:1722–1726. doi: 10.1016/j.ajem.2020.05.073. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Martinelli N., Montagnana M., Pizzolo F., et al. A relative ADAMTS13 deficiency supports the presence of a secondary microangiopathy in COVID 19. Thromb Res. 2020;193:170–172. doi: 10.1016/j.thromres.2020.07.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Henry B.M., de Oliveira M.H.S., Benoit S., et al. Hematologic, biochemical and immune biomarker abnormalities associated with severe illness and mortality in coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chem Lab Med. 2020;58:1021–1028. doi: 10.1515/cclm-2020-0369. [DOI] [PubMed] [Google Scholar]
- 32.Singbartl K., Formeck C.L., Kellum J.A. Kidney-Immune System Crosstalk in AKI. Seminars in Nephrology. 2019;39:96–106. doi: 10.1016/j.semnephrol.2018.10.007. [DOI] [PubMed] [Google Scholar]
- 33.Lippi G., Sanchis-Gomar F., Henry B.M. COVID-19: unravelling the clinical progression of nature’s virtually perfect biological weapon. Annals of Translational Medicine. 2020;8:693. doi: 10.21037/atm-20-3989. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Chao C.-T., Wang J., Wu H.-Y., Huang J.-W., Chien K.-L. Age modifies the risk factor profiles for acute kidney injury among recently diagnosed type 2 diabetic patients: a population-based study. GeroScience. 2018;40:201–217. doi: 10.1007/s11357-018-0013-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Lamb E.J., O’Riordan S.E., Webb M.C., Newman D.J. Serum Cystatin C May Be a Better Marker of Renal Impairment than Creatinine. Journal of the American Geriatrics Society. 2003;51:1674–1675. doi: 10.1046/j.1532-5415.2003.515244.x. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.