ABSTRACT
Background
The 2021 clinical guidelines of the Kidney Disease: Improving Global Outcomes emphasize the importance of the histological activity index (AI) in the management of lupus nephritis (LN). Patients with LN and a high AI have poor renal outcomes and high rates of nephritic relapse. In this study we constructed prediction models for the AI in LN.
Methods
The study population comprised 337 patients diagnosed with LN using kidney biopsy. The participants were randomly divided into training and testing cohorts. They were further divided into high-activity (AI >2) and low-activity (AI ≤2) groups. This study developed two clinical prediction models using logistic regression and least absolute shrinkage and selection operator (LASSO) analyses with laboratory test results collected at the time of kidney biopsy. The performance of models was assessed using 5-fold cross-validation and validated in the testing cohort. A nomogram for individual assessment was constructed based on the preferable model.
Results
Multivariate analysis showed that higher mean arterial pressure, lower estimated glomerular filtration rate, lower complement 3 level, higher urinary erythrocytes count and anti-double-stranded DNA seropositivity were independent risk factors for high histologic activity in LN. Both models performed well in the testing cohort regarding the discriminatory ability to identify patients with an AI >2. The average area under the curve of 5-fold cross-validation was 0.855 in the logistic model and 0.896 in the LASSO model. A webtool based on the LASSO model was created for clinicians to enter baseline clinical parameters to produce a probability score of an AI >2.
Conclusions
The established nomogram provides a quantitative auxiliary tool for distinguishing LN patients with a high AI and helps physicians make clinical decisions in their comprehensive assessment.
Keywords: activity index, kidney biopsy, LASSO, lupus nephritis, nomogram
Graphical Abstract
Graphical Abstract.
Watch the video of this contribution at https://academic.oup.com/ndt/pages/author_videos
KEY LEARNING POINTS.
What was known:
Patients with lupus nephritis (LN) and high histopathological activity have poor renal outcomes and high rates of nephritic relapse.
Clinical remission of LN is not consistent with renal histologic remission.
The activity index (AI) is important in the management of lupus nephritis.
This study adds:
This study constructed a nomogram for predicting the pathohistological AI based on routine examinations results.
Potential impact:
Diagnostic or repeat kidney biopsy, intensive immunosuppressive therapy and disease surveillance are recommended for patients with a high suspicion of high histologic activity.
Avoid a repeat kidney biopsy in patients with low risk of active LN.
INTRODUCTION
Lupus nephritis (LN) is a common and serious complications of systemic lupus erythematosus (SLE). Most patients are diagnosed with LN within 5 years after diagnosis with SLE and ≈5–20% patients with LN progress to end-stage renal disease [1, 2].
The ultimate therapeutic goal of LN is to preserve renal function as much as possible and a key factor in the management of LN is to accurately determine the degree of renal histological activity. The activity index (AI) and chronicity index (CI) of the histopathological features of LN were proposed by Austin et al. [3] and modified by the International Society of Nephrology in 2018. It is recommended that the A, C and A/C parameters in class III/IV be substituted by the AI and CI and the use of these indices is not restricted to class III/IV but is applied to all classes [4]. In 2021, AI and CI were listed in the Kidney Disease: Improving Global Outcomes (KDIGO) clinical guidelines for the first time, and the guidelines pointed out that clinicians need to pay more attention to potentially reversible active lesions [5]; however, the treatment strategies for the AI still need further research. Based on previous literature on disease progression and renal pathology, LN patients with an AI >2 have an increasing risk of poor renal outcome and are predictive of LN relapse after withdrawal of maintenance immunosuppression [6–10].
The induction treatment for LN mostly depends on pathological classification, which is then switched to maintenance treatment when complete clinical remission is achieved. However, some studies have found that clinical remission, as evaluated by proteinuria and renal function, is not consistent with renal histological remission [7, 11–13]. Moreover, the changes and transformations in the histologic classification of LN over time have been observed in ≈40–76% of patients [1, 14]. There is no doubt that histopathological features are critical guidance for the management of LN, but it is an invasive procedure and its indications remain controversial. Clinicians have been using a holistic assessment of abnormal clinical manifestations and specific criteria and quantitative evaluation tools for repeat kidney biopsies have been lacking until now.
In this study we constructed a nomogram for the degree of histologic activity using a series of blood and urine tests results, which helps physicians objectively evaluate histologic activity, thereby helping to make clinical decisions about performing kidney biopsy early and in a timely manner while avoiding unnecessary renal biopsies.
MATERIALS AND METHODS
Study design and population
The study population included patients with LN confirmed by kidney biopsy between January 2015 and June 2020 at the First Hospital of Zhejiang University School of Medicine (288 patients), the First Affiliated Hospital of Ningbo University (73 patients) and the Fourth Hospital of Zhejiang University School of Medicine in China (11 patients). Patients with any of the following conditions were excluded: age <18 years, any other types of renal disease and missing renal pathology reports or critical data (Supplementary Fig. S1).
In this study, 337 patients were included and randomly divided in a ratio of 7:3 into training (235 patients) and testing (102 patients) cohorts. The baseline clinical characteristics of the study cohorts are shown in Table 1. Based on the results of previous studies on the AI [6–8, 13], the training cohort was divided into two groups: high (AI >2) and low (AI ≤2). Logistic regression and least absolute shrinkage and selection operator (LASSO) analyses were used to develop models to predict the activity of histology in LN if the AI is >2. Details of the statistical analysis pipeline are shown in Supplementary Fig. S2.
Table 1:
Demographic and clinical characteristics of the training and testing cohorts
| Characteristics | Total (N = 337) | Training cohort (n = 235) | Testing cohort (n = 102) |
|---|---|---|---|
| Female, n (%) | 279 (82.8) | 201 (85.5) | 78 (76.5) |
| Age (years), median (IQR) | 36 (25–46) | 36 (26–45) | 36 (26–45) |
| BMI (kg/m2), median (IQR) | 22.2 (19.8–24.0) | 21.2 (19.8–24.0) | 22.1 (19.6–23.9) |
| MAP (mmHg), mean ± SD | 100.5 ± 13.9 | 101.3 ± 13.7 | 98.7 ± 14.1 |
| LN classification, n (%) | |||
| Ⅰ, Ⅱ | 9 (2.7) | 6 (2.6) | 3 (2.95) |
| Ⅲ, Ⅳ, Ⅲ/Ⅳ ± Ⅴ | 201 (59.6) | 142 (60.4) | 59 (57.8) |
| Ⅴ | 50 (14.8) | 35 (14.9) | 15 (14.7) |
| Ⅵ | 5 (1.5) | 3 (1.3) | 2 (2.0) |
| Post-treatment | 72 (21.4) | 49 (20.9) | 23 (22.5) |
| AI, median (IQR) | 5 (2–7) | 5 (2–7) | 5 (2–7) |
| AI >2, n (%) | 233 (69.1) | 164 (69.8) | 69 (67.6) |
| CI, median (IQR) | 3 (2–3) | 3 (2–3) | 3 (2–3) |
| WBC count (× 109/l), median (IQR) | 6.0 (3.6–7.5) | 5.9 (3.6–7.2) | 6.1 (3.9–8.0) |
| Hb (g/l), median (IQR) | 103.1 (87.0–117.0) | 103.7 (87.0–117.0) | 101.8 (87.8–116.5) |
| Hct (%), mean ± SD | 31.6 ± 6.0 | 31.7 ± 6.1 | 31.3 ± 5.8 |
| Plt (× 109/l), median (IQR) | 183.8 (127.0–236.5) | 185.6 (127.0–238.0) | 180.0 (127.0–223.0) |
| Serum albumin (g/L), mean ± SD | 27.9 ± 6.8 | 27.6 ± 6.5 | 28.7 ± 7.4 |
| SCr (μmol/l), median (IQR) | 110.8 (60.0–122.0) | 106.2 (60.0–115.0) | 121.3 (58.8–151.0) |
| BUN (mmol/l), median (IQR) | 9.4 (4.8–11.4) | 8.9 (4.8–11.0) | 10.4 (5.1–12.3) |
| UA (μmol/l), median (IQR) | 401.0 (315.0–486.0) | 399.0 (314.0–481.0) | 406.5 (313.8–512.1) |
| eGFR (ml/min/1.73 m2, median (IQR) | 83.5 (53.5–114.7) | 85.3 (57.7–114.1) | 79.4 (42.9–116.3) |
| ESR (mm/h), median (IQR) | 36.1 (17.0–51.0) | 37.4 (17.0–51.0) | 33.0 (16.8–45.3) |
| C3 (mg/dl), median (IQR) | 52.2 (35.0–66.0) | 52.5 (36.0–66.0) | 51.6 (34.0–65.0) |
| C4 (mg/dl), median (IQR) | 9.7 (4.0–13.8) | 9.5 (3.0–13.0) | 8.8 (4.0–14.0) |
| CRP (mg/l), median (IQR) | 5.2 (1.4–5.1) | 4.7 (1.3–4.8) | 6.1 (1.5–6.4) |
| logANAa, median (IQR) | 2.3 (2.2–2.5) | 2.3 (2.2–2.5) | 2.3 (2.1–2.5) |
| Anti-dsDNA seropositive, n (%) | 196 (58.2) | 137 (58.3) | 59 (57.8) |
| UPCR (g/g), median (IQR) | 3.37 (1.52–4.34) | 3.48 (1.54–4.42) | 3.13 (1.44–3.83) |
| log2 URBCb (/μl), median (IQR) | 6. 3(4.7–7.8) | 6.1 (4.5–7.7) | 6.5 (5.1–8.1) |
| uCast positive, n (%) | 125 (37.1) | 91 (38.7) | 34 (33.3) |
WBC, white blood cell; Hb, haemoglobin; Hct, haematocrit; Plt, platelet; UA, uric acid; ESR, erythrocyte sedimentation rate; CRP: C-reactive protein; ANA: antinuclear antibody; uCast, urinary cast.
Titre expressed as logANA after log transformed base 10.
The dispersion of URBC is large (the coefficient of variation of URBC is 3.79). The count of red cells in urea is expressed as log2 URBC after log transformed base 2.
Clinical data and definitions
Demographic information, history, renal pathology results and blood and urine test results were obtained from the electronic medical records of the three hospitals. Pathology reports for all the cases were retrospectively analysed and the AI and classification were re-evaluated according to modified activity and chronicity indices and clarification of definitions [4]. In this study, some patients underwent kidney biopsy after a period of treatment. The renal pathology of these patients was evaluated by specialized pathologists, but it cannot be defined as a certain classification (Ⅰ–Ⅵ), which was described as ‘post-treatment’. Laboratory test data used to construct the prediction models were collected 1–3 days before the kidney biopsy. Estimated glomerular filtration rate (eGFR) was calculated using the Chronic Kidney Disease Epidemiology Collaboration equation [15]. To evaluate the treatment response of the study population, therapy regimens and serum creatinine (SCr) levels, eGFR and spot urine protein:creatinine ratio (UPCR) were collected after 6 months of treatment. The treatment response was defined based on the 2021 KDIGO guidelines. Complete clinical renal response (CRR) was defined as a UPCR <0.5 g/g and improved or stable renal function (eGFR ±10–15% of baseline). Partial renal response (PRR) was defined as a UPCR <3.0 g/g with at least a 50% reduction from baseline and improved or stable renal function (eGFR ±10–15% of baseline). Failure to achieve CRR or PRR was defined as no response (NR). This study was approved by the Ethics Committee of the First Hospital of the Zhejiang University School of Medicine (2020 IIT No. 571).
Statistics
Continuous variables are presented as mean ± standard deviation (SD) or median and interquartile range (IQR), and groups were compared using the t-test or Mann–Whitney U test. Categorical variables are presented as numbers and percentages, and groups were compared using the χ2 test or Fisher's exact test. The correlation between AI as a continuous variable and other factors was analysed using Spearman's correlation. In the statistical tests, continuous variables with large coefficients of variation were log-transformed to reduce variability. Logistic regression [stepwise regression according to the Akaike information criterion (AIC)] and LASSO analyses were used to filter the clinical variables and construct prediction models, respectively (logistic model, LASSO model). The value of λ in the LASSO model corresponds to one standard error greater than the minimum mean squared error (MSE) (lambda.1se). Performance of the prediction model was evaluated using the area under the curve (AUC) of the receiver operating characteristics (ROC) curve and the calibration curve. The optimal cut-off value of the ROC curve was determined using the maximum Youden's index. The average AUC of K-fold cross-validation (k = 5) (cvAUC) was used to assess the generalization capability of the model. Both the models were validated in the testing cohort. A nomogram was constructed based on the regression equation of the preferable model. All statistical analyses were performed using SPSS Statistics version 25 (IBM, Armonk, NY, USA) and R version 4.2.2 (R Foundation for Statistical Computing, Vienna, Austria). P-values <.05 were considered statistically significant.
RESULTS
Baseline characteristics of the training and testing cohorts
A total of 337 patients with LN were included in this study, including 235 in the training cohort and 102 in the testing cohort; 82.8% of the patients in this study were females and the median age was 36 years. Approximately 60% of the patients were diagnosed with proliferative LN (Ⅲ, Ⅳ and Ⅲ/Ⅳ + Ⅴ). The AI in the study cohort ranged from 0 to 17, with a median of 5, and 233 patients (69.1%) had an AI >2, while the CI ranged from 0 to 9, with a median of 3. Approximately 20% of the patients underwent kidney biopsy after treatment rather than at the time of initial diagnosis, and almost all post-treatment patients had an AI >0 (n = 70); more than half of these patients had an AI >2 (n = 37), while 74% of those with a first LN diagnosis had an AI >2.
Additionally, we retrospectively analysed the relationship between induction therapy and the rate of clinical renal remission at 6 months after kidney biopsy in the study cohort. There were 68 patients in the low-activity group and 114 in the high-activity group after excluding missing data, and the CRRs were 61.8% and 71.9%, respectively. There were no significant differences in the rates of CRR and PRR between the two groups; however, in the high-activity group, glucocorticoids combined with immunosuppressive therapy was the most common regimen versus glucocorticoids alone (immunosuppressive agents referring to cyclophosphamide, calcineurin inhibitors and mycophenolic acid analogues, etc.) (χ2 = 5.240, P = .029).
Factors affecting the degree of histologic activity
Among all the clinical variables in our study, only SCr showed a significant correlation with AI (r > 0.4, P < .05). The results of the univariate and multivariate logistic regression analyses are presented in Table 2. Multivariate logistic regression analysis showed that higher mean arterial pressure (MAP) {odds ratio [OR] 1.037 [95% confidence interval (CI) 1.012–1.063], P = .003], lower eGFR [OR 0.975 (95% CI 0.965–0.984), P < .001], lower complement 3 (C3) level [OR 0.964 (95% CI 0.949–0.978), P < .001], higher urinary erythrocytes counts (URBC) [log2 transformed, OR 1.182 (95% CI 1.031–1.265), P = .019] and anti-double-stranded DNA (anti-dsDNA) seropositive [OR 3.998 (95% CI 2.160–7.578), P < .001] were independent risk factors for high histologic activity in LN. The AUCs for these factors indicated a poor discriminatory ability with respect to the degree of histologic activity (Supplementary Fig. S3). The sensitivities and specificities of these individual parameters were low.
Table 2:
Correlation analysis of the AI and multivariate logistic regression analysis of factors associated with AI >2
| Correlation analysis | Univariate | Multivariate | ||||
|---|---|---|---|---|---|---|
| Variables | Spearman’s r | P-value | OR (95% CI) | P-value | OR (95% CI) | P-value |
| CIa | 0.167 | .002 | – | – | ||
| Female | 0.010 | .853 | 1.209 (0.645–2.268) | .553 | ||
| Age (years) | −0.008 | .881 | 0.999 (0.981–1.017) | .916 | ||
| BMI (kg/m2) | −0.086 | .115 | 0.965 (0.905–1.029) | .278 | ||
| MAP (mmHg) | 0.239 | <.001 | 1.048 (1.029–1.068) | <.001 | 1.037 (1.012–1.063) | .003 |
| WBC (× 109/l) | 0.040 | .465 | 1.008 (0.936–1.086) | .835 | ||
| Hb (g/l) | −0.317 | <.001 | 0.966 (0.954–0.978) | <.001 | ||
| Hct (%) | −0.370 | <.001 | 0.874 (0.836–0.914) | <.001 | ||
| Plt (× 109/l) | −0.182 | .001 | 0.994 (0.991–0.997) | <.001 | ||
| Serum albumin (g/l) | −0.288 | <.001 | 0.924 (0.890–0.959) | <.001 | ||
| SCr (μmol/l) | 0.405 | <.001 | 1.018 (1.011–1.025) | <.001 | ||
| BUN (mmol/l) | 0.357 | <.001 | 1.225 (1.139–1.318) | <.001 | ||
| UA (μmol/l) | 0.277 | <.001 | 1.006 (1.004–1.008) | <.001 | ||
| eGFR (ml/min/1.73 m2) | −0.382 | <.001 | 0.971 (0.962–0.979) | <.001 | 0.975 (0.965–0.984) | <.001 |
| ESR (mm/h) | −0.057 | .301 | 0.993 (0.984–1.003) | .164 | ||
| C3 (mg/dl) | −0.346 | <.001 | 0.958 (0.947–0.970) | <.001 | 0.964 (0.949–0.978) | .001 |
| C4 (mg/dl) | −0.203 | <.001 | 0.942 (0.914–0.971) | <.001 | ||
| CRP (mg/l) | 0.034 | .540 | 1.004 (0.977–1.031) | .798 | ||
| logANA | 0.156 | .004 | 1.253 (0.764–2.054) | .372 | ||
| Anti-dsDNA positive | 0.313 | <.001 | 3.891 (2.391–6.332) | <.001 | 3.998 (2.160–7.578) | <.001 |
| UPCR (g/g) | 0.138 | .011 | 1.103 (0.998–1.218) | .054 | ||
| log2 URBC (/μl) | 0.294 | <.001 | 1.390 (1.236–1.563) | <.001 | 1.182 (1.031–1.265) | .019 |
| uCast | 0.283 | <.001 | 3.186 (1.849–5.490) | <.001 | ||
aCIs were excluded in the multivariate analysis.
The training cohort was divided into low- and high-activity groups, according to the AI, and the differences in variables at baseline between the two groups are presented in Table 3. There were significant differences in pathological category (Fisher’s test, P < .001); the major classification in the low-activity group was class V, whereas it was proliferative LN in the high-activity group. Compared with the low-activity group, patients in the high-activity group had more impaired renal function, including higher levels of SCr, blood urea nitrogen (BUN), uric acid (UA), URBC and UPCR (P < .001). There were no differences in sex or age between the groups.
Table 3:
Clinical characteristics of low- and high-activity groups at the time of kidney biopsy
| Characteristics | Low activity (AI ≤2) (n = 71) | High activity (AI >2) (n = 164) | P-value |
|---|---|---|---|
| LN classification, n (%) | |||
| Ⅰ, Ⅱ | 4 (5.6) | 2 (1.2) | <.001 |
| Ⅲ, Ⅳ, Ⅲ/Ⅳ ± Ⅴ | 10 (14.1) | 132 (80.5) | |
| Ⅴ | 32 (45.1) | 3 (1.8) | |
| Ⅵ | – | 3 (1.8) | |
| Post-treatment, n (%) | 25 (35.2) | 24 (14.6) | |
| CI, median (IQR) | 2.2 (2.0–3.0) | 2.8 (2.0–3.0) | .133 |
| Female, n (%) | 62 (87.3) | 139 (84.8) | .690 |
| Age (years), median (IQR) | 36.3 (28.0–43.0) | 35.8 (25.0–46.0) | .680 |
| BMI (kg/m2), median (IQR) | 22.6 (19.9–24.3) | 22.0 (19.7–23.7) | .414 |
| MAP (mmHg), mean ± SD | 95.7 ± 13.2 | 103.8 ± 13.2 | <.001 |
| WBC count (× 109/l), median (IQR) | 5.9 (3.6–7.1) | 5.9 (3.5–7.4) | .600 |
| Hb (g/l), median (IQR) | 115.2 (102.0–127.0) | 98.7 (84.3–109.8) | <.001 |
| Hct (%), median (IQR) | 35.4 (31.9–38.1) | 30.1 (25.6–33.4) | <.001 |
| Plt (× 109/l), median (IQR) | 210.9 (166.0–251.0) | 174.7 (117.5–214.5) | <.001 |
| Serum albumin (g/l), median (IQR) | 30.2 (24.0–34.4) | 26.5 (22.6–30.5) | <.001 |
| SCr (μmol/l), median (IQR) | 71.0 (50.0–70.0) | 121.5 (65.0–133.3) | <.001 |
| BUN (mmol/l), median (IQR) | 5.9 (4.1–6.7) | 10.2 (5.6–13.9) | <.001 |
| UA (μmol/l), median (IQR) | 345.4 (285.0–391.0) | 421.8 (332.1–502.8) | <.001 |
| eGFR (ml/min/1.73 m2), median (IQR) | 107.6 (98.1–124.2) | 75.7 (44.9–108.7) | <.001 |
| ESR (mm/h), median (IQR) | 38.7 (16.0–62.0) | 36.8 (18.0–49.8) | .767 |
| C3 (mg/dl), median (IQR) | 69.8 (50.0–86.0) | 45.0 (33.0–55.8) | <.001 |
| C4 (mg/dl), median (IQR) | 10 (5–17) | 6 (3–11) | <.001 |
| CRP (mg/l), median (IQR) | 3.7 (1.0–3.2) | 5.2 (1.5–5.1) | .007 |
| logANA, median (IQR) | 2.3 (2.2–2.5) | 2.3 (2.2–2.5) | .617 |
| UPCR (g/g), median (IQR) | 3.20 (1.02–4.12) | 3.60 (1.85–4.48) | .032 |
| log2 URBC/(μl), mean ± SD | 4.87 ± 2.1 | 6.69 ± 2.3 | <.001 |
| Anti-dsDNA seropositive, n (%) | 25 (35.2) | 112 (68.3) | <.001 |
| uCast positive, n (%) | 14 (19.7) | 77 (47.0) | <.001 |
Development of prediction models
The correlation coefficients between the variables screened by univariate analysis are presented in Supplementary Fig. S4 and the collinearity diagnostics of these variables were performed before the logistic regression analysis (Supplementary Table S1). A logistic model was established based on MAP, serum albumin, haematocrit, eGFR, URBC, UPCR, C3 and anti-dsDNA (AIC = 185.15). The AUC of the logistic model was 0.902 (95% CI 0.852–0.952, P = .025). This model could identify patients whose AI was >2 with a specificity of 80.3% (risk score >0.562) (Supplementary Fig. S5a and Tables S2 and S3).
We have to choose a threshold to select the variables based on the logistic regression analysis model. This process could be arbitrary. However, LASSO analysis can select important variables by penalizing the absolute value of the regression coefficient and removing non-influential covariates. Therefore, we further applied the LASSO regression coefficients to construct the prediction model (λ = lambda.1se). The LASSO model comprised seven parameters, which were similar to those in the logistic model, excluding the UPCR (AIC = 186.3). This model also showed good discrimination [AUC = 0.896 (95% CI 0.848–0.949), P = 0.026], with a specificity of 84.5% (risk score >0.644) (Fig. 1a and Supplementary Tables S4 and 5). The calibration curve of two models in the training cohort is shown in Supplementary Fig. S5b and Fig. 1b, respectively.
Figure 1:
ROC curves and calibration curves of the LASSO model in the training cohort. (a) ROC curves. (b) Calibration curves. Grey line: nomogram AI = observed AI; blue line: actual calibration; red line: adjusted curves with bootstrapping samples (1000 repetitions).
Validation of prediction models
The cross-validation was developed by 5-fold cross-validation (Supplementary Fig. S8 and Fig. 2) and the cvAUCs of the logistic and LASSO models were 0.855 and 0.896, respectively, indicating that the generalization capability of the LASSO model was superior to that of the logistic model. The performances of the two models in the testing cohort were evaluated using the ROC curves and calibration curves (Supplementary Fig. S9 and Fig. 3). Both the logistic and LASSO models indicated satisfactory discriminatory abilities [AUC = 0.830 (95% CI 0.747–0.914), P = .043; AUC = 0.831 (95% CI 0.748–0.914), P = .042]. Our complementary analyses revealed that both models presented ideal discrimination in patients with normal C3 concentrations or anti-dsDNA seronegativity or both in the testing cohort. However, the small sample size was not statistically significant; notably, both models showed a specificity of >80% in different clinical contexts (Supplementary Tables S2–5).
Figure 2:
Cross-validation of the LASSO model.
Figure 3:
Validation of the LASSO model in the testing cohort. (a) ROC curves. Subgroup analysis: the discrimination of the LASSO model in the testing cohort of patients with normal C3 concentration and/or anti-dsDNA seronegative. (b) Calibration curves. Grey line: nomogram AI = observed AI; blue line: actual calibration; red line: adjusted curves with bootstrapping samples (1000 repetitions).
Constructing a nomogram for individual assessment
The purpose of the predictive models developed in this study is to assist physicians in making clinical decisions, including performing kidney biopsy or avoiding unnecessary punctures, especially for non-specialists. Therefore, the model results should be fairly objective. We prefer using the LASSO model, as this method can be used for both variable selection and regularization, meanwhile, it has higher specificity and generalization power than the logistic model. A nomogram was constructed for individual assessment based on the regression equation of the LASSO model (Fig. 4). And an initial version of a simple online webtool called ‘AI calculator’ was built (https://ln-ai-calculator.com).
Figure 4:
A nomograms of the LASSO model.
The curves for the specificity and negative predictive value (NPV) of the LASSO model at different thresholds are presented in Fig. 5. Physicians can choose different thresholds depending on the purpose of the application. The cut-off can be adjusted to achieve 90% specificity (risk score >0.702) if the purpose is to perform a biopsy or strengthen immunosuppressive therapy and disease surveillance. If the goal is to avoid an unnecessary kidney biopsy, the threshold is adjusted to achieve a 90.0% NPV (risk score <0.507), but this reduced the specificity to 50.7%.
Figure 5:
The curves of the performance metrics of the LASSO models at different thresholds. sp: specificity; se: sensitivity; PPV: positive predictive value; NPV: negative predictive value.
To show the application of the nomogram more clearly, we provide the specific applications of two cases that both have some abnormal clinical parameters (Supplementary Table S6–5 and Figs. S12 and S14). Case 1 was a patient having atypical clinical manifestations (normal C3 and anti-dsDNA seronegative) with high renal histologic AI. Case 2 presented a clinical scenario in which a kidney biopsy could be avoided based on the result of the predictive model. The pathological features of these two patients are also provided in Supplementary Figs. S13 and S15.
DISCUSSION
In this study we developed and validated two models to determine the degree of histologic AI. Both models performed well in terms of their discriminatory ability and clinical utility. The LASSO model was further presented as a nomogram for individual assessment. This nomogram can be used as a quantitative auxiliary tool to predict AI in comprehensive clinical assessments and assist physicians in making clinical decisions. It is possible to identify patients with atypical clinical manifestations of high histologic activity and vice versa. Moreover, it is a potentially feasible tool for reducing consistency in evidence-based medical studies related to lupus.
The clinical manifestations of SLE involving the kidney are highly variable and because of the low correlation between laboratory tests and histopathology, there is always uncertainty regarding when or whether LN recovers [16–19]. Specific criteria and quantitative evaluation tools for repeat kidney biopsy have been lacking until now. Moreover, repeat kidney biopsy has the decisive advantage of estimating activity and classification shifts, but the optimal timing of repeat kidney biopsy has been controversial [20]. The International Society of Nephrology issued a modified AI and the KDIGO affirmed its significance; however, there is still a lack of guidance on the practical application of AI. Marcel et al. [7] found that in patients with LN, those with complete clinical renal remission at least 12 months but with an AI >2 relapsed within 24 months after discontinuing immunosuppression. Histologic activity upon cessation of immunosuppressive therapy is a major risk factor for LN recurrence [7]. Parodis et al. [8] found that an AI >2 in a repeat kidney biopsy after 24 months of maintenance treatment was significantly associated with LN recurrence. Alsuwaida et al. [6] reported that the 10-year renal survival rate of patients with LN was only 44%, which was significantly lower than that in patients with an AI ≤2 (100% and 80%) . Based on the above research results, we believe that LN patient with an AI >2 have a high rate of nephritic flares even after achieving complete clinical remission and a poor long-term renal prognosis, and their immunosuppressive regimens should be adjusted depending on renal histopathological manifestations. Our study found significant differences in most laboratory tests between the high- and low-activity groups, and similar CRRs but different treatments in the two groups. This suggests that the management of LN patients with an AI >2 should pay more attention to histopathological features.
Multivariate regression analysis suggested that MAP, eGFR, C3, URBC and anti-dsDNA were independent risk factors for histologic activity in LN. The anti-dsDNA titre is more suitable as a continuous variable than a dichotomous variable for developing a prediction model. But in this retrospective cohort, specific values of anti-dsDNA titres from the hospitals involved were not available before 2020. It has been reported that up to 74% of patients with SLE have elevated arterial blood pressure (BP) and are at a significantly higher risk of cardiovascular events [21], and hypertension is one of the major risk factors for poor renal prognosis in LN [22, 23]. BP fluctuates considerably in patients with LN, primarily because of systemic inflammation, glucocorticoids and renal involvement. The pathophysiology of hypertension in LN includes an impaired BP–natriuresis relationship due to renal vasoconstriction, alterations in the function of renal tubular and vascular endothelial cells and activation of the renin–angiotensin system. The incidence of hypertension increases the severity of renal histologic activity and the loss of renal function [21, 24]. Considering the numerous factors influencing BP and its complex pathophysiological mechanisms, persistent exposure to hypertension (standard measurement) is more reliable than a single BP.
Male patients with SLE have a higher incidence of nephritis than females and they are more likely to develop renal failure [25]; these differences are more pronounced in Caucasians than in African Americans [26]. However, a survey that included 1790 patients from China with SLE showed no sex differences in renal involvement [27]. In our study cohort, there was no statistically significant relationship between sex and AI. Both racial differences and our small cohort size may account for these statistically insignificant differences.
Evaluation of haematuria is commonly used in the assessment of glomerular disease, but studies on the role of haematuria in monitoring and predicting adverse outcomes have shown inconsistent results, possibly due to the failure to consider dysmorphic RBCs or RBC casts [28]. The occurrence of haematuria patients with SLE is highly correlated with short-term renal and non-renal disease activity [29, 30]. In a Mexican study cohort, the number of erythrocytes and acanthocytes in urine collected before biopsy positively correlated with the AI and CI, showing good discriminatory ability for detecting proliferative LN [31]. However, according to long-term data from the Euro-Lupus Nephritis Trial, the diagnostic accuracy and sensitivity for good long-term renal outcomes were undermined by the addition of URBCs (≤5/HFP) to proteinuria as a composite predictor [32]. In this study, we analysed the association between AI and URBC in routine urine tests. Before developing the prediction model, we found that the coefficient of variation of URBC was 3.79, which was much higher than that of other clinical variables; thus it was log2-transformed to reduce the dispersion. Although the results of the present study suggest that URBC is strongly associated with high histologic activity, the reliability remains questionable, owing to the lack of analysis of erythrocyte morphology. However, it should be emphasized that haematuria can provide valuable information to clinicians when confirming its glomerular nature.
Proteinuria remains a widely used index for evaluating treatment responses. In this study, UPCR positively correlated with the AI (r = 0.138, P = .011) but not with high histologic activity [OR 1.103 (95% CI 0.998−1.218), P = 0.054], and UPCR was excluded when constructing the prediction model using LASSO analysis. One possible reason is that the pathophysiological mechanism of proteinuria in LN is complex, whereas the AI criteria only cover partial features. Another reason may be the basis of grouping; we divided the training cohort into two groups using the AI with a cut-off value of 2 based on the results of previous studies. Notably, proteinuria remains an important indicator for the management of patients with LN, as it is a strong prognostic factor [33]. The renal prognosis of patients with a >50% reduction in proteinuria after 6 months of treatment and proteinuria <0.7 g/24 hours after 12 months of treatment was good [34, 35].
This study has some limitations. The study population included patients who underwent kidney biopsy and pathological diagnosis of LN, no patient underwent procedural biopsy, and we focused only on the clinical features at the time of the puncture, including patients with a variety of clinical conditions, such as those with an unclear diagnosis, those with a multiyear history of SLE and those during follow-up. The course of the disease and progression of fibrosis are unknown, even in patients with a suspected diagnosis of LN at the initial visit. A sufficiently large sample size containing multiple clinical scenarios may improve the predictive power of the model. The total AI score was 24, but in fact, the maximum AI of the participants in this study was 17, and very few cases had an AI >12. Moreover, a cut-off value of 2 was used to classify the low and high histologic activity groups based on the results of previous studies. Whether other thresholds are clinically significant needs to be confirmed by large-sample studies with long-term follow-up. According to Parodis et al. [8], both an AI >2 and >3 predict nephritic relapse in proliferative LN.
Additionally, reassessment of the AI in this study was based on pathological reports. Ideally, the re-evaluation of histopathological specimens would be an optimal protocol. But there are practical difficulties in re-evaluating the slides of this retrospective cohort, such as the loss of information from slides stored for long periods and the lack of sufficient tissue samples for re-evaluation in some cases. Furthermore, the original pathology of all the participants in this study was independently evaluated by two pathologists and discussed with supervising clinicians, after which a standardized, uniformly formatted formal report containing sufficient quantitative information for reassessing the AI was issued. Taking all these into consideration, after consulting with the pathologists, it was decided that re-evaluate the AI based on the written reports by two pathologists. In addition, the chronic features and other special pathophysiology, such as thrombotic microangiopathy (TMA) and anti-phospholipid syndrome nephropathy (APSN), were not discussed because of the lack of long-term prognostic renal outcomes and no patient was diagnosed with TMA and APSN in this study. A CI >3 was shown to be associated with poor long-term outcomes in small sample studies [8, 11]. The co-occurrence of TMA and APSN also negatively affects the renal prognosis of patients with LN [36]. Li et al. [37] found that TMA was associated with more severe clinical and histologic activity in LN, and the AI and CI were higher in LN patients with renal TMA than in LN patients without TMA. This is consistent with the conclusions of Song et al. [38]. In another study, a similar phenomenon was observed in which SLE patients with APSN had higher AIs and CIs than those without APSN [39]. Finally, all study populations were from China, thus the application of the nomogram is limited, owing to ethnic differences in SLE/LN.
The prediction models in this study were not validated in an external independent cohort but in a testing set randomly drawn from the original cohort. The source of the study samples was limited and the independence of the validation cohort was weak. But our research provides a new perspective for estimating the degree of renal histologic activity in patients with LN, thereby assisting in making clinical decisions. We have built an initial version of the web calculator for the AI and plan to optimize it in subsequent studies by collecting more information, including test results such as anti-dsDNA titres, URBCs of different morphology and novel biomarkers such as genes and metabolites. In addition to the AI, other important pathological features such as chronicity, TMA and catastrophic APSN will also be taken into account. A variety of clinical scenarios could be categorized and analysed, such as the diagnosis of patients with suspected LN, repeat kidney biopsies to evaluate disease activity and procedural biopsies. Moreover, in website applications, the predictive model can be updated through continuous training using increasing amounts of new data.
CONCLUSION
In this study we developed a nomogram to evaluate the renal histological AI in patients with LN, based on blood and urine laboratory tests. This model is comparatively accurate and quantitative in distinguishing patients with a high histologic AI and possesses good external applicability. Additionally, a web tool based on this nomogram for individual evaluation could be used as an auxiliary tool in comprehensive assessment to help physicians make clinical decisions.
Supplementary Material
ACKNOWLEDGEMENTS
This study was approved by the Ethics Committee of the First Affiliated Hospital, Zhejiang University School of Medicine, Zhejiang, Hangzhou, China (ethical approval IIT2020571).
Contributor Information
Cui Gao, Department of Nephrology, Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China.
Xueyan Bian, Department of Nephrology, First Affiliated Hospital of Ningbo University, Ningbo, Zhejiang, China.
Longlong Wu, Department of Nephrology, Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China.
Qian Zhan, Department of Nephrology, Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China.
Fengfei Yu, Department of Nephrology, Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China.
Hong Pan, Department of Nephrology, Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China.
Fei Han, Kidney Disease Center, First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, Zhejiang, China.
Yong-Fei Wang, School of Medicine and Warshel Institute for Computational Biology, Chinese University of Hong Kong, Shenzhen, Guangdong, China; Department of Paediatrics and Adolescent Medicine, University of Hong Kong, Hong Kong, China.
Yi Yang, Department of Nephrology, Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China; International Institutes of Medicine, Zhejiang University, Yiwu, Zhejiang, China.
FUNDING
This study was supported by the National Nature Science Foundation of China (82170681 and 81670621 to Y.Y.), the Key R&D Program of Zhejiang Province in China (grant 2020C03034 to F.H.). Y.-F.W. thanks the research start-up fund of the Chinese University of Hong Kong, Shenzhen (CUHK-SZ; UDF01002831/K10120220256) and support from Shenzhen-Hong Kong Cooperation Zone for Technology and Innovation (HZQB-KCZYB-2020056), CUHK-SZ Futian Biomedical Innovation R&D Center.
AUTHORS’ CONTRIBUTIONS
Y.Y. was responsible for the idea of the study. C.G., X.B., L.W. and F.Y. were responsible for the collection and management of data. C.G., X.B., L.W. and Q.Z. were responsible for the statistical analysis and drafted the manuscript. Y.Y., Y.-F.W. and F.H. supervised the project and wrote the final version of the manuscript. All the authors contributed to the planning, design and implementation of this study and approved the final version of the manuscript.
DATA AVAILABILITY STATEMENT
The data underlying this article are available in the article.
CONFLICT OF INTEREST STATEMENT
The authors declare no conflicts of interest.
REFERENCES
- 1. Mahajan A, Amelio J, Gairy K et al. Systemic lupus erythematosus, lupus nephritis and end-stage renal disease: a pragmatic review mapping disease severity and progression. Lupus 2020;29:1011–20. 10.1177/0961203320932219 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Anders H-J, Saxena R, Zhao M-h et al. Lupus nephritis. Nat Rev Dis Primers 2020;6:7. 10.1038/s41572-019-0141-9 [DOI] [PubMed] [Google Scholar]
- 3. Austin HA 3rd, Muenz LR, Joyce KM et al. Diffuse proliferative lupus nephritis: identification of specific pathologic features affecting renal outcome. Kidney Int 1984;25:689–95. 10.1038/ki.1984.75 [DOI] [PubMed] [Google Scholar]
- 4. Bajema IM, Wilhelmus S, Alpers CE et al. Revision of the International Society of Nephrology/Renal Pathology Society classification for lupus nephritis: clarification of definitions, and modified National Institutes of Health activity and chronicity indices. Kidney Int 2018;93:789–96. 10.1016/j.kint.2017.11.023 [DOI] [PubMed] [Google Scholar]
- 5. Kidney Disease: Improving Global Outcomes Glomerular Diseases Work Group . KDIGO 2021 clinical practice guideline for the management of glomerular diseases. Kidney Int 2021;100(4 Suppl):S1–276. [DOI] [PubMed] [Google Scholar]
- 6. Alsuwaida A, Husain S, Alghonaim M et al. Strategy for second kidney biopsy in patients with lupus nephritis. Nephrol Dial Transplant 2012;27:1472–8. 10.1093/ndt/gfr517 [DOI] [PubMed] [Google Scholar]
- 7. De Rosa M, Azzato F, Toblli JE et al. A prospective observational cohort study highlights kidney biopsy findings of lupus nephritis patients in remission who flare following withdrawal of maintenance therapy. Kidney Int 2018;94:788–94. 10.1016/j.kint.2018.05.021 [DOI] [PubMed] [Google Scholar]
- 8. Parodis I, Adamichou C, Aydin S et al. Per-protocol repeat kidney biopsy portends relapse and long-term outcome in incident cases of proliferative lupus nephritis. Rheumatology (Oxford) 2020;59:3424–34. 10.1093/rheumatology/keaa129 [DOI] [PubMed] [Google Scholar]
- 9. Tao J, Wang H, Wang SX et al. The predictive value of crescents in the disease progression of lupus nephritis based on the 2018 International Society of Nephrology/Renal Pathology Society Revision System: a large cohort study from China. Ren Fail 2020;42:166–72. 10.1080/0886022X.2020.1726385 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Tao J, Wang H, Yu XJ et al. A validation of the 2018 revision of International Society of Nephrology/Renal Pathology Society Classification for Lupus Nephritis: a cohort study from China. Am J Nephrol 2020;51:483–92. 10.1159/000507213 [DOI] [PubMed] [Google Scholar]
- 11. Malvar A, Pirruccio P, Alberton V et al. Histologic versus clinical remission in proliferative lupus nephritis. Nephrol Dial Transplant 2017;32:1338–44. 10.1093/ndt/gfv296 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Zickert A, Sundelin B, Svenungsson E et al. Role of early repeated renal biopsies in lupus nephritis. Lupus Sci Med 2014;1:e000018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Alvarado AS, Malvar A, Lococo B et al. The value of repeat kidney biopsy in quiescent Argentinian lupus nephritis patients. Lupus 2014;23:840–7. 10.1177/0961203313518625 [DOI] [PubMed] [Google Scholar]
- 14. Pakozdi A, Pyne D, Sheaff M et al. Utility of a repeat renal biopsy in lupus nephritis: a single centre experience. Nephrol Dial Transplant 2018;33:507–13. 10.1093/ndt/gfx019 [DOI] [PubMed] [Google Scholar]
- 15. Inker LA, Schmid CH, Tighiouart H et al. Estimating glomerular filtration rate from serum creatinine and cystatin C. N Engl J Med 2012;367:20–9. 10.1056/NEJMoa1114248 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Jayne D, Bajema IM. “In my beginning is my end”: usefulness of repeat kidney biopsies in lupus nephritis. Kidney Int 2020;97:27–9. 10.1016/j.kint.2019.10.002 [DOI] [PubMed] [Google Scholar]
- 17. Ayoub I, Cassol C, Almaani S et al. The kidney biopsy in systemic lupus erythematosus: a view of the past and a vision of the future. Adv Chronic Kidney Dis 2019;26:360–8. 10.1053/j.ackd.2019.08.015 [DOI] [PubMed] [Google Scholar]
- 18. Zabaleta-Lanz ME, Munoz LE, Tapanes FJ et al. Further description of early clinically silent lupus nephritis. Lupus 2006;15:845–51. 10.1177/0961203306070002 [DOI] [PubMed] [Google Scholar]
- 19. Hanly JG, O'Keeffe AG, Su L et al. The frequency and outcome of lupus nephritis: results from an international inception cohort study. Rheumatology (Oxford) 2016;55:252–62. 10.1093/rheumatology/kev311 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Moroni G, Depetri F, Ponticelli C. Lupus nephritis: when and how often to biopsy and what does it mean? J Autoimmun 2016;74:27–40. 10.1016/j.jaut.2016.06.006 [DOI] [PubMed] [Google Scholar]
- 21. Tselios K, Koumaras C, Urowitz MB et al. Do current arterial hypertension treatment guidelines apply to systemic lupus erythematosus patients? A critical appraisal. Semin Arthritis Rheum 2014;43:521–5. 10.1016/j.semarthrit.2013.07.007 [DOI] [PubMed] [Google Scholar]
- 22. Momtaz M, Fayed A, Wadie M et al. Retrospective analysis of nephritis response and renal outcome in a cohort of 928 Egyptian lupus nephritis patients: a university hospital experience. Lupus 2017;26:1564–70. 10.1177/0961203317716320 [DOI] [PubMed] [Google Scholar]
- 23. Mahmoud GA, Zayed HS, Ghoniem SA. Renal outcomes among Egyptian lupus nephritis patients: a retrospective analysis of 135 cases from a single centre. Lupus 2015;24:331–8. 10.1177/0961203314567751 [DOI] [PubMed] [Google Scholar]
- 24. Shaharir SS, Mustafar R, Mohd R et al. Persistent hypertension in lupus nephritis and the associated risk factors. Clin Rheumatol 2015;34:93–7. 10.1007/s10067-014-2802-0 [DOI] [PubMed] [Google Scholar]
- 25. Parikh SV, Almaani S, Brodsky S et al. Update on lupus nephritis: Core Curriculum 2020. Am J Kidney Dis 2020;76:265–81. 10.1053/j.ajkd.2019.10.017 [DOI] [PubMed] [Google Scholar]
- 26. Tan TC, Fang H, Magder LS et al. Differences between male and female systemic lupus erythematosus in a multiethnic population. J Rheumatol 2012;39:759–69. 10.3899/jrheum.111061 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Feng J-B, Ni J-D, Yao X et al. Gender and age influence on clinical and laboratory features in Chinese patients with systemic lupus erythematosus: 1,790 cases. Rheumatol Int 2010;30:1017–23. 10.1007/s00296-009-1087-0 [DOI] [PubMed] [Google Scholar]
- 28. Saha MK, Massicotte-Azarniouch D, Reynolds ML et al. Glomerular hematuria and the utility of urine microscopy: a review. Am J Kidney Dis 2022;80:383–92. 10.1053/j.ajkd.2022.02.022 [DOI] [PubMed] [Google Scholar]
- 29. Ding JYC, Ibanez D, Gladman DD et al. Isolated hematuria and sterile pyuria may indicate systemic lupus erythematosus activity. J Rheumatol 2015;42:437–40. 10.3899/jrheum.140415 [DOI] [PubMed] [Google Scholar]
- 30. Rahman P, Gladman DD, Ibanez D et al. Significance of isolated hematuria and isolated pyuria in systemic lupus erythematosus. Lupus 2001;10:418–23. 10.1191/096120301678646164 [DOI] [PubMed] [Google Scholar]
- 31. Martínez-Martínez MU, Llamazares-Azuara LM, Martínez-Galla D et al. Urinary sediment suggests lupus nephritis histology. Lupus 2017;26:580–7. 10.1177/0961203316669241 [DOI] [PubMed] [Google Scholar]
- 32. Dall'Era M, Cisternas MG, Smilek DE et al. Predictors of long-term renal outcome in lupus nephritis trials: lessons learned from the Euro-Lupus Nephritis cohort. Arthritis Rheumatol 2015;67:1305–13. 10.1002/art.39026 [DOI] [PubMed] [Google Scholar]
- 33. Kostopoulou M, Pitsigavdaki S, Bertsias G. Lupus nephritis: improving treatment options. Drugs 2022;82:735–48. 10.1007/s40265-022-01715-1 [DOI] [PubMed] [Google Scholar]
- 34. Korbet SM, Lewis EJ. Severe lupus nephritis: the predictive value of a ≥50% reduction in proteinuria at 6 months. Nephrol Dial Transplant 2013;28:2313–8. 10.1093/ndt/gft201 [DOI] [PubMed] [Google Scholar]
- 35. Tamirou F, Lauwerys BR, Dall'Era M et al. A proteinuria cut-off level of 0.7 g/day after 12 months of treatment best predicts long-term renal outcome in lupus nephritis: data from the MAINTAIN Nephritis Trial. Lupus Sci Med 2015;2:e000123. 10.1136/lupus-2015-000123 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Kotzen ES, Roy S, Jain K. Antiphospholipid syndrome nephropathy and other thrombotic microangiopathies among patients with systemic lupus erythematosus. Adv Chronic Kidney Dis 2019;26:376–86. 10.1053/j.ackd.2019.08.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Chao L, Desmond YHY, Gavin C et al. Clinical outcomes and clinico-pathological correlations in lupus nephritis with kidney biopsy showing thrombotic microangiopathy. J Rheumatol 2019;46:1478–84. 10.3899/jrheum.180773 [DOI] [PubMed] [Google Scholar]
- 38. Song D, Wu LH, Wang FM et al. The spectrum of renal thrombotic microangiopathy in lupus nephritis. Arthritis Res Ther 2013;15:R12. 10.1186/ar4142 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Cheunsuchon B, Rungkaew P, Chawanasuntorapoj R et al. Prevalence and clinicopathologic findings of antiphospholipid syndrome nephropathy in Thai systemic lupus erythematosus patients who underwent renal biopsies. Nephrology 2007;12:474–80. 10.1111/j.1440-1797.2007.00792.x [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data underlying this article are available in the article.