The Improved Kidney Risk Score in ANCA-Associated Vasculitis for Clinical Practice and Trials

Visual Abstract

Abstract

Significance Statement

Reliable prediction tools are needed to personalize treatment in ANCA-associated GN. More than 1500 patients were collated in an international longitudinal study to revise the ANCA kidney risk score. The score showed satisfactory performance, mimicking the original study (Harrell's C=0.779). In the development cohort of 959 patients, no additional parameters aiding the tool were detected, but replacing the GFR with creatinine identified an additional cutoff. The parameter interstitial fibrosis and tubular atrophy was modified to allow wider access, risk points were reweighted, and a fourth risk group was created, improving predictive ability (C=0.831). In the validation, the new model performed similarly well with excellent calibration and discrimination (n=480, C=0.821). The revised score optimizes prognostication for clinical practice and trials.

Background

Reliable prediction tools are needed to personalize treatment in ANCA-associated GN. A retrospective international longitudinal cohort was collated to revise the ANCA renal risk score.

Methods

The primary end point was ESKD with patients censored at last follow-up. Cox proportional hazards were used to reweight risk factors. Kaplan–Meier curves, Harrell's C statistic, receiver operating characteristics, and calibration plots were used to assess model performance.

Results

Of 1591 patients, 1439 were included in the final analyses, 2:1 randomly allocated per center to development and validation cohorts (52% male, median age 64 years). In the development cohort (n=959), the ANCA renal risk score was validated and calibrated, and parameters were reinvestigated modifying interstitial fibrosis and tubular atrophy allowing semiquantitative reporting. An additional cutoff for kidney function (K) was identified, and serum creatinine replaced GFR (K0: <250 µmol/L=0, K1: 250–450 µmol/L=4, K2: >450 µmol/L=11 points). The risk points for the percentage of normal glomeruli (N) and interstitial fibrosis and tubular atrophy (T) were reweighted (N0: >25%=0, N1: 10%–25%=4, N2: <10%=7, T0: none/mild or <25%=0, T1: ≥ mild-moderate or ≥25%=3 points), and four risk groups created: low (0–4 points), moderate (5–11), high (12–18), and very high (21). Discrimination was C=0.831, and the 3-year kidney survival was 96%, 79%, 54%, and 19%, respectively. The revised score performed similarly well in the validation cohort with excellent calibration and discrimination (n=480, C=0.821).

Conclusions

The updated score optimizes clinicopathologic prognostication for clinical practice and trials.

Introduction

ANCA-associated vasculitides are autoimmune disorders characterized by inflammation of small blood vessels resulting in irreversible organ damage. Kidney involvement is common, and disease presentation is heterogeneous. Kidney failure is a major predictor for adverse patient outcomes, and the development of effective, reliable prognostication tools is crucial to enable personalized vasculitis care.

Bajema et al. highlighted in 1996 the importance of characterizing and quantifying histopathological lesions to predict kidney outcomes in vasculitis.¹ Cohort studies detected the importance of unaffected, normal glomeruli, alongside tubular atrophy and interstitial fibrosis, as prognostic markers.^2–7 Thereafter, prospective analyses confirmed the percentage of normal glomeruli, the degree of tubulointerstitial changes, and arteriosclerosis as having predictive ability.^8,9 Formulas were proposed, combining kidney function at presentation with histopathological features to improve outcome prediction but were not widely adopted into clinical practice.^8,9 A chronicity index score, combining four histopathological changes of chronic inflammation (global glomerulosclerosis, sclerosed crescents, interstitial fibrosis, and tubular atrophy [IFTA]), was described in 2014.¹⁰ The chronicity index score inversely associated with dialysis independence at 4 months, but no futility threshold for therapeutic intervention was detected.

A histopathological classification for ANCA-associated GN (ANCA GN) was proposed by Berden et al. in 2010, focusing exclusively on glomerular pathology; biopsies were described as focal, crescentic, sclerotic, or mixed depending on most lesions.^11–14 The focal class demonstrated the best kidney survival, highlighting unaffected glomeruli as a most powerful predictor. Although attractive because of its simplicity, the classes lacked discriminative power to predict kidney survival, and the focus of the classification lies on distinguishing histological phenotypes.^15–20

The Mayo Clinical Chronicity Score proposed a standardized reporting of chronic changes in 2017 using the parameters global glomerulosclerosis, interstitial fibrosis, and tubular atrophy and arteriosclerosis.²¹ The score was not designed for a specific disease but as a consensus assessment for chronic damage. The Mayo Clinical Chronicity Score was validated on a single-center retrospective ANCA GN cohort demonstrating the association of chronic changes with decreased recovery of kidney function and increased risk of ESKD.²²

The ANCA renal risk score (ARRS) was developed on a prospective multicenter cohort in 2018.²³ It originated from a detailed investigation of glomerular and tubulointerstitial lesions, quantifying eight glomerular and four tubulointerstitial changes, to determine their predictive power. The proportion of normal glomeruli was superior to all combinations of lesions, and combining pathological and biochemical data increased risk prediction of ESKD further. The ARRS is an aggregate of individually weighted parameters using the percentage of normal glomeruli, the degree of IFTA in the kidney biopsy, and the eGFR (ml/min per 1.73 m²).²⁴

The ARRS was found to have excellent discrimination, and since its publication, multiple retrospective cohort studies have been performed on cumulatively over 2000 patients across three continents.^25–39 It was recently incorporated into a proposed tool to identify patients who may benefit from plasma exchange after the results of the plasma exchange and glucocorticoid dosing in the treatment of anti-neutrophil cytoplasm antibody-associated vasculitis trial.^40,41 A recent meta-analysis of 11 ARRS validations with a median number of 130 patients per study (range, 37–252) highlighted the need for larger-scale studies.³⁸

Cox proportional hazards (PHs) models allow easy comparison of discriminative ability through the use of Harrell's C.⁴² Baseline survival, critical to calibration, is rarely reported alongside prognostic models, and calibration of prognostic models therefore cannot be performed,^43–47 despite improved guidance for the development and reporting of prognostic models by the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis statement.^47,48 Calibration tests the accuracy of a model by comparing a new test result with the standard. No calibration was performed in the 11 validation studies reported in the meta-analysis.^{20,23,25–27,32,33,35,37,49,50} To widely adopt the score into practice, predictions need to be reliable and generalizable to different populations.⁵¹ Therefore, we collated a cohort of over 1500 patients from 11 referral centers and national registries on three continents with up to 20 years of follow-up data to (I) validate and calibrate the ARRS, (II) investigate histological and biochemical parameters on a larger cohort for parameter accessibility and to determine whether additional predictors could be added to the model, (III) refine risk parameters and risk groups, (IV) propose, and (V) validate and calibrate an improved score ready to assist clinicians in the management of ANCA GN and for use in a prospective trial of risk stratification.

Methods

Patient Cohort

This study was a retrospective longitudinal cohort study. Data were collected from 11 kidney referral centers and registries (Maine-Anjou Registry, Angers, France; Baltimore, USA; Bursa, Turkey; London, Manchester, Preston, Salford, United Kingdom; Mexico City, Mexico; Prague, Czechia; and the Irish and Scottish registries). Patients with biopsy-proven ANCA GN diagnosed from January 1987 to December 2021, with follow-up data recorded to February 2022, were included. Inclusion criteria were age 18 years and older, pauci-immune GN with acute and/or chronic vasculitic lesions consistent with ANCA GN on biopsy, data for the three components of the ARRS, and 12 weeks of follow-up. ANCA were detected by ELISA for myeloperoxidase (MPO) or proteinase 3 (PR3). Patients with double-positive MPO and PR3 antibodies and additional other immune-mediated kidney diseases, for example, antiglomerular basement membrane disease, IgA, and lupus nephritis were excluded. The sample size was pragmatic on the basis of all eligible patients from each center. The study was performed in accordance with the Declaration of Helsinki. Data were collected following the Caldicott principles and guidelines of the respective local ethics committees: Manchester Biobank REC 16/NW/0119, West of Scotland REC 5 20/WS/0181, St James'/Tallaght REC 019-09 List 33 (040), Angers University Hospital CE 2020/84, Johns Hopkins IRB00090103, Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán IRE-2928-19-20-1, Bursa Uludağ University Faculty of Medicine March 29, 2016, May/June 2016.

The end point was ESKD (measured by time to ESKD) and was defined as the need for RRT for at least 12 weeks and its continuation to the last follow-up assessed by the patients' clinical teams. Time zero was the date of the kidney biopsy. Censoring for the end point was at last follow-up, end of study, or death. Kidney function was measured in serum creatinine and eGFR, the latter using the CKD Epidemiology Collaboration equation without race.²⁴

Kidney Histology

Normal glomeruli were defined as glomeruli without any scarring, crescents, or fibrinoid necrosis within the tuft. Cellular crescents were defined according to the consensus of the Renal Pathology Society (as crescents with a composition of more than 75% cells and fibrin and <25% fibrous matrix⁵²). The original ARRS was calculated combining the individually weighted point scores for eGFR at presentation (G0: >15 ml/min per 1.73 m², G1: ≤15 ml/min per 1.73 m²), percentage of normal glomeruli (N0: >25%, N1: 10%–25%, N2: <10%), and the percentage of IFTA (T0: IFTA <25%, T1: IFTA ≥25%). Each parameter was assigned points as previously described (G0=0, G1=3, N0=0, N1=4, N2=6, T0=0, T1=2), and their summation produced the total score determining the allocation of patients into one of the three risk groups: low (0), moderate (2–7), and high (8–11 points).

Model Building and Statistics

Data were not imputed, and models were built on a complete-case basis because the amount of missing data (after excluded patients) was negligible (<0.01%).

Stratified by center, the cohort was randomly split in a ratio of 2:1 to create development and validation sets. First, the development cohort was used for the external validation and calibration of the original ARRS (objective I). Then, the development cohort was used to investigate the accessibility of the score parameters (objective II) and to derive alternative models refining the score (objective III). Proposing the new score (objective IV), the validation cohort was used for the validation and calibration of the new models (objective V).

Model performance was assessed for discrimination using Harrell's C statistic, a generalization of the area under the receiver operating characteristic for Cox models⁴² and receiver operating characteristic curves. Calibration was performed as advised by McLernon et al.⁵³ (objectives I and V). An expert panel of eight experienced nephrologists and nephropathologists was formed to judge the accessibility and adaptation of parameters (objective II). Multivariable models were built using Cox PH models (objective III). The panel investigated each significant parameter of the multivariable analysis for its suitability as a risk parameter, and backwards elimination was used with a threshold of P = 0.1 to select among these (objective III). An exploratory regression tree built with R Package rpart⁵⁴ (using the default parameters) was used to determine whether differing cutoffs would improve the prediction model (objective III). The cutoffs along with the clinical judgment of the panel were used to form the risk groups to maximize utility.

We generated new risk points (objective IV) for the model with categorical covariates. For this, we used the β-coefficients from the Cox models and divided them by the smallest coefficient of the model to retain proportionality. We then tripled and rounded the values to the nearest integer. The β-coefficients were tripled rather than doubled as previously to highlight the difference between the weight of normal glomeruli and IFTA.

An additional continuous model using the same parameters as the final model was derived without cutoffs (objective IV). Baseline survival equations were calculated using the methods proposed by Royston and Altman.⁴⁷ Integrated in a web application, the continuous model was designed to provide survival chances informing clinicians and patients of a detailed prognosis.

The final models were compared with the original model using the validation cohort. The difference in performance was evaluated using a paired bootstrap sample of the C statistics (1000 replicates; objective V). Time-dependent receiver operating characteristic curves were calculated for 1, 3, and 5 years using the method described by Blanche et al.⁵⁵ (objective V). The PH assumption was assessed to investigate the consistency of the risk prediction over time using the Grambsch and Therneau global test⁵⁶ and visually reviewing the Kaplan–Meier plots.⁵³ A competing risk sensitivity analysis investigating death before ESKD as a competing risk was performed using the Fine and Gray approach.⁵⁷ The validation cohort was split on either side of January 1, 2015, to assess the influence of different periods on model performance (objective V).

Analyses were performed using R v4.2.1 (R Core Team, Vienna, Austria). The study was reported according to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis guidelines.^58,59

Results

Basic Characteristics

A total of 1591 patient records were submitted by their centers, of which 1439 patients met inclusion criteria for the validation (Supplemental Figure 1A). To create development and validation sets, the 1439 patients were randomly split in a 2:1 ratio by center into two cohorts with a development cohort consisting of 959 patients and a validation cohort of 480 patients. The basic characteristics of the cohorts are depicted in Table 1. Median follow-up was 3.6 years (interquartile range, 1.1–5.9 years), 33% of patients had at least 5 years, and 8.4% had at least 10 years of follow-up (Table 2). Patient flow through the study is detailed in Supplemental Figure 1B.

Table 1.

Baseline demographic factors

Patient Cohorts	All Patients (N=1439)	Development (n=959)	Validation (n=480)
Age at diagnosis, median (IQR)	64.0 (53.0–73.0)	63.4 (52.3–72.5)	64.5 (53.9–73.2)
Male sex, n (%)	750 (52.1)	484 (50.5)	266 (55.4)
Diagnosis, n (%)
GPA	568 (39.5)	388 (40.5)	180 (37.5)
MPA	820 (57.0)	538 (56.1)	282 (58.8)
EGPA	19 (1.3)	14 (1.5)	5 (1.0)
ANCA negative PING	31 (2.2)	19 (2.0)	12 (2.5)
ANCA type, n (%)
MPO	762 (53.0)	509 (53.1)	253 (52.7)
PR3	579 (40.2)	389 (40.6)	190 (39.6)
ANCA negative	98 (6.8)	61 (6.4)	37 (7.7)
eGFR, ml/min per 1.73 m2, median (IQR)	21.0 (10.9–36.4)	21.8 (11.3–38.0)	18.0 (9.8–35.0)
Creatinine, μmol/L, median (IQR)	234.0 (150.3–402.0)	225.0 (147.0–383.5)	257.0 (157.5–449.5)
RRT at presentation, n (%)	291 (20.2)	185 (19.3)	106 (22.1)
Glomeruli on biopsy, median (IQR)	17.0 (12.0–23.0)	17.0 (12.0–23.0)	17.0 (12.0–23.2)
Percentage normal glomeruli, median (IQR)	35.5 (14.3–65.2)	36.0 (14.3–66.7)	34.7 (14.3–62.5)
Percentage crescentic glomeruli, median (IQR)	20.0 (5.9–46.4)	19.0 (5.8–44.4)	21.4 (6.3–50.0)
IFTA, n (%)
None—mild	689 (47.9)	469 (48.9)	220 (45.8)
Mild to moderate—severe	750 (52.1)	490 (51.1)	260 (54.2)
ARRS, median (IQR)	3.0 (0.0–7.0)	3.0 (0.0–7.0)	4.0 (2.0–7.0)
Risk group, n (%)
Low	370 (25.7)	258 (26.9)	112 (23.3)
Moderate	742 (51.6)	484 (50.5)	258 (53.8)
High	327 (22.7)	217 (22.6)	110 (22.9)

ARRS, ANCA renal risk score; EGPA, eosinophilic granulomatosis with polyangiitis; GPA, granulomatosis with polyangiitis; IFTA, interstitial fibrosis and tubular atrophy; MPA, microscopic polyangiitis; MPO, myeloperoxidase; PING, pauci-immune necrotizing GN; PR3, proteinase 3.

Table 2.

Clinical outcomes

Cohorts	All Patients (N=1439)	Development (n=959)	Validation (n=480)
Follow-up, yr	3.6 (1.1–5.9)	3.6 (1.1–5.9)	3.6 (1.0–6.0)
Recovery, n (%)	136/291 (46.7)	83/185 (44.9)	53/106 (50)
ESKD, n (%)	325 (22.6)	207 (21.6)	118 (24.6)
Death, n (%)	315 (21.9)	206 (21.5)	109 (22.7)
Death w/o ESKD, n (%)	199 (13.8)	135 (14.1)	64 (13.3)

Recovery, recovery of kidney function; w/o, without.

Patient Outcomes

In the development cohort, 207 patients (21.6%) progressed to ESKD compared with 118 patients (24.6%) in the validation cohort (Table 2). A total of 185 patients of the development cohort required RRT at presentation, and 83 (44.9%) of these recovered kidney function. In the validation set, 53/106 (50%) recovered kidney function. Two hundred and six (21.5%) and 109 (22.7%) patients died in the development and validation cohorts, respectively.

A total of 1114 patients (77.4%) were ESKD-free at their last follow-up (752 patients, 78.4% in the development and 362 patients, 75.4% in the validation cohort). A total of 199 (13.8%) of these ESKD-free patients died during follow-up (135 patients, 14.1% in the development cohort and 64 patients, 13.3% in the validation cohort).

Objective I: Validation and Calibration of the Original ARRS

Validation of the Original ARRS

For the validation of the original ARRS, the development cohort was restricted to biopsies reporting the degree of IFTA by percentage (n=456). The discrimination of the ARRS was 0.779 (95% confidence interval [CI], 0.739 to 0.819). The 3-year kidney survival was 75.8% (95% CI, 71.8% to 80.1%). Regarding the risk groups, the 3-year kidney survival was 93.7% (95% CI, 88.9% to 98.7%), 82.1% (95% CI, 76.7% to 87.8%), 49.6% (95% CI, 41.3% to 59.6%) in the low-risk, moderate-risk, and high-risk groups, respectively. The kidney survival of the three risk groups is depicted in Figure 1.

Figure 1.

Kidney survival according to the stratification of the original ARRS. Kaplan–Meier curve depicting the development of ESKD of patients with ANCA GN. Patients of the development cohort are assigned points as per the ARRS according to the eGFR (G0: >15 ml/min per 1.73 m², G1: ≤15 ml/min per 1.73 m²), the percentage of normal glomeruli in the kidney biopsy (N0: >25%, N1: 10–25%, N2: <10%), and the degree of IFTA (T0: <25%, T1: ≥25%). Points are calculated (G0=0, G1=3, N0=0, N1=4, N2=6, T0=0, T1=2) and risk groups created—low-risk (0), moderate-risk (2–7), and high-risk group (8–11). The number of patients at risk in each group at each time point is stated below the graph. Patient outcomes differ per risk group, C=0.779, P < 0.001. ANCA GN, ANCA-associated GN; ARRS, ANCA renal risk score; IFTA, interstitial fibrosis and tubular atrophy.

Calibration of the Original ARRS

Data from the original cohort²³ were reanalyzed to determine survival function and calibration. The predictions were based on the 115 patients in the original cohort. Supplemental Figure 2 demonstrates the calibration plots at 1-, 3-, and 5-year follow-up. Calibration was reasonable for an external cohort (year 3 slope: 0.66, intercept: 0.27); the observed survival was slightly higher than predicted.

Objective II: Accessibility and Adaptation of Risk Parameters

Risk Parameter IFTA

The reporting of IFTA is often performed in a semiquantitative manner. To allow worldwide availability of the risk score, we investigated the congruency of semiquantitative and quantitative reporting of IFTA. Supplemental Figure 3 demonstrates the correlation between quantitative and semiquantitative reporting of IFTA in the development cohort (n=269). IFTA reporting showed a satisfactory and close to perfect congruency in the low gradings of none and mild. Therefore, we retained the original cutoff of IFTA ≥25% and allowed the semiquantitative alternative of reporting with the cutoff of IFTA ≥ mild to moderate.

Revalidation of the Simplified Original ARRS

For the validation of the simplified original ARRS, the whole development cohort was used. The discrimination of the ARRS in this cohort was C=0.800 (95% CI, 0.768 to 0.832; P < 0.001), comparable with the original development cohort in 2018 (C=0.832). The 3-year kidney survival of the development cohort was 80.9% (95% CI, 78.3% to 83.6%). Regarding the risk groups, the 3-year kidney survival was 94.8% (95% CI, 91.9% to 97.9%), 86.1% (95% CI, 82.8% to 89.5%), 52.4% (95% CI, 45.7% to 60.0%) in the low-risk, moderate-risk, and high-risk groups, respectively. The kidney survival of the three risk groups is depicted in Supplemental Figure 4A.

Calibration of the Simplified Original ARRS

Supplemental Figure 4B demonstrates the calibration plots at 1-, 3-, and 5-year follow-up. Calibration was again reasonable for an external cohort (year 3 slope: 0.70, intercept: 0.28); the observed survival was slightly higher than predicted. The baseline survival for a patient with ARRS=0 is detailed in Supplemental Equation 1.

There was strong agreement in the model between the original score with percentage IFTA only and the simplified original score with all patients, demonstrating that semiquantitative grading can be used in lieu of percentage grading.

Objective III: Review of Predictors of Kidney Survival

On univariable analyses, the percentage of normal glomeruli, cellular crescents, IFTA, and the kidney function at presentation were associated with kidney outcome (Supplemental Table 1A). On further multivariable analyses, using Cox PH models, three parameters remained predictors of ESKD (Supplemental Table 1B). The logarithm of serum creatinine at presentation demonstrated a strong association with ESKD (hazard ratio [HR], 3.541; 95% CI, 2.675 to 4.687; P < 0.001), and the presence of a higher percentage of normal glomeruli associated with a reduced risk of ESKD (HR, 0.977; 95% CI, 0.967 to 0.987; P < 0.001). There was also a risk associated with more severe IFTA (HR, 1.673; 95% CI, 1.159 to 2.415; P = 0.006). Other parameters, such as the ANCA antibody subtype PR3 versus MPO, failed to show an association with outcome (HR, 0.892; 95% CI, 0.637 to 1.250; P = 0.506). While there was an association between the percentage of cellular crescents and kidney survival on the univariable analysis, there was no detectable association after adjustment for the percentage of normal glomeruli (HR, 0.996; 95% CI, 0.991 to 1.002; P = 0.175).

Objective IV: Revision and Modification of the ARRS

In the development cohort (n=959), backwards elimination with a threshold of P = 0.1 was performed, and no other clinicopathological parameters were associated with ESKD on multivariable models. Consequently, no additional parameters were incorporated into the revised ARRS.

New Revised Model: The Score with Creatinine and Four Risk Groups

In acute kidney disease, eGFR compresses a broad range of creatinine values into a smaller range of eGFR values. Taking age, sex, and ethnicity components into the equation does not improve the reliability of the measured function when there are acute changes in kidney function but reduces the discriminative power of the biochemical measures of kidney function. Therefore, a model with creatinine instead of eGFR was developed to maximize reliability.⁶⁰ On a regression tree analysis, the primary node was detected at a creatinine of 454 µmol/L. Further cutoffs were identified at 239 and 967 µmol/L, the latter was not incorporated because of the small subcohort of patients (Supplemental Figure 5). Additional cutoffs for the percentage of normal glomeruli were similar to the original tree analysis from 2018 at 7.3% and 27%, and therefore, no changes were deemed necessary. Using the above-described method, risk points were assigned as follows for creatinine: K0: <250 µmol/L=0 points; K1: 250–450 µmol/L=4 points; K2: >450 µmol/L=11 points; percentage of normal glomeruli: N0: >25%=0 points; N1: 10%–25%=4 points; N2: <10%=7 points; IFTA: T0: none/mild or <25%=0 points; T1: IFTA ≥ mild to moderate or ≥25%=3 points (Table 3). An interaction between terms was considered, but none was found. Risk groups were defined separating low (0–4), moderate (5–11), high (12–18), and very high–risk groups (21) (as depicted in Table 4; note that the risk points 1, 2, 19, 20 are not calculable). Figure 2 demonstrates the discriminative power and kidney survival of the risk groups of the new ANCA kidney risk score (AKRiS) over time. The 3-year kidney survival was 96.0% (95% CI, 94.1% to 98.0%), 79.4% (95% CI, 74.3% to 84.8%), 53.8% (95% CI, 45.0% to 64.3%), and 18.5% (95% CI, 10.8% to 31.8%) in the low-risk, moderate-risk, high-risk, and very high–risk groups, respectively (C=0.831, 95% CI, 0.801 to 0.861, P < 0.001).

Table 3.

Model parameters and performance

Variable	Original Model		Simplified Model		New AKRiS Model
	n=456 n (%)	Points	n=959 n (%)	Points	β	HR	Points
eGFR, ml/min per 1.73 m2
G0: ≥15	279 (61.2)	0	608 (63.4)	0
G1: <15	177 (38.8)	3	351 (36.6)	3
Creatinine, μmol/L
K0: <250			534 (55.7)		Ref		0
K1: 250–450			241 (25.1)		0.661	1.94 (1.30–2.89)	4
K2: >450			184 (19.2)		1.886	6.59 (4.59–9.45)	11
Normal glomeruli, %
N0: >25	245 (53.7)	0	590 (61.5)	0	Ref		0
N1: 10–25	90 (19.7)	4	177 (18.5)	4	0.650	1.92 (1.30–2.82)	4
N2: <10	121 (26.5)	6	192 (20.0)	6	1.199	3.32 (2.35–4.69)	7
IFTA, %
T0: <25	216 (47.4)	0
T1: ≥25	240 (52.6)	2
IFTA (simplified), %
T0: none, mild, <25			469 (48.9)	0	Ref		0
T1: ≥ mild-moderate, ≥25			490 (51.1)	2	0.527	1.69 (1.28–2.26)	3
C statistic (95% CI)	0.779 (0.739 to 0.819)		0.800 (0.768 to 0.832)		0.831 (0.801 to 0.861)

AKRiS, ANCA kidney risk score; CI, confidence interval; HR, hazard ratio; IFTA, interstitial fibrosis and tubular atrophy.

Table 4.

Risk groups for the ANCA kidney risk score

Group	Points
Low	0–4
Moderate	5–11
High	12–18
Very high	21

Survival function in Supplemental Equation 2. Risk points 1, 2, 19, and 20 not available.

Figure 2.

Kidney survival according to the stratification of the new AKRiS. Kaplan–Meier curve depicting the development of ESKD of patients with ANCA GN of the development (solid line) and validation cohort (dashed line). Patients are assigned points as per the AKRiS according to the serum creatinine (K0: <250 µmol/L, K1: 250–450 µmol/L, K2: >450 µmol/L), the percentage of normal glomeruli in the kidney biopsy (N0: >25%, N1: 10%–25%, N2: <10%), and the degree for IFTA (T0: none/mild or <25%, T1: ≥ mild-moderate or ≥25%). Points are calculated (K0=0, K1=4, K2=11, N0=0, N1=4, N2=7, T0=0, T1=3) and risk groups created—low (0–4), moderate (5–11), high (12–18), and very high (21). The number of patients at risk in each group at each time point is stated below the graph. Patient outcomes differ per risk group, C=0.831, P < 0.001. AKRiS, ANCA kidney risk score.

Continuous Score Model with Percentage Risk

A complementary continuous model was developed using the logarithm of creatinine, the percentage of normal glomeruli, and the IFTA categories as covariates. The model demonstrated excellent discrimination (C=0.833; 95% CI, 0.805 to 0.861; P < 0.001, Table 5). The model allowed detailed estimates of kidney survival but required significant computation. We created a webpage enabling the calculation of the risk percentage for ESKD at 1, 3, and 5 years for the individual patient at the time of presentation using the three parameters serum creatinine, the percentage of normal glomeruli, and the IFTA category. Supplemental Figure 6 has the link and QR code to access the software.

Table 5.

Continuous ANCA kidney risk score

Variables	Units	β Coefficient
Creatinine	Log, μmol/L	1.250
Normal glomeruli	Per 1%	-0.0167
IFTA	≥Mild-moderate or ≥25%	0.616
	C statistic (95% CI)	0.833 (0.805 to 0.861)

Survival function in Supplemental Equation 3. CI, confidence interval; IFTA, interstitial fibrosis and tubular atrophy; log, natural logarithm.

Objective V: Validation and Calibration of the AKRiS

Performance of the Score in the Validation Cohort

The discrimination of the AKRiS in the validation cohort was C=0.821 compared with C=0.783 for the original ARRS (difference=0.038; 95% CI, 0.012 to 0.064; P = 0.003). The time-dependent area under the curves for the AKRiS and the original ARRS were calculated. These showed sustained and consistently improved performance (Supplemental Figure 7). The area under the curves was 0.844 versus 0.803 (P = 0.026), 0.839 versus 0.812 (P = 0.172), and 0.820 versus 0.777 (P = 0.051) for 1, 3, and 5 years, respectively.

The 3-year kidney survival of the validation cohort was 76.6% (95% CI, 72.6% to 80.7%). In regard to the risk groups of the AKRiS, the 3-year kidney survival was 96.5% (95% CI, 94.0% to 99.1%), 75.1% (95% CI, 67.6% to 83.4%), 49.3% (95% CI, 39.4% to 61.8%), and 12.0% (95% CI, 4.2% to 34.7%) in the low-risk, moderate-risk, high-risk, and very high–risk groups, respectively (C=0.821; 95% CI, 0.744 to 0.846; P < 0.001). In the calibration of the new models, there was strong agreement between the expected and the observed events in the validation cohort (year 3 slope: 0.985, intercept: 0.008). Supplemental Figure 8B demonstrates the calibration plots for the New and the Continuous AKRiS models at 1, 3, and 5 years of follow-up.

The PHs assumption was assessed, and the Grambsch and Therneau global test^53,56 detected disproportionality because of the nature of the rapidly progressive disease (P < 0.001). The nonproportionality demonstrated minor violations with robust predictions over time when inspected on the calibration plots.

Effects of Competing Risks and the Time of Presentation on the Performance of the AKRiS

In a sensitivity analysis, we used the Fine–Gray⁵⁷ method with death before ESKD as a competing risk. The AKRiS retained performance and calibration in the validation cohort when applied to a competing-risks setting (C=0.817 [0.787 to 0.847], Supplemental Figure 9). We investigated the influence of the score on patient survival, and the model did not predict death (C=0.569).

The model was then tested on a temporal split by the presentation date of the validation cohort on January 1, 2015. The discrimination of the score was higher in patients presenting in the more recent period with Harrell's C_≥2015=0.868 (95% CI, 0.828 to 0.908) when compared with the period before 2015 with Harrell's C_<2015=0.782 (95% CI, 0.722 to 0.848; P = 0.008, Supplemental Table 2).

Discussion

Kidney and patient outcomes remain unsatisfactory in ANCA GN because of heterogeneity in disease presentation and treatment-related adverse events, with infection being the most common cause of death in the first year. Immunosuppressive therapy needs to be tailored to the individual patient to avoid treatment failure and minimize toxicity. Reliable prediction tools are urgently required to guide clinicians in treatment decisions. Consensus on relative risk will also facilitate better outcome assessment in interventional trials through stratification of patients. Oncology has led the way to personalized therapy and established rigorous disease staging and patient fitness assessments to optimize treatment outcomes. There is a clear and urgent unmet need to personalize medicine in ANCA GN.

The ARRS was published in 2018 categorizing patients in risk groups according to histological and clinical parameters, discriminating kidney survival.²³ Validations of over 2000 patients on three continents demonstrated the reliability of the prediction tool. Here, we collated the largest cohort of ANCA GN patients with biopsy data to date, validated, and estimated the calibration error in the ARRS and present a revised and improved score optimizing outcome prediction. The derivation and validation of the original score was missing long-term outcome data, and previous validation studies have not provided meaningful assessments of calibration error. Collating a cohort of patients from different centers and national registries around the world allowed us to observe kidney survival in patients with up to 10–15 years of follow-up and to calibrate the original score. Gathering a large number of events enabled us to investigate our original risk parameters and assess the quality of the original model, updating and further improving the tool. The next step will be to assess the utility of the prediction tool to guide treatment.

We found no additional parameters aiding the tool, but by switching from eGFR to creatinine, we identified an additional cutoff that improved discrimination. Numerous factors can interfere with laboratory measures of creatinine, but a wide range of creatinine values are comprised in fewer eGFR values diminishing the usability of eGFR in AKI. In rapidly progressive AKI, serum creatinine will take days before it reflects the true GFR, and eGFR estimations were developed in patients with stable CKD, are population-based, and even less appropriate for use in AKI. We were also able to modify the parameter IFTA to allow wider access, to reweight the risk points (by factor 3), and to create a fourth risk group (very high risk) providing more reliable predictions.

Different risk parameters for adverse outcome have been proposed in the past, and anti-MPO antibody specificity has repeatedly been shown to associate with a higher risk for ESKD. In our cohort, this effect was not significant after adjustment for other parameters. Despite the size of the cohort, anti-MPO specificity did not independently predict outcome and was not found to improve outcome prediction. Our findings would suggest the tool can be used in different populations with different basic characteristics.^25,32

Our study was limited by its retrospective design and the observation period including patients from 1987 to 2021, but no other design would have been feasible to achieve our aims of a large-scale validation with a long follow-up. We detected an improved model performance of the revised score over time, highlighting the need for future updates of the prediction tool to adjust to changes in vasculitis treatment and patient survival. We were not able to assess the effect of ethnicity and extrarenal disease and aim to investigate these in the next update. Using real-world data have the benefit of creating datasets that include complex and severely ill patients who are usually not recruited to interventional trials. Using pathological findings for our scoring, interobserver variability needed to be factored in. In ANCA GN, reporting of glomerular and tubulointerstitial lesions has demonstrated significant variability and affects the reliability of prediction tools. Here, we investigated the effect of cellular crescents on outcome prediction and did not detect any significant signal. A limitation is that we did not investigate other pathological lesions, for example, intracapillary and extracapillary fibrinoid necrosis. However, the interobserver variability of this early lesion is significant, and we believe that the effect on outcome prediction would have been diminished by its unreliable reporting. We also demonstrated the variability in IFTA reporting and the challenges of using that risk predictor. The higher degrees of IFTA exhibit substantial interobserver disagreement, making moderate-to-severe IFTA less suitable for prediction models. We showed congruency of quantitative and semiquantitative reporting of the lower degrees of IFTA; thus, the use of the semiquantitative reporting will support a wider utility of the score.

A common measure of model performance is the C statistic, and in our case Harrell's C, the extension for time-to-event models, which in development ranges from 0.5 to 1. For models to be implemented in routine clinical practice, they require strong performance and clear clinical utility and usability. While one could add an ever-increasing number of predictors into a model, it would be subject to overfitting and making the tool more difficult to use in practice. The designing of a prediction tool requires statistical and practical trade-offs. Van Royen et al. published several barriers preventing the implementation of clinical prediction models into practice,⁵¹ and we believe the AKRiS has overcome these: The score demonstrated that it is fit for purpose using routinely collected predictors on an easily measured outcome. With prospective trials incorporating the tool, there is a clear path to its widespread implementation and adoption.

Prognostic tools have failed to detect a treatment futility threshold in ANCA GN. Standard kidney biopsies encounter about 15–20 glomeruli per biopsy. This number seems unable to detect the cutoff of glomerular injury predictable of treatment failure in ANCA GN. Kidneys seem to function sufficiently to provide dialysis independence with a very small proportion of unaffected glomeruli. Additional factors such as hyperfiltration damage over time and relapsing disease affect long-term survival and are challenging to incorporate into current prediction tools. The highest-risk group demonstrated a 3-year kidney survival of 18.5% and 12% in the development and validation cohorts. Of 54 patients who required RRT at diagnosis in the very high–risk group, seven recovered kidney function (13%). The AKRiS provides a valuable risk prediction for ESKD improving the education of health care professionals and patients. Given the predictive power of the model, the ultimate aim is the prospective use of the tool to inform therapy. At present, the tool can assist the clinical assessment when discussing risks and merits of different treatment strategies.

In conclusion, the ARRS provides a strong discriminative ability, and the update improves the existing paradigm. We recommend those using the original ARRS change to the updated AKRiS using creatinine. The revised and improved kidney score will provide stratification of patients for personalized treatments and in clinical trials and will allow clinicians to give reliable estimates of prognosis.

Supplementary Material

jasn-35-335-s001.pdf ^{(1.3MB, pdf)}

Disclosures

I. Bajema reports Consultancy: Aurinia, Boehringer Ingelheim, CatBio, GSK, Hansa, Novartis, Otsuka, Toleranzia, Vera, and Vifor; Advisory or Leadership Role: C3 Glomerulopathy review board (Novartis, The Netherlands) and Glomerular Disease Council (Vifor Pharma); and Other Interests or Relationships: Director of Bajema Institute of Pathology, Past President of Renal Pathology Society, and Vice-President of European Vasculitis Society (EUVAS). S. Bate reports Employer: IQVIA and Manchester University NHS Foundation Trust. B. Brilland reports Honoraria: CSL Vifor and Sanofi. S.R. Brix reports Honoraria: Vifor; Advisory or Leadership Role: Vifor; and Speakers Bureau: Vifor. N. Brown reports Honoraria: Vifor; and Other Interests or Relationships: Chair of Education Committee, United Kingdom and Ireland Vasculitis Society. N. Dhaun reports Consultancy: Travere Pharmaceutical; Research Funding: Pfizer and Travere Pharma; and Advisory or Leadership Role: Travere. A. Dhaygude reports Honoraria: Speaker fees for AstraZenica and Advisory or Leadership Role: Vifor Pharma. G. Duruvu reports Consultancy: Amgen, Aurinia, Calliditas, ChemoCentryx, GSK, and Otsuka. D. Ilyas reports Research Funding: PhD Research Projects funded by Invizius Limited. D.R.W. Jayne reports Consultancy: AstraZeneca, Boehringer-Ingelheim, Chemocentryx, Chinook, GSK, Novartis, Otsuka, Roche/Genentech, Takeda, and Vifor; Ownership Interest: Aurinia; Research Funding: GSK, Roche/Genentech, and Vifor; Honoraria: GSK, Otsuka, and Vifor; Advisory or Leadership Role: Aurinia; and Speakers Bureau: GSK and Vifor. J.C. Jennette reports Employer: Sangamo Therapeutics, Inc. R.B. Jones reports Research Funding: GSK and Roche; Honoraria: Roche; and Advisory or Leadership Role: Advisory boards for CSL Vifor and GSK. J.S. Lees reports Honoraria: AstraZeneca. M.A. Little reports Research Funding: CSL Vifor; Patents or Royalties: Euroimmun; and Speakers Bureau: CSL Vifor. S.P. McAdoo reports Consultancy: CSL Vifor and GSK; Research Funding: AstraZeneca and Therini Bio; Honoraria: Celltrion, Rigel Pharmaceuticals, and ThermoFisher Scientific; and Advisory or Leadership Role: Co-chair United Kingdom and Ireland Vasculitis Society (unpaid). J.M. Mejía-Vilet reports Honoraria: AstraZeneca, Boehringer-Ingelheim, Novartis, and Roche. E. Nic an Riogh reports Research Funding: Health Research Board Ireland and Wellcome (203930/B/16/Z), the Health Service Executive, National Doctors Training and Planning and the Health and Social Care, Research and Development Division, Northern Ireland supports the Irish Clinical Academic Training (ICAT) Program, of which Dr. E. Nic an Riogh is a Wellcome-HRB Irish Clinical Academic Training (ICAT) Fellow. C.D. Pusey reports Consultancy: Alentis and Vifor; Honoraria: Amgen and Med Update Europe; and Advisory or Leadership Role: CJASN. K.I. Stevens reports Consultancy: Bayer, Boehringer Ingelheim, and Vifor; and Advisory or Leadership Role: ERAEUROCONGRESS Ltd. Company Director (no remuneration). V. Tesar reports Consultancy: AstraZeneca, Bayer, Boehringer-Ingelheim, Calliditas, Eli Lilly, Fresenius Medical Care, GSK, Novartis, Omeros, Otsuka, Swixx BioPharma, and Travere; Honoraria: AstraZeneca, Bayer, Boehringer-Ingelheim, Calliditas, Eli Lilly, Fresenius Medical Care, Novartis, Omeros, Swixx Biopharma, and Travere; and Advisory or Leadership Role: member of the scientific advisory board of B. Braun, Fresenius Medical Care and member of the steering committee of clinical trials sponsored by Calliditas, Novartis, Omeros, Otsuka, and Travere. T. Wiech reports Honoraria: Alexion, Bayer, Eleva, GlaxoSmithKline GmbH, and Novartis; and Advisory or Leadership Role: Eleva and Novartis. All remaining authors have nothing to disclose.

Funding

This work was supported by the Wellcome Trust Institutional Translational Partnership Award 222061/Z/20/Z to S.R. Brix.

Author Contributions

Conceptualization: Silke R. Brix.

Data curation: Jean Francois Augusto, Mehmet Fethullah Aydın, Benoit Brilland, Silke R. Brix, Nina Brown, Arslan Butt, Gavin B. Chapman, Francesca Costigliolo, Neeraj Dhaun, Ajay Dhaygude, Duruvu Geetha, Lauren Floyd, Andrea Hinojosa-Azaola, Duha Ilyas, Marek Kollar, Vojtech Kratky, Jennifer S. Lees, Mark A. Little, Eduardo Martín-Nares, Stephen P. McAdoo, Dominic McGovern, Juan Manuel Mejía-Vilet, Eithne Nic an Riogh, Charles D. Pusey, Avi Rosenberg, Jennifer Scott, Kate I. Stevens, Pek Ghe Tan, Vladimir Tesar, Abdülmecit Yildiz.

Formal analysis: Sebastian Bate, Stephen A. Roberts.

Funding acquisition: Silke R. Brix.

Investigation: Sebastian Bate, Silke R. Brix, Duruvu Geetha, Mark A. Little, Stephen P. McAdoo, Vladimir Tesar.

Methodology: Sebastian Bate, Silke R. Brix, Duruvu Geetha, Mark A. Little, Stephen P. McAdoo, Stephen A. Roberts, Vladimir Tesar.

Project administration: Sebastian Bate, Silke R. Brix.

Resources: Silke R. Brix.

Software: Sebastian Bate.

Supervision: Ingeborg Bajema, Silke R. Brix, David R.W. Jayne, J. Charles Jennette, Thorsten Wiech.

Validation: Sebastian Bate.

Visualization: Sebastian Bate.

Writing – original draft: Sebastian Bate, Silke R. Brix, Dominic McGovern.

Writing – review & editing: Jean Francois Augusto, Mehmet Fethullah Aydın, Ingeborg Bajema, Sebastian Bate, Benoit Brilland, Silke R. Brix, Nina Brown, Arslan Butt, Gavin B. Chapman, Francesca Costigliolo, Neeraj Dhaun, Ajay Dhaygude, Duruvu Geetha, Lauren Floyd, Andrea Hinojosa-Azaola, Duha Ilyas, David R.W. Jayne, J. Charles Jennette, Rachel B. Jones, Marek Kollar, Vojtech Kratky, Jennifer S. Lees, Mark A. Little, Eduardo Martín-Nares, Stephen P. McAdoo, Dominic McGovern, Juan Manuel Mejía-Vilet, Eithne Nic an Riogh, Charles D. Pusey, Stephen A. Roberts, Avi Rosenberg, Jennifer Scott, Kate I. Stevens, Pek Ghe Tan, Vladimir Tesar, Thorsten Wiech, Abdülmecit Yildiz.

Data Sharing Statement

Anonymized data created for the study are or will be available in a persistent repository on publication.

Supplemental Material

This article contains the following supplemental material online at http://links.lww.com/JSN/E562.

Supplemental Table 1A. Univariable models for ESKD.

Supplemental Table 1B. Multivariable model for ESKD.

Supplemental Table 2. Kidney survival by risk group and presentation date in the validation cohort.

Supplemental Figure 1A. Flow chart diagram showing patient recruitment to the study.

Supplemental Figure 1B. Flow chart diagram showing patient flow through the study.

Supplemental Figure 2. Calibration plots of the original ARRS.

Supplemental Figure 3. Correlation of IFTA grading and percentage.

Supplemental Figure 4A. Kidney survival according to the stratification of the simplified ANCA renal risk score (ARRS).

Supplemental Figure 4B. Calibration plots of the simplified ARRS.

Supplemental Figure 5. Exploratory decision tree for ESKD.

Supplemental Figure 6. Proof of concept for AKRiS online calculator.

Supplemental Figure 7. Receiver operating characteristic curves of the AKRiS and the original ARRS.

Supplemental Figure 8. Calibration plots of the new and continuous AKRiS model in the validation cohort.

Supplemental Figure 9. Calibration plots of the AKRiS under a Fine–Gray model.

Supplemental Equation 1. Semiparametric survival equation of the original ARRS.

Supplemental Equation 2. Semiparametric survival equation of the new AKRiS.

Supplemental Equation 3. Semiparametric survival equation of the continuous AKRiS model.