Performance Prediction for Surgical Outcomes in Partial Nephrectomy Using Nephrometry Scores: A Comparison of Arterial Based Complexity (ABC), RENAL, and PADUA Systems

Abstract

Background:

Several standardized scoring systems are used to quantify renal tumor complexity on the basis of anatomic features to predict perioperative and postoperative outcomes of partial nephrectomy (PN).

Objective:

To compare the predictive accuracy and utility of the Arterial Based Complexity (ABC), RENAL, and PADUA scores.

Design, setting, and participants:

Between January 2013 and March 2016, 304 patients at our institution underwent PN plus complete triphasic contrast computed tomography (CT) scans. Two urologists independently scored CT images to retrospectively evaluate each patient using the ABC, RENAL, and PADUA nephrometry scoring systems.

Outcome measurements and statistical analysis:

Interobserver variability was reported for each of the three nephrometry scores; κ = 1 represented perfect agreement between the two urologists and κ = 0 represented as much agreement as expected by chance. Univariate and multivariable linear regression models were used to investigate associations of the nephrometry scores with estimated blood loss (EBL), ischemia time, and estimated glomerular filtration rate (eGFR) at 18 mo. Coefficients of determination (R²) were compared to determine which nephrometry score accounted for the most variation in outcome.

Results and limitations:

The κ value was 0.52 for ABC, 0.53 for RENAL, and 0.63 for PADUA (all p ≤ 0.001). On univariate analysis, there were no significant associations between nephrometry scores and postoperative eGFR; all three scores were highly associated with ischemia time (p < 0.0001) and EBL (p ≤ 0.001). R² was not significantly different among the three scoring systems. On multivariable analysis, all three nephrometry scores were significantly associated with ischemia time (p < 0.0001) and EBL (p ≤ 0.01); only the RENAL score was associated with postoperative eGFR (p = 0.044), so its performance on this metric could not be compared to that of ABC or PADUA.

Conclusions:

The ABC, RENAL, and PADUA systems have similar performance for predicting EBL and ischemia time outcomes in PN, and are thus equally useful for assessing PN complexity. Further education and training are needed to reduce interobserver variability.

Patient summary:

A new score system called Arterial Based Complexity (ABC) can be used to evaluate the complexity of a renal tumor and predict how difficult the tumor resection (partial nephrectomy) may be. This system performs well compared to other established systems and seems easy to learn and use.

1. Introduction

In recent years, partial nephrectomy (PN) has been increasingly used for the surgical treatment of localized renal tumors [1]. To provide clinicians with standardized and reproducible tools to quantify renal tumor complexity on the basis of anatomic features, several different nephrometry scores have been developed [2–6]. These scores were introduced into clinical practice for predicting PN surgical complexity, the risk of perioperative complications, and postoperative outcomes [7–10].

The RENAL (radius, exophytic/endophytic, nearness, anterior/posterior, location) and PADUA (preoperative aspects and dimensions used for anatomic) nephrometry scores were initially described in 2009 and are the most commonly used [2,3]. Although these scores play an important role in renal tumor assessment and may impact clinical decisions, their use in daily practice is limited by issues of reproducibility and accurate prediction of clinical outcomes. Several studies have demonstrated that the reader’s depth of experience strongly influences nephrometry score reproducibility [11,12]. Moreover, the RENAL and PADUA scoring systems were initially constructed to communicate tumor anatomy; their ability to predict clinical outcomes of PN remains controversial, with reports of inconsistent results [13–16].

The Arterial Based Complexity (ABC) scoring system was recently introduced to assess the surgical complexity of renal masses using the relationship between the tumor and renal arterial vascular anatomy [17]. This new nephrometry score gives special importance to arterial branches that will probably be dissected or ligated during PN. The initial study on ABC showed that it is intuitive and correlated with perioperative morbidity. To more fully evaluate ABC, we compared its reproducibility and predictive ability to those of the RENAL and PADUA nephrometry scores for PN outcomes.

2. Patients and methods

2.1. Patient selection

After institutional review board approval, we identified 782 patients who underwent PN at Memorial Sloan Kettering Cancer Center (MSK) between January 2013 and March 2016. However, only 304 patients had a complete triphasic contrast computed tomography (CT) scan performed in our institution and could be included in the analysis of ABC, RENAL, and PADUA nephrometry scores. RENAL and PADUA scores were calculated as previously reported [2,3].

2.2. Nephrometry performance

Two urologists (R.G.A. and F.A.) independently evaluated triphasic CT scans to arrive at the ABC, RENAL, and PADUA scores. The ABC score was based on a single factor as initially described [17]: the relationship between tumor and renal vessels. Category 1 comprises tumors involving only the renal cortex, and thus encompassing the interlobular and arcuate arteries. Category 2 includes tumors extending to the renal medulla and reaching the line connecting the tip of the renal papillae, and therefore requiring transection of the interlobar arteries. Category 3S consists of tumors extending into the renal sinus towards the central collecting system and involving the segmental arteries and their branches. Category 3H comprises tumors in proximity to or involving the renal hilum. Measurements for RENAL and PADUA scores involved multiple factors, such as tumor diameter, endophytic or exophytic aspect, nearness to the collector system and renal sinus, the relation to polar lines, and medial or lateral position. The urologists were blinded to patient characteristics, outcomes, and the scoring of the other urologist. Both were well accustomed to the RENAL and PADUA systems, having used them since 2009, but were using ABC for the first time. For each of the three nephrometry scores, interobserver variability was calculated. We first compared the two urologists’ tumor scoring using the intraclass correlation coefficient (ICC) for the continuous RENAL and PADUA scores, and the κ statistic for the ordinal ABC system. To compare the RENAL and PADUA scores to the ABC score, we also reported the κ value for the categorized RENAL score (low complexity 4–6, moderate complexity 7–9, high complexity 10–12) and the categorized PADUA score (low complexity 6–7, moderate complexity 8–9, high complexity 10–13). κ values ranged from −1 to 1, with 1 indicating perfect agreement between scorers, 0 indicating as much agreement as expected by chance, and <0 indicating less agreement than expected by chance. We also calculated the average percentage of times the urologists matched each other exactly in pairwise combinations when scoring the same patient, as well as when the urologists differed by no more than 1 point, as a secondary assessment of interobserver variability.

2.3. Statistical analysis

We investigated the association between each of the three nephrometry scores and PN perioperative and postoperative outcomes. Univariate linear regression was used to assess association of the nephrometry scores with estimated blood loss (EBL), ischemia time, and estimated glomerular filtration rate (eGFR) at 18 mo (defined as the eGFR measurement taken closest to the 18-mo point between 12 and 24 mo and calculated using the Modification of Diet in Renal Disease equation). Every model has an associated R² value, which represents the amount of variability in the outcome that is explained by the covariates included in the model. If a significant association between nephrometry score and outcome was found, we compared the coefficient of determination R² between models to determine which nephrometry score accounted for the most variation in outcome. To investigate whether nephrometry scores added value to other prognostic factors for case mix adjustment, we used multivariable linear regression models. Each tumor was scored twice, and the higher of the two scores was used in the regression modeling.

RENAL and PADUA scores were entered into the model as continuous variables (RENAL range 4–12, PADUA range 6–14), and the ABC score was included as a categorical variable (categories 1, 2, 3S, and 3H). Multivariable models were adjusted for preoperative eGFR, sex, age, Charlson comorbidity index (0, 1, or ≥2), and maximum tumor dimension. If nephrometry scores were significantly associated with an outcome when controlling for prognostic factors, we again compared R² between nephrometry scores. We used bootstrap resampling to estimate 95% confidence intervals (CIs) for the difference in R² between nephrometry scores. To assess for the possibility of nonlinearity, we performed a sensitivity analysis in which all analyses were repeated using a squared term for RENAL and PADUA scores. In another sensitivity analysis, we assessed using the lower of the two nephrometry scores and a randomly chosen nephrometry score in the regression models. All analyses were conducted using Stata 13 (StataCorp, College Station, TX, USA).

3. Results

The patient characteristics are presented in Table 1. The majority of patients were male (63%), aged >60 yr (median age 61 yr, interquartile range 54–68), and underwent open PN (62%).

Table 1–

Patient characteristics (n = 304)

Parameter	Result
Male, n (%)	193 (63)
Median age at surgery, yr (IQR)	61 (54–68)
Type of partial nephrectomy, n (%)
Open	187 (62)
Laparoscopic	28 (9.2)
Robotic	89 (29)
Charlson comorbidity index, n(%)
0	188 (62)
1	42 (14)
≥2	74 (24)
Median tumor size, cm (IQR) [n = 302]	3.0 (2.1–4.0)
Median preoperative eGFR, ml/min/1.73 m2(IQR) [n = 282]	86 (71–97)
RENAL score, median (IQR)	8 (6–9)
Low complexity (score 4–6), n(%)	84 (28)
Moderate complexity (score 7–9), n(%)	176 (58)
High complexity (score 10–12), n(%)	44 (14)
PADUA score, median (IQR)	8.5 (7–10)
Low complexity (score 6–7), n(%)	93 (31)
Moderate complexity (score 8–9), n(%)	107 (35)
High complexity (score 10–13), n(%)	104 (34)
ABC score, n(%)
1	25 (8.2)
2	118 (39)
3S	105 (35)
3H	56 (18)

eGFR = estimated glomerular filtration rate; IQR = interquartile range.

3.1. Interobserver variability

The ICC was 0.77 (95% CI 0.69–0.83) for the RENAL score and 0.82 (95% CI 0.76–0.86) for PADUA (Fig. 1A,B). Concordance of RENAL and PADUA scores between scorers is also presented in Figure 1A,B. κ was 0.52 for the ABC score, 0.53 for RENAL, and 0.63 for PADUA. Table 2 shows the concordance of ABC scores between scorers.

Fig. 1 –

Concordance between the two reviewers for (A) the RENAL score and (B) the PADUA score.

Table 2–

Agreement in ABC scores between reviewers

Reviewer 1 score	Samples in agreement, n(%)
	Reviewer 2 score
	1	2	3S	3H
1	22 (85)	14 (11)	1 (1.0)	0 (0)
2	3 (12)	98 (80)	38 (37)	8 (17)
3S	1 (3.8)	9 (7.3)	55 (54)	13 (28)
3H	0 (0)	2 (1.6)	8 (7.8)	25 (54)

3.2. Association with outcomes

The RENAL, PADUA, and ABC scores were all highly associated with ischemia time (all p < 0.0001) and EBL (all p ≤ 0.001; Table 3). The PADUA and ABC scores were not significantly associated with postoperative eGFR (p = 0.3 and p = 0.2, respectively). There was some evidence of an association between postoperative eGFR and RENAL score, although this did not reach statistical significance (p = 0.051). The R² values were not compared because of the lack of significance on univariate analysis.

Table 3 –

Univariate model results for the outcomes EBL, ischemia time, and eGFR at 18 mo after partial nephrectomy ^a

Score	EBL (n = 299)			Ischemia time (n = 289)			eGFR at 18 mo (n = 143)
	β (95% CI)	p value	R 2	β (95% CI)	p value	R 2	β (95% CI)	p value	R 2
RENAL	33.8 (17.4–50.2)	<0.0001	0.052	2.39 (1.73–3.04)	<0.0001	0.152	−1.39 (−2.79 to 0.01)	0.051
PADUA	34.7 (17.7–51.7)	<0.0001	0.052	2.41 (1.73–3.08)	<0.0001	0.146	−0.87 (−2.45 to 0.70)	0.3
ABC 1	Reference	0.001	0.057	Reference	<0.0001	0.141	Reference	0.2
2	−33.8 (−152.3 to 84.7)			6.39 (1.39–11.4)			−6.78 (−15.8 to 2.28)
3S	68.8 (−50.7 to 188)			11.5 (6.42–16.5)			−6.32 (−15.6 to 2.96)
3H	138 (9.26–268)			15.3 (9.90–20.7)			−11.7 (−22.2 to −1.06)

CI = confidence interval; EBL = estimated blood loss; eGFR = estimated glomerular filtration rate; IQR = interquartile range.

^aβ represents the value increase for the outcome per 1-point increase in nephrometry score. For example, a 1-point increase in RENAL score would result, on average, in a 33.8-ml increase in EBL.

While all three nephrometry scores were significantly associated with EBL and ischemia time on multivariable analysis (Table 4), we found no significant difference in the magnitude of variation accounted for by each nephrometry score (Table 5). The RENAL score was significantly associated with eGFR at 18 mo (p = 0.044), but PADUA and ABC were not (p = 0.3 and p = 0.079, respectively). However, there was an association between 18-mo eGFR and the ABC scores for more complex tumors: 3S −9.04 (95% CI −16.3 to −1.75) and 3H −9.77 (95% CI −17.8 to −1.72). R² values were not compared for this outcome because of the lack of significance of PADUA and overall ABC scores. The sensitivity analysis using nonlinear terms for RENAL and PADUA scores and the sensitivity analysis using the lower of the two nephrometry scores and a randomly chosen score produced similar results (data not shown).

Table 4 –

Multivariable model results for EBL, ischemia time, and eGFR outcomes at 18 mo after partial nephrectomy ^a

Score	EBL (n = 276)			Ischemia time (n = 265)			eGFR at 18 mo (n = 143)
	β (95% CI)	p value	R 2	β (95% CI)	p value	R 2	β (95% CI)	p value	R 2
RENAL	26.7 (7.68–45.7)	0.006	0.107	2.00 (1.25–2.74)	<0.0001	0.195	−1.09 (−2.16 to −0.03)	0.044	0.573
PADUA	28.7 (8.97–48.4)	0.005	0.109	2.14 (1.37– 2.91)	<0.0001	0.201	−0.71 (−1.92 to 0.50)	0.3
ABC 1	Reference	0.010	0.120	Reference	<0.0001	0.189	Reference	0.079
2	−35.5 (−160 to 89.2)			6.34 (1.18–11.5)			−6.26 (−12.8 to 0.29)
3S	44.9 (−91.1 to 181)			10.0 (4.44–15.6)			−9.04 (−16.3 to −1.75)
3H	130 (−15.7 to 276)			13.9 (7.97–19.8)			−9.77 (−17.8 to −1.72)

CI = confidence interval; EBL = estimated blood loss; eGFR = estimated glomerular filtration rate; IQR = interquartile range.

^aModels were adjusted for preoperative eGFR, sex, age, Charlson comorbidity index, tumor size, and nephrometry score. β represents the value increase for the outcome per 1-point increase in nephrometry score. For example, a 1-point increase in RENAL score would result, on average, in a 26.7-ml increase in EBL.

Table 5 –

Difference in R²(with 95% confidence interval) between nephrometry scores for univariate and multivariable models

Outcome	ABC vs RENAL	ABC vs PADUA	RENAL vs PADUA
Univariate
Estimated blood loss	0.005 (−0.033 to 0.059)	0.005 (−0.029 to 0.052)	0.001 (−0.024 to 0.026)
Ischemia time	−0.011 (−0.065 to 0.045)	−0.006 (−0.052 to 0.045)	0.005 (−0.037 to 0.046)
Multivariable
Estimated blood loss	0.013 (−0.020 to 0.065)	0.011 (−0.018 to 0.058)	−0.002 (−0.024 to 0.017)
Ischemia time	−0.006 (−0.050 to 0.043)	−0.012 (−0.057 to 0.033)	−0.006 (−0.048 to 0.028)

The new Arterial Based Complexity (ABC) scoring system for evaluating kidney tumors before partial nephrectomy (PN) performed as well as the established RENAL and PADUA scoring systems. ABC performance in predicting estimated blood loss and ischemia time outcomes in PN was similar to RENAL and PADUA. The ABC score provides similar information on tumor complexity and perioperative outcomes with, presumably, a shorter learning curve than the other two systems.

4. Discussion

Our study revealed no reason to favor one score over another in terms of ability to predict perioperative endpoints such as ischemia time and EBL. This result is supported by other series that compared the three scoring systems and their prediction of PN outcomes [18–20], although we make an argument below for the greater ease of use of the ABC system. Nephrometry scores are important tools in kidney cancer management. These scores allow the categorization of renal tumors and comparison of different treatments in different institutions, especially for PN. Although these systems were not built to predict surgical complications, many studies have investigated their use as predictors of PN outcomes. While there are several contradictory results [14,21], it seems that associations with ischemia time, EBL, and high-grade complication rates are the most common factors related to higher RENAL and PADUA scores [7–10].

For the RENAL and PADUA scores, it has been shown that tumor proximity to the renal hilum (nearness) is the most important predictor of PN perioperative outcomes, especially ischemia time and EBL [7,15]. Another contemporary critical assessment of these two scoring systems reported that only radius, nearness to the collecting system, and polar location were clinically significant factors [21]. This probably reflects the fact that tumor correlation with renal artery branches, the principle behind the ABC scoring system, is the most important aspect of tumor complexity and can predict perioperative outcomes.

Naturally, some correlation between the three scores was expected: tumors close to the renal sinus, a factor used in the RENAL and PADUA systems, are consequently related to major artery branches, as used in the ABC system, and tend to have higher scores, indicating more complex surgery and a potential risk of complications. In our analysis, there were no significant differences between the three systems in correlation with EBL or ischemia time. There are potential roles for renal nephrometry scores for predicting other outcomes, such as postoperative complications. An association between tumor complexity and postoperative major complications (>2 in the Clavien-Dindo classification system) after PN was observed in a single-center study of 390 patients [7]. The RENAL score was also independently associated with the occurrence of major complications after laparoscopic cryoablation in a multicenter analysis of 77 patients [22]. In our cohort, only 17 patients (5.6%) had grade ≥2 complications, which was insufficient for multivariable analysis. This notably low rate, even with half or more of our patients presenting with moderately or highly complex tumors (53% ABC score of 3, 72% RENAL score 7–12, and 69% PADUA score 8–13), supports our belief that surgeon expertise plays the primary role in influencing the surgical complication rate, meaning that tumor complexity scores are only of secondary relevance. In fact, in high-volume centers, PN usually has very low complication rates, potentially decreasing the correlation with tumor complexity in this setting. Another controversial issue concerns the ability of these scores to predict renal function recovery after PN. Previous studies that included only patients with a solitary kidney, in whom eGFR is not affected by the normal contralateral kidney compensation, failed to demonstrate the an ability of the RENAL score to predict function loss after PN [23]. Another recent report that used the spectrum score also showed no relation between the RENAL score and acute renal dysfunction after PN; the only predictors of post-PN acute renal dysfunction were ischemia time and type (cold vs warm) [24]. In our study, the RENAL score had some significant correlation to eGFR recovery after PN, while the ABC score showed a similar relation only for complex tumors (3S and 3H). However, considering that complex tumors are more often related to complications after PN, this fact demonstrates that risk stratification according to the ABC score has good accuracy. Our results showing that higher nephrometry scores are linked to higher EBL, longer ischemia time, and (in some cases) prediction of a decrease in postoperative eGFR are supported by other authors and reinforce the concept that partial resection of more complex tumors often involves longer ischemia time and greater blood and parenchyma loss, directly impacting kidney function recovery.

An important limitation of the scoring systems is inconsistency in scoring, especially regarding tumor position such as hilum proximity and relation to polar lines. For the RENAL and PADUA systems, several publications have reported that measurement variability was high and the subsequent interobserver agreement (IOA) rate was low [12,14,25,26]. This variability is probably responsible for the conflicting results seen in other studies, especially those related to prediction of PN outcomes and, of course, is presumably a function of observers not having extensive enough experience or familiarity with these scoring systems. A contemporary analysis of 299 patient records found that IOA varied between 51% and 80%; nearness to the collecting system was the most unpredictable variable (IOA 51%) followed by radius (IOA 66%), and location (IOA 73%) [5]. This variability may lead to under- or overestimation of renal mass scoring, which could impact the potential clinical outcome predictions of these scores. The degree of IOA necessary for wide use of renal nephrometry scores is not yet defined, but it seems rational to aim for the highest possible agreement.

Potential limitations of our study design include the single-institution analysis; the scoring by two urologists only without an experienced radiologist as a reference; the absence of a C-index score comparison; and the absence of time measurement for scoring. It has been shown for the classic nephrometry scores (RENAL and PADUA) that reproducibility between urologists and radiologists is not always good [11]; in any case, the primary endpoint of this study was to assess the utility of a more intuitive tool for clinicians, the ABC score, in comparison to the classic scores. We did not include the C-index as a comparator since PADUA and RENAL are the most commonly used systems and considered the standard systems for PN performance prediction. For validation of the ABC system, we believe that a comparison to the C-index is less meaningful. However, we admit that comparing the C-index and ABC systems might have been interesting as the two systems share some principles, such as not considering tumor size or relation to polar lines. Our perception that scoring is easier using the ABC system when compared to RENAL and PADUA is currently a subjective observation. Considering that both urologists had 8 yr of experience with RENAL and PADUA but were using ABC for the first time, our impression is that the ABC learning curve is easier and more intuitive. Of course, an objective timing comparison to prove this fact is absolutely necessary in the future. One final limitation is that our study population was restricted to patients with triphasic contrast CT scan performed at our institution, since such a scan provides better correlation between tumor and renal artery branches to calculate the ABC score. All three scoring systems benefit from using standardized CT images (ie, those taken at a single institution) for consistency and interobserver agreement when determining the measurements and tumor features that contribute toward the nephrometry score.

5. Conclusions

The ABC performance in predicting EBL and ischemia time outcomes in PN is similar to that of RENAL and PADUA. The ABC score provides similar information on tumor complexity and perioperative outcomes. However, further education and training are needed to reduce interobserver variability.