Abstract
Background
Solid small renal masses (SRMs, ≤4 cm) represent benign and malignant tumors. Among SRMs, clear cell renal cell carcinoma (ccRCC) is frequently aggressive. Compared to invasive percutaneous biopsies, a proposed clear cell likelihood score (ccLS) aims to diagnose ccRCC non-invasively using multiparametric MRI (mpMRI), but lacks external validation.
Purpose
To evaluate the performance of and interobserver agreement for ccLS to diagnose ccRCC among solid SRMs.
Materials and methods
This retrospective, multicenter, cross-sectional study included patients with consecutive solid (≥25% approximate volume enhancement) SRMs undergoing mpMRI between December 2012 and December 2019 at five academic medical centers with histological confirmation of diagnosis. Masses with macroscopic fat were excluded. After a 1.5-hour training session, two abdominal radiologists per center independently rendered a ccLS for 50 masses. The diagnostic performance for ccRCC was calculated using random-effects logistic regression modeling. The distribution of ccRCC by ccLS was tabulated. Interobserver agreement for ccLS was evaluated with Fleiss Kappa.
Results
241 patients (mean±SD age, 60 ± 13 years, 174 men) with 250 solid SRMs were evaluated. The mean ±SD size was 25±8 mm (range 10–39 mm). 48% (119/250) of SRMs were ccRCC. The sensitivity, specificity, and positive predictive value for the diagnosis of ccRCC (95% CI) when ccLS≥4 were 75% (68%, 81%), 78% (72%, 84%), and 76% (69%, 81%), respectively. The negative predictive value of ccLS≤2 was 88% (81%, 93%). The percentages of ccRCC according to the ccLS were 6% (range, 0%–18%), 38% (range, 0%–100%), 32% (range, 18%–53%), 72% (range, 20%–100%), and 81% (range, 50%–100%) for ccLS 1–5, respectively. The mean interobserver agreement was moderate (Kappa=0.58 [95% CI: 0.42, 0.75]).
Conclusion
The clear cell likelihood score applied to multiparametric MRI had moderate interobserver agreement and differentiated clear cell renal cell carcinoma from other solid renal masses with a negative predictive value of 88%.
Summary
Clear cell likelihood scores provide a framework for standardized multiparametric MRI assessment of solid small renal masses with moderate diagnostic accuracy for clear cell renal cell carcinoma.
Introduction
Renal masses are commonly encountered as incidental findings in patients undergoing cross-sectional imaging (1, 2). The majority of solid small (≤4 cm [cT1a]) renal masses (SRMs) are malignant, but up to 20% are benign (3, 4). Moreover, even when malignant, cT1a renal cell carcinoma (RCC) is frequently indolent and when treated, rarely recurs locally or metastasizes (5). Indeed, cT1a RCC is an uncommon cause of patient mortality, particularly in older patients who may have other competing comorbidities (6). Among the various RCC subtypes, clear cell RCC (ccRCC) is the most common and is often aggressive; based on this combination of characteristics, ccRCC is the most common cause of disease progression and metastasis in active surveillance (AS) populations (7).
Various methods have been proposed to differentiate benign from malignant and indolent from aggressive SRMs in clinical practice. Renal mass biopsy can differentiate between the various histologic subtypes of solid SRMs (8). However, a biopsy represents an additional diagnostic procedure, is invasive (median complication rate 8%) (8), is not feasible in all patients, and is non-diagnostic in up to 20% of renal masses (9). Moreover, renal mass biopsy is not widely utilized in clinical practice due to the perception among many urologists that its results do not alter management (10).
Noninvasive imaging diagnosis of renal mass subtype among solid SRMs by CT or MRI represents an alternative to biopsy, has been studied extensively, and has been shown, to some extent, to be accurate for the diagnosis of some renal mass histopathologic subtypes (11–15). However, studies on this topic have been mainly limited to single-center retrospective case-control series and therefore show selection bias (13, 14, 16–19). To date, unlike for cystic renal masses, where imaging assessment is performed using the Bosniak classification (20), there is no broadly accepted standardized method for stratifying the risk of solid renal masses.
The clear cell likelihood score (ccLS) system is a 5-tier Likert scale that estimates the likelihood of an SRM being a ccRCC (1= very unlikely; 2= unlikely; 3= intermediate likelihood; 4= likely; and 5= very likely) (21). The ccLS system is a standardized framework generated using multiparametric MRI (mp-MRI) and a guiding algorithm (22). Preliminary data support the use of the ccLS as a potentially accurate and reproducible means to diagnose ccRCC (23, 24). However, its diagnostic performance has been reported at only two institutions within the same medical center in which the system was created (23, 24). Importantly, although the ccLS algorithm provides a methodology for assigning the likelihood of ccRCC, a subjective component exists in the interpretation of mpMRI examinations. Thus, the generalizability of the ccLS system is currently unknown. The present study aims to evaluate the performance of and interobserver agreement for the ccLS system in the diagnosis of ccRCC in solid SRMs.
Materials and Methods
Patients
This multicenter study was designed, supported by, and organized through the Society of Abdominal Radiology Disease Focused Panel on RCC (25). Approval was obtained from the institutional review boards (IRBs) of the five participating sites, (Ottawa Health Science Network Research Ethics Board IRB#20150932–01H; University of Nebraska Medical Center IRB, IRB#0744–19-EP; Mass General Brigham IRB IRB#2017P000455; Michigan Medicine IRB IRB#156057 and NYU Grossman School of Medicine IRB, IRB#i19–01522) and the need for informed consent was waived. Deidentified data were shared securely through a data-sharing agreement among the participating sites.
Our study was a retrospective cross-sectional analysis including indeterminate solid SRMs (≤4 cm) evaluated with mpMRI prior to histopathological confirmation by surgical excision or image-guided percutaneous biopsy. Sample size justification is provided in Supplementary Material, Appendix 1. Masses were identified by a single reviewer at each site (BB, DB, EE/ZB, IG, NS; years of post-fellowship experience 1–10) who was not involved in image interpretation. Consecutive institutional retrospective searches of imaging and pathology databases were performed on and prior to December 31, 2019.
Adult patients with solid (approximately ≥25 enhancing) SRMs and histologic diagnosis within 1 year of mpMRI were included. A summary of SRM histological criteria used in this study are provided in Supplementary Material, Appendix 2. Patients with infiltrating masses or containing macroscopic fat, genetic predisposition for RCC, >3 masses in one kidney, or not adequate mpMRI exam (see below) or pathologic diagnosis (see Supplemental material) were excluded (Figure 1). Patient sex and age at the time of MRI were recorded from the electronic medical records.
Figure 1.
Flow diagram illustrating the methodology for patient and renal mass identification and inclusion and exclusion criteria for the multiparametric MRI evaluation of the clear cell renal cell carcinoma likelihood score (ccLS).
MRI protocol
A radiologist at each site not involved in image interpretation reviewed MRI examinations to confirm that the following minimum acquisition requirements were met: 1) 1.5- or 3-Tesla whole-body MRI scanner; 2) axial and/or coronal T2-weighted (T2W) single-shot fast or turbo spin echo, 2) dual-echo chemical shift (in and opposed phase) T1-weighted (T1W) gradient-recalled echo (GRE), 3) axial single-shot echo-planar diffusion-weighted imaging (DWI) with a low (≤200 mm2/sec) and high (≥500 mm2/sec) b-value and 4) multiphasic contrast-enhanced fat-suppressed (FS) T1W GRE obtained before and after intravenous administration of a gadolinium-based contrast agent with a power injector during the corticomedullary (~30 seconds) and nephrographic (~100 seconds) phases with same acquisition parameters comparing pre-contrast and post-contrast sequences. Any patient without these minimum acquisition requirements was excluded. Additional MRI-specific parameter details are described elsewhere (26–28) and are summarized for all sites in Supplementary Tables 2a–e.
Radiologist interpretation
Image interpretation was conducted independently at each site by two abdominal imaging fellowship-trained radiologists (total: 10 radiologists [CW, RH-Ottawa; EG, RD-Nebraska; AS, SGS-Brigham and Women’s; NC, MD-Michigan; AD, NH-NYU] across 5 sites, 5–30 years of post-fellowship experience). Interpretation was performed using the institutional Picture Archiving and Communication Systems (PACS). The interpreting radiologists were blinded to the clinical information, including the histopathological diagnosis.
Prior to interpretation, each radiologist attended a 1.5-hour virtual teaching session (Microsoft Teams, Microsoft Corporation) where the senior author (IP) reviewed the use of the ccLS version 2.0 algorithm (22) and presented several illustrative case examples. The senior author or other radiologists at the institution where ccLS was developed did not participate in the review of mpMRIs or contribute mpMRI examinations to the study sample. Radiologists independently completed a secure, data-validated, data extraction sheet (Microsoft Excel, Microsoft Corporation) that included 6 ccLS imaging features: (1) signal intensity on T2W, 2) corticomedullary enhancement, 3) microscopic fat, 4) restriction on DWI, 5) segmental enhancement inversion, and 6) the arterial-to-delayed enhancement ratio [ADER]) and the final ccLS. A tip sheet and summary diagram of the ccLS system was also provided that summarized the ccLS algorithm in written and graphical format (Supplementary Table 3 and Supplementary Figure 1). Completed data extraction sheets were shared securely and compiled for pooled analyses. No medical images were shared in our study, and radiologists interpreted images only from their own institution.
Statistical analysis
Data were tabulated and are presented as the mean ± standard deviation (range) for quantitative data with summary statistics for categorical variables. Comparisons of demographic variables and the distribution of ccRCC SRMs across participating institutions were performed using the Kruskal-Wallis test or analysis of variance (ANOVA). A receiver operating characteristic (ROC) curve was generated, and diagnostic accuracy, including 95% CI, was calculated for pooled data using random-effects logistic regression where both the radiologist and sites were considered as random effects. Empiric ROC curves and diagnostic accuracy were also calculated at the site-specific level for each reader using 2×2 tables. A ccLS of ≥4 was considered to be a positive result for ccRCC, and a ccLS of ≤ 2 was considered to be a negative result for ccRCC (23). The percentage and range of percentages by site of ccRCC and malignancy by ccLS category were tabulated for each reader and overall. Interobserver agreement was evaluated for ccLS scoring using the Fleiss kappa statistic, where 0–0.2 was slight, 0.21–0.40 was fair, 0.41–0.60 was moderate, 0.61–0.80 was substantial and .81-.99 was almost perfect agreement. The average kappa values with 95% CIs are reported for the pooled data (29). A p value <.05 was considered statistically significant. Data analyses were performed using STATA version 15.1 (StataCorp).
Results
Patient characteristics
There were 250 solid SRMs in 241 patients (8 patients had 2 masses, and 1 patient had 3 masses). The mean ± standard deviation (SD) age of patients was 60 ± 13 years and there were 174 men. A summary of the other patient demographics is provided in Table 1. There were 48% (119/250) ccRCC and 52% (131/250) non-ccRCC diagnoses. We found no difference in the percentage of ccRCC diagnoses (Table 1) across participating institutions (p=.65).
Table 1.
Patient demographic and renal mass characteristics across five participating academic institutions for the evaluations of multiparametric MRI examinations using the clear cell renal cell carcinoma likelihood score (ccLS)
| Variables | Institution 1 (n=50) | Institution 2 (n=50) | Institution 3 (n=50) | Institution 4 (n=50) | Institution 5 (n=50) | All (n=250) | P-value1 | |
|---|---|---|---|---|---|---|---|---|
| Age (years) | 60 ± 13 | 60 ± 13 | 63 ± 12 | 60 ± 13 | 59 ± 12 | 60±13 | 1.00 | |
| Sex | Men | 28 | 36 | 36 | 39 | 35 | 174 (70%) | .73 (*) |
| Women | 22 | 14 | 14 | 11 | 15 | 76 (30%) | ||
| Size (mm) | 24 ± 8 | 24 ± 9 | 25 ± 8 | 24 ± 8 | 25 ± 9 | 25±8 | 1.00 | |
| Histological diagnosis | Clear cell renal cell carcinoma (RCC) | 50% (25/50) | 40% (20/50) | 50% (25/50) | 44% (22/50) | 54% (27/50) | 47.6% (119/250) | .65 (#) |
| All other histologies | 50% (25/50) | 60% (30/50) | 50% (25/50) | 56% (28/50) | 46% (23/50) | 52.4% (131/250) | ||
| Papillary RCC | 13 | 11 | 10 | 16 | 7 | 57 | ||
| Oncocytoma | 5 | 3 | 7 | 4 | 8 | 27 | ||
| Chromophobe RCC | 1 | 5 | 2 | 1 | 5 | 14 | ||
| Fat-poor angiomyolipoma | 3 | 4 | 2 | 9 | ||||
| RCC unclassified | 2 | 2 | 2 | 1 | 7 | |||
| Clear cell-papillary RCC | 5 | 5 | ||||||
| Oncocytic neoplasm | 2 | 1 | 1 | 4 | ||||
| Metastasis | 1 (lung cancer) | 1 (colon cancer) | 2 | |||||
| Mucinous tubular and spindle cell tumor | 1 | 1 | 2 | |||||
| Epithelioid angiomyolipoma | 1 | 1 | ||||||
| Multilocular cystic neoplasm of low malignant potential | 1 | 1 | ||||||
| Lymphoma | 1 | 1 | ||||||
| Sclerosing PEComa (low grade epithelioid neoplasm) | 1 | 1 |
-Comparisons performed to evaluate for any potential differences in patient or renal mass characteristics among institutions using Kruskal-Wallis for categorical variables or ANOVA for quantitative data.
Reflects the analysis across all institutions of both men and women at each institution.
Reflects a comparison in the distribution of ccRCC vs other histology across all institutions.
Diagnostic accuracy
Individual and pooled ROC curves for the diagnosis of ccRCC using the ccLS are provided in Figure 2 and Supplementary Tables 4 and 5. The area under the ROC curve (AUC) for the pooled results from the ten radiologists was 0.80 (95% CI: 0.77, 0.84). Using a threshold of a ccLS≥4 to diagnose ccRCC resulted in a pooled sensitivity and specificity of 75% (95% CI: 68%, 81%) and 78% (95% CI: 72%, 84%), respectively, with a positive predictive value (PPV) and negative predictive value (NPV) of 76% (95% CI: 69%, 81%) and 77% (95% CI: 72%, 82%), respectively. There were no differences in calculated accuracy comparing performance using a threshold of a ccLS≥4 to diagnose ccRCC when accounting for radiologists and site as variables in the random effects model (p=.19, .49 respectively). Using a threshold of a ccLS≤2 to exclude a diagnosis of ccRCC resulted in a pooled NPV of 88% (95% CI 81%, 93%). There were no differences in calculated accuracy comparing performance using a threshold of a ccLS≤2 to diagnose ccRCC when accounting for radiologists and sites as variables in the random effects model (p=.19 and .38 respectively).
Figure 2.
Receiver operating characteristic (ROC) curve depicting the diagnostic performance of the multiparametric MRI clear cell likelihood score system across ten radiologists at five academic institutions (a) and overall (b), with the results pooled using a random-effects logistic regression model. AUC, area under the ROC curve.
Percentage of ccRCC and malignancy by ccLS
The pooled percentages of ccRCC diagnoses with ccLS were 6% (range, 0%–18%) for a ccLS of 1, 38% (range, 0%–100%) for a ccLS of 2, 32% (range, 18%–53%) for a ccLS of 3, 72% (range, 20%–100%) for a ccLS of 4, and 81% (range, 50%–100%) for a ccLS of 5 (Figure 3). The institution-specific and individual radiologist-level distributions of ccRCC by ccLS are provided in Table 2.
Figure 3.
Histogram plot depicting the distribution of clear cell renal cell carcinomas (ccRCCs) and other histologies by multiparametric MRI clear cell likelihood score (ccLS) category determined by averaging data from ten radiologists across five academic institutions (two radiologists per institution) evaluating 250 unique cT1a solid renal masses. Whiskers represent the lower and upper ranges of masses classified into each ccLS category by individual radiologists.
Table 2.
Percentage of clear cell RCCs by multiparametric MRI clear cell likelihood score (ccLS) category with 95% CIs for individual radiologists and range for overall1.
| ccLS 1 (95% CI) | ccLS 2 (95% CI) | ccLS 3 (95% CI) | ccLS 4 (95% CI) | ccLS 5 (95% CI) | |
|---|---|---|---|---|---|
| Institution 1 | |||||
| Radiologist 1 | 0% (0/5) [0, 60%] | 0% (0/1) [0, 100%]2 | 25% (3/12) [6%, 57%] | 62% (13/21) [38%, 82%] | 82% (9/11) [48%, 98%) |
| Radiologist 2 | 0% (0/8) [0%, 38%] | 0% (0/2) [0%, 100%) | 53% (9/17) [28%, 77%] | 20% (1/5) [5%, 72%] | 83% (15/18) [59%, 96%) |
| Institution 2 | |||||
| Radiologist 1 | 0% (0/11) [0%, 27%] | 0% (0/1) [0%, 100%] | 18% (2/11) [2%, 52%] | 75% (6/8) [35%, 97%] | 63% (12/19) [38%, 84%] |
| Radiologist 2 | 0% (0/12) [0%, 25%] | 50% (1/2) [1%, 99%] | 19% (3/16) [4%, 46%] | 100% (12/12) [74%, 100%] | 50% (4/8) [16%, 84%] |
| Institution 3 | |||||
| Radiologist 1 | 0% (0/10) [0%, 30%] | 33% (1/3) [8%, 91%] | 38% (6/16) [15%, 65%] | 81% (13/16) [54%, 96%] | 100% (5/5) [48%, 100%] |
| Radiologist 2 | 15% (2/13) [2%, 45%] | 50% (1/2) [1%, 99%] | 27% (4/15) [8%, 55%] | 88% (15/17) [64%, 99%] | 100% (3/3) [29%, 100%] |
| Institution 4 | |||||
| Radiologist 1 | 0% (0/17) [0%, 18%] | 100% (2/2) [16%, 100%] | 38% (3/8) [9%, 76%] | 60% (6/10) [26%, 88%] | 85% (11/13) [55%, 98%] |
| Radiologist 2 | 18% (2/11) [2%, 52%] | 50% (2/4) [7%, 93%] | 29% (5/17) [10%, 56%] |
71% (5/7) [21%, 96%] | 73% (8/11) [39%, 84%] |
| Institution 5 | |||||
| Radiologist 1 | 17% (1/6) [4%, 54%] | 33% (1/3) [8%, 91%] | 31% (5/16) [11%, 59%] | 64% (9/14) [35%, 87%] | 100% (11/11) [72%, 100%] |
| Radiologist 2 | 17% (1/6) [4%, 54%] | 0% (0/1) [0%, 100%] | 35% (6/17) [14%, 62%] | 57% (8/14) [29%, 82%] | 100% (12/12) [74%, 100%] |
| OVERALL (SUM of 10 readers) | 6% (6/99) [0, 18%] | 38% (8/21) [0, 100%] | 32% (46/145) [18, 53%] | 72% (88/124) [20, 100%] | 81% (90/111) [50, 100%] |
-Overall results obtained by summing individual results from 10 radiologists with range reported.
-The upper bound of the 95% CI for instances of zero events was calculated by ‘the rule of 3’ (43).
The percentages of patients with malignant histology by ccLS are shown in Table 3. The percentage of malignant tumors among those with a ccLS≥4 in our sample was 84% (range 40%, 100%). Overall, the mean percentage of benign histology by ccLS was 6% (6/99) for ccLS1, 19% (4/21) for ccLS2, 28% (40/145) for ccLS3, 21% (26/124) for ccLS4, and 11% (12/111) for ccLS5.
Table 3.
Percentage of malignant renal masses1 by multiparametric MRI clear cell likelihood score category for individual radiologists with 95% CI and overall with range.
| ccLS 1 (95% CI) | ccLS 2 (95% CI) | ccLS 3 (95% CI) | ccLS 4 (95% CI) | ccLS 5 (95% CI) | |
|---|---|---|---|---|---|
| Institution 1 | |||||
| Radiologist 1 | 100% (5/5) (48%, 100%) | 100% (1/1) (2.5%, 100%) | 83% (10/12) (52%, 98%) | 71% (15/21) (48%, 89%) | 73% (8/11) (39%, 94%) |
| Radiologist 2 | 88% (7/8) (47%, 100%) | 100% (2/2) (16%, 100%) | 82% (14/17) (57%, 96%) | 40% (2/5) (5%, 85%) | 78% (14/18) (52%, 94%) |
| Institution 2 | |||||
| Radiologist 1 | 100% (11/11) (72%, 100%) | 100% (1/1) (2.5%, 100%) | 82% (9/11) (42%, 98%) | 88% (7/8) (47%, 100%) | 84% (16/19) (60%, 97%) |
| Radiologist 2 | 100% (12/12) (74%, 100%) | 50% (1/2) (1%, 99%) | 69% (11/16) (41%, 89%) | 100% (12/12) (74%, 100%) | 100% (8/8) (63%, 100%) |
| Institution 3 | |||||
| Radiologist 1 | 80% (8/10) (44%, 97%) | 67% (2/3) (9%, 99%) | 69% (11/16) (41%, 89%) | 81% (13/16) (54%, 96%) | 100% (5/5) (48%, 100%) |
| Radiologist 2 | 85% (11/13) (55%, 98%) | 50% (1/2) (1%, 99%) | 60% (9/15) (32%, 84%) | 88% (15/17) (64%, 99%) | 100% (3/3) (29%, 100%) |
| Institution 4 | |||||
| Radiologist 1 | 94% (16/17) (71%, 100%) | 100% (2/2) (16%, 100%) | 63% (5/8) (24%, 91%) | 80% (8/10) (44%, 97%) | 92% (12/13) (64%, 100%) |
| Radiologist 2 | 100% (11/11) (72%, 100%) | 75% (3/4) (19%, 95%) | 71% (12/17) (44%, 90%) |
100% (7/7) (59%, 100%) | 91% (10/11) (59%, 100%) |
| Institution 5 | |||||
| Radiologist 1 | 100% (6/6) (54%, 100%) | 100% (3/3) (29%, 100%) | 69% (11/16) (41%, 89%) | 71% (10/14) (42%, 92%) | 100% (11/11) (72%, 100%) |
| Radiologist 2 | 100% (6/6) (54%, 100%) | 100% (1/1) (2.5%, 100%) | 76% (13/17) (50%, 93%) | 64% (9/14) (35%, 87%) | 100% (12/12) (74%, 100%) |
| Overall percentage2 of malignant diagnoses (Range across 10 radiologists) | 94% (93/99) (80%, 100%) | 81% (17/21) (50%, 100%) | 72% (105/145) (60%, 83%) | 79% (98/124) (40%, 100%) | 89% (99/111) (73%, 100%) |
=Dichotomization of diagnoses into benign versus malignant categories depicted in Supplementary Table 1.
=Overall percentage obtained by summing individual radiologist (N=10) results and dividing by the entire dataset (N=250×2=500 tumors).
The mean number of SRMs receiving a ccLS ≤2 across all sites was 12/50 (24%). The average distribution of histologies in ccLS ≤2 SRMs was 63% papillary RCC, 12% ccRCC, 8% chromophobe RCC, 4% oncocytoma, 3% fat-poor angiomyolipoma, and 9% other diagnoses.
Inter-observer agreement
The interobserver agreement for ccLS scoring is presented in Table 4. The average interobserver agreement was moderate (Kappa=0.58 [95% CI: 0.42, 0.75]).
Table 4.
Interobserver agreement (Fleiss Kappa) for the multiparametric MRI clear cell likelihood score with 95% CIs by institution and overall.
| Weighted Kappa | 95% CI | |
|---|---|---|
| Institution 1 | 0.46 | 0.28–0.63 |
| Institution 2 | 0.58 | 0.39–0.77 |
| Institution 3 | 0.66 | 0.46–0.85 |
| Institution 4 | 0.41 | 0.26–0.60 |
| Institution 5 | 0.82 | 0.64–1.00 |
| OVERALL | 0.58 | 0.42–0.75 |
Discussion
The need for better risk stratification strategies for indeterminate small renal masses (SRMs) that are solid is arguably the main barrier to the wide acceptance of active surveillance in clinical practice (7). The clinical implementation of such strategies necessitates tools that are widely adoptable, reliable, and reproducible for the diagnosis of disease with higher risk for metastasis and/or local progression. An approach that enables the diagnosis of clear cell renal cell carcinoma (ccRCC) among solid SRMs would facilitate appropriate stratification of most aggressive tumors since ccRCC is the most common subtype of RCC and is often aggressive (7).
Our study evaluated the performance of and the interobserver agreement for the clear cell likelihood score (ccLS) in the diagnosis of ccRCC in indeterminate solid SRMs. Our results indicate that using the cutoff of a ccLS ≥4 to diagnose ccRCC has moderate sensitivity (75%), specificity (78%) and positive predictive value (PPV; 77%) among indeterminate small SRMs imaged with mpMRI. Moreover, a ccLS of 1 or 2 had high (88%) negative predictive value for ccRCC. The interobserver agreement for ccLSs across five academic medical centers and ten radiologists was moderate (Kappa=0.58).
The observed sensitivity of 75% (95% CI: 68%, 81%) and specificity of 78% (95% CI: 72%, 84%) when applying a ccLS≥4 to diagnose ccRCC in this multicenter study were similar to those of the original, single-center report by Canvasser et al. (sensitivity of 78% and specificity of 80%)(23). Similarly, the 76% (69%, 81%) PPV was comparable to the 80% PPV found by Canvasser et al. The percentage of malignant tumors among those with a ccLS≥4 in our sample was 84% (range 40%, 100%). This high percentage of malignancy among masses with a ccLS≥4 can inform patients and urologists about the decision to treat without first performing a percutaneous renal mass biopsy.
However, a challenge in this decision-making is that the percentage of malignant masses among masses with a ccLS≥4 is close to the overall percentage of malignant masses without applying the ccLS (80%)(30, 31). Nevertheless, the overall benign nephrectomy rate at the population level may be further reduced using a proposed management strategy avoiding initial surgeries in patients with a ccLS≤2 (i.e., management with AS) and considering biopsy in ccLS3 SRMs (32). Such an approach would be supported by the known high prevalence (68%) of papillary RCCs among SRMs with a ccLS≤2 and oncocytic neoplasms among SRMs with a ccLS of 3 (38%) (23, 24, 33), the exceedingly low (<1%) incidence of metastatic disease development for these histological subtypes while undergoing active surveillance (7), and the reported improved 5-year disease-free survival of papillary RCC and chromophobe RCC compared to that of ccRCC after the treatment of the primary tumor (34).
However, this approach should consider patient factors such as age and competing comorbidities, and would require further validation prior to clinical acceptance. Additionally, increased experience in the use of the ccLS over 2 years and protocol standardization led to a higher reported rate of malignant disease among ccLS4 (93%) and ccLS5 (95%) SRMs at the institution where the ccLS was developed (33). Future multicenter studies may optimize the ccLS system to minimize the number of benign masses assigned a ccLS≥4. The moderate PPV (76%) of ccLS we observed was in the setting of a lower prevalence of ccRCC for SRMs in our study sample (48%), which was similar to the 50% found by Canvasser et al and in a series of 947 SRMs (31). This prevalence contrasts with the overall 60–88% prevalence of ccRCC in renal masses of any size (35, 36).
On average, 24% of SRMs had a ccLS≤2 (Table 4), which is lower than that reported previously (30–35%)(23, 24). Although 92% of ccLS≤2 SRMs were malignant in our study sample, they are expected to represent predominantly indolent neoplasms (24, 33). Indeed, almost 4 of 5 (79%) ccLS≤2 SRMs in our study were papillary RCC, chromophobe RCC or benign (oncocytoma or fat poor angiomyolipoma). Although papillary RCC (and rarely chromophobe RCC) may exhibit aggressive histology, these aggressive variants represent less than 5% of SRMs and are an uncommon (<1%) cause of metastatic disease (7).
Furthermore, the negative predictive value (NPV) of ccLS ≤2 to exclude ccRCC was 88% (95% CI: 81%, 93%) (vs. 93%, reported by Canvasser et al.). Thus, the decision to manage ccLS ≤2 SRMs with active surveillance without biopsy would lead to 12% of ccRCCs with that score being surveilled without a diagnosis. The decision to manage an SRM with active surveillance is multifactorial and heavily influenced by patient preference (i.e., risk tolerance), life expectancy, and competing comorbidities (37). The high NPV of ccLS ≤2 for ccRCC and the high prevalence of indolent histology for malignant masses assigned ccLS ≤2 may assist patients and urologists in the decision to proceed with active surveillance, renal mass biopsy, or therapeutic intervention.
Further improvements in the diagnosis of ccRCC may be achievable through better standardization of mpMRI techniques. While minimal technical requirements were part of our study protocol, substantial differences in the acquisition parameters were present across sites, which may have important implications. For example, the use of a lower flip angle (9–15 degrees) in spoiled gradient echo chemical shift acquisitions results in less T1 weighting and may lower the sensitivity to small amounts of microscopic fat (38), a major criterion of the ccLS algorithm; the absence of microscopic fat may have shifted some ccRCC masses from ccLS5 to ccLS4 and prevented upgrading to ccLS 3 in some SRMs. Johnson et al (24) used the same mpMRI protocol and standardized definitions for 63 masses assigned a prospective ccLS and reported an improved sensitivity, specificity, and PPV for ccRCC of 89%, 79%, and 84%, respectively, with a ccLS ≥4, and an improved NPV for ccRCC of 100% with a ccLS ≤2 compared to the original report by Canvasser et al (23), but the sample size was small.
The average interobserver agreement for ccLS scoring was moderate (Kappa = 0.58 [95% CI 0.42, 0.75]) and similar to the interobserver agreement reported by Canvasser et al (Kappa = 0.53 [range 0.38–0.64]) (23). This preserved agreement, despite a multi-center versus single-center study design (i.e., 10 vs 7 radiologists, 5 vs 2 sites, 1.5 hour training vs weekly case reviews), may be influenced by using only 2 radiologists per site in our study (compared to 7 radiologists at one institution in the study of Canvasser et al (23)) and the evolution of the ccLS system; specifically, the current version of the ccLS system provides clearer interpretation criteria and tie-breaking rules that were not present in earlier versions (22). While not directly comparable, the interobserver agreement for ccLS is similar to that of other accepted Likert-based reporting schemes, such as the Prostate Imaging Reporting and Data System (PI-RADS) v2 (39–41).
Our study has limitations. First, although our sample was created consecutively through a retrospective search across the five participating sites, selection bias is possible due to the retrospective design. For example, some SRMs may have been followed with active surveillance without histologic confirmation. A large number of potentially eligible renal masses were excluded; however, our final population showed a similar distribution of clear cell versus other tumors as reported previously (42). Second, although we required a degree of standardization of MRI protocols across institutions, since the database was procured retrospectively, differences in technique within the basic requirements of the study protocol occurred (e.g., different b values for DWI and different methods of contrast timing). Our logistic regression model included radiologist and site as random effects. The inclusion of tumor as a third random effect would be ideal; however, was not possible due to overfitting of the predicted model.
In conclusion, the reported single-center diagnostic performance of the clear cell renal cell carcinoma (ccRCC) likelihood score (ccLS) system for the evaluation of solid small renal masses was determined in a multicenter, multireader setting. This system differentiated ccRCC from other solid SRMs with moderate sensitivity, specificity, and positive predictive value. A ccLS ≤2 had a good negative predictive value for ccRCC. Moderate interobserver agreement was achieved with limited training, indicating the potential for the general adoption of the ccLS after further refinement of its diagnostic accuracy. Future studies are needed to assess the effect of technical parameters and interpretation criteria standardization on ccLS performance.
Supplementary Material
Figure 4.
MRI in 56-year old female with indeterminate 3.2 cm left renal mass. A) Coronal T2-weighted single shot fast spin echo demonstrates a left renal mass (arrow) with heterogeneous signal intensity greater than that of the adjacent renal cortex (asterisk). Note severe atrophy of both kidneys due to end-stage renal disease. B) Axial fat saturated T1-weighted spoiled gradient echo image acquired during the corticomedullary phase after administration of a bolus of gadolinium (0.1 mmol/kg of gadobenate dimeglumine) demonstrates intense heterogeneous enhancement, higher than that of the adjacent renal cortex. Quantitative analysis demonstrated >75% enhancement in the renal mass compared to the renal cortex. Axial T1-weighted gradient echo in-phase (C) and opposed-phase (D) images demonstrate unequivocal presence of microscopic fat in the mass, with apparent decrease signal intensity in (D) compared to (C). Both reviewers assigned a ccLS of 5 in this mass. After nephrectomy a diagnosis of clear cell renal cell carcinoma was confirmed.
Figure 5.
MRI in 64-year old male with indeterminate 2.6 cm left renal mass. A) Axial T2-weighted single shot fast spin echo demonstrates a left renal mass (arrow) with homogeneous signal intensity lower than that of the adjacent renal cortex (asterisk). B) Axial fat saturated T1-weighted spoiled gradient echo image acquired during the corticomedullary phase after administration of a bolus of gadolinium (0.1 mmol/kg of gadobenate dimeglumine) demonstrates homogeneous mild enhancement in the mass compared to the adjacent renal cortex. Quantitative analysis demonstrated <40% enhancement in the renal mass compared to the renal cortex. A ccLS of 1 was assigned by both readers and papillary renal cell carcinoma was confirmed after nephrectomy.
Figure 6.
MRI in 61-year old male with indeterminate 1.2 cm left renal mass. A) Axial T2-weighted single shot fast spin echo demonstrates a left renal mass (arrow) with slightly heterogeneous signal intensity that is greater than that of the adjacent renal cortex (asterisk). Note a simple cyst (C) in the right kidney. B) Axial fat saturated T1-weighted spoiled gradient echo image acquired during the corticomedullary phase after administration of a bolus of gadolinium (0.1 mmol/kg of gadobutrol) demonstrates intense heterogeneous enhancement, higher than that of the adjacent renal cortex. Quantitative analysis demonstrated >75% enhancement in the renal mass compared to the renal cortex. Axial T1-weighted gradient echo in-phase (C) and opposed-phase (D) images demonstrate no evidence of microscopic fat in the mass, with similar signal intensity in (D) compared to (C). Both reviewers assigned a ccLS of 4. After nephrectomy, a diagnosis of renal oncocytoma was confirmed.
Key Results.
In a retrospective review of 241 patients with 250 solid small renal masses (SRMs, ≤4 cm) undergoing multiparametric MRI, clear cell renal cell carcinoma (ccRCC) was found in 48%.
The MRI clear cell likelihood score (ccLS) differentiated ccRCC from other SRMs with moderate sensitivity (75%), specificity (78%), and positive predictive value (76%). Negative predictive value was 88%.
Mean interobserver agreement for the ccLS system was moderate (Kappa=0.58).
Abbreviation List
- RCC
Renal cell carcinoma
- ccRCC
Clear cell renal cell carcinoma
- mpMRI
Multiparametric MRI
- ccLS
Clear cell likelihood score
- SRM
Small renal mass
Contributor Information
Nicola Schieda, Department of Medical Imaging, The Ottawa Hospital, University of Ottawa. Ottawa, Ontario, Canada..
Matthew S Davenport, Department of Radiology, University of Michigan. Ann Arbor, MI, USA..
Stuart G. Silverman, Department of Radiology, Brigham and Women’s Hospital. Harvard Medical School Boston, MA..
Barun Bagga, Department of Radiology, NYU Langone Medical Center. New York, NY, USA.
Daniel Barkmeier, Department of Radiology, University of Michigan. Ann Arbor, MI, USA.
Zane Blank, Department of Radiology. University of Nebraska Medical Center. Omaha, Nebraska.
Nicole E Curci, Department of Radiology, University of Michigan. Ann Arbor, MI, USA..
Ankur Doshi, Department of Radiology. NYU Langone Medical Center. New York, NY, USA
Ryan Downey, Department of Radiology. University of Nebraska Medical Center. Omaha, Nebraska.
Elizabeth Edney, Department of Radiology. University of Nebraska Medical Center. Omaha, Nebraska.
Elon Granader, Department of Radiology. University of Nebraska Medical Center. Omaha, Nebraska..
Isha Gujrathi, Department of Radiology, Brigham and Women’s Hospital. Harvard Medical School Boston, MA..
Rebecca M. Hibbert, Department of Medical Imaging, The Ottawa Hospital, University of Ottawa. Ottawa, Ontario, Canada..
Nicole Hindman, Department of Radiology. NYU Langone Medical Center, New York, NY, USA..
Cynthia Walsh, Department of Medical Imaging, The Ottawa Hospital, University of Ottawa. Ottawa, Ontario, Canada..
Tim Ramsay, Ottawa Hospital Research Institute. Ottawa, Ontario, Canada..
Atul B. Shinagare, Department of Radiology, Brigham and Women’s Hospital. Harvard Medical School Boston, MA..
Ivan Pedrosa, University of Texas Southwestern Medical Center. Dallas, TX.
References
- 1.Meyer HJ, Pfeil A, Schramm D, Bach AG, Surov A. Renal incidental findings on computed tomography: Frequency and distribution in a large non selected cohort. Medicine (Baltimore) 2017;96(26):e7039. doi: 10.1097/MD.0000000000007039 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.O’Connor SD, Pickhardt PJ, Kim DH, Oliva MR, Silverman SG. Incidental finding of renal masses at unenhanced CT: prevalence and analysis of features for guiding management. AJR Am J Roentgenol 2011;197(1):139–145. doi: 10.2214/AJR.10.5920 [DOI] [PubMed] [Google Scholar]
- 3.Remzi M, Ozsoy M, Klingler HC, Susani M, Waldert M, Seitz C, Schmidbauer J, Marberger M. Are small renal tumors harmless? Analysis of histopathological features according to tumors 4 cm or less in diameter. J Urol 2006;176(3):896–899. doi: S0022–5347(06)01076–7 [pii] 10.1016/j.juro.2006.04.047 [DOI] [PubMed] [Google Scholar]
- 4.Pahernik S, Ziegler S, Roos F, Melchior SW, Thuroff JW. Small renal tumors: correlation of clinical and pathological features with tumor size. J Urol 2007;178(2):414–417; discussion 416–417. doi: 10.1016/j.juro.2007.03.129 [DOI] [PubMed] [Google Scholar]
- 5.Lee H, Lee JK, Kim K, Kwak C, Kim HH, Byun S-S, Lee SE, Hong SK. Risk of metastasis for T1a renal cell carcinoma. World Journal of Urology 2016;34(4):553–559. doi: 10.1007/s00345-015-1659-4 [DOI] [PubMed] [Google Scholar]
- 6.Bianchi M, Gandaglia G, Trinh QD, Hansen J, Becker A, Abdollah F, Tian Z, Lughezzani G, Roghmann F, Briganti A, Montorsi F, Karakiewicz PI, Sun M. A population-based competing-risks analysis of survival after nephrectomy for renal cell carcinoma. Urologic oncology 2014;32(1):46.e41–47. doi: 10.1016/j.urolonc.2013.06.010 [DOI] [PubMed] [Google Scholar]
- 7.Finelli A, Cheung DC, Al-Matar A, Evans AJ, Morash CG, Pautler SE, Siemens DR, Tanguay S, Rendon RA, Gleave ME, Drachenberg DE, Chin JL, Fleshner NE, Haider MA, Kachura JR, Sykes J, Jewett MAS. Small Renal Mass Surveillance: Histology-specific Growth Rates in a Biopsy-characterized Cohort. Eur Urol 2020;78(3):460–467. doi: 10.1016/j.eururo.2020.06.053 [DOI] [PubMed] [Google Scholar]
- 8.Marconi L, Dabestani S, Lam TB, Hofmann F, Stewart F, Norrie J, Bex A, Bensalah K, Canfield SE, Hora M, Kuczyk MA, Merseburger AS, Mulders PFA, Powles T, Staehler M, Ljungberg B, Volpe A. Systematic Review and Meta-analysis of Diagnostic Accuracy of Percutaneous Renal Tumour Biopsy. Eur Urol 2016;69(4):660–673. doi: 10.1016/j.eururo.2015.07.072 [DOI] [PubMed] [Google Scholar]
- 9.Lim CS, Schieda N, Silverman SG. Update on Indications for Percutaneous Renal Mass Biopsy in the Era of Advanced CT and MRI. AJR Am J Roentgenol 2019:1–10. doi: 10.2214/AJR.19.21093 [DOI] [PubMed] [Google Scholar]
- 10.Richard PO, Martin L, Lavallée LT, Violette PD, Komisarenko M, Evans AJ, Jain K, Jewett MAS, Finelli A. Identifying the use and barriers to the adoption of renal tumour biopsy in the management of small renal masses. Canadian Urological Association journal = Journal de l’Association des urologues du Canada 2018;12(8):260–266. doi: 10.5489/cuaj.5065 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Herts BR, Coll DM, Novick AC, Obuchowski N, Linnell G, Wirth SL, Baker ME. Enhancement characteristics of papillary renal neoplasms revealed on triphasic helical CT of the kidneys. AJR Am J Roentgenol 2002;178(2):367–372. doi: 10.2214/ajr.178.2.1780367 [DOI] [PubMed] [Google Scholar]
- 12.Sasiwimonphan K, Takahashi N, Leibovich BC, Carter RE, Atwell TD, Kawashima A. Small (<4 cm) renal mass: differentiation of angiomyolipoma without visible fat from renal cell carcinoma utilizing MR imaging. Radiology 2012;263(1):160–168. doi: 10.1148/radiol.12111205 [DOI] [PubMed] [Google Scholar]
- 13.Sun MR, Ngo L, Genega EM, Atkins MB, Finn ME, Rofsky NM, Pedrosa I. Renal cell carcinoma: dynamic contrast-enhanced MR imaging for differentiation of tumor subtypes--correlation with pathologic findings. Radiology 2009;250(3):793–802. doi: 10.1148/radiol.2503080995 [DOI] [PubMed] [Google Scholar]
- 14.Young JR, Margolis D, Sauk S, Pantuck AJ, Sayre J, Raman SS. Clear cell renal cell carcinoma: discrimination from other renal cell carcinoma subtypes and oncocytoma at multiphasic multidetector CT. Radiology 2013;267(2):444–453. doi: 10.1148/radiol.13112617 radiol.13112617 [pii] [DOI] [PubMed] [Google Scholar]
- 15.Zhang J, Lefkowitz RA, Ishill NM, Wang L, Moskowitz CS, Russo P, Eisenberg H, Hricak H. Solid renal cortical tumors: differentiation with CT. Radiology 2007;244(2):494–504. doi: 10.1148/radiol.2442060927 [DOI] [PubMed] [Google Scholar]
- 16.Chiarello MA, Mali RD, Kang SK. Diagnostic Accuracy of MRI for Detection of Papillary Renal Cell Carcinoma: A Systematic Review and Meta-Analysis. AJR Am J Roentgenol 2018;211(4):812–821. doi: 10.2214/AJR.17.19462 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Galmiche C, Bernhard JC, Yacoub M, Ravaud A, Grenier N, Cornelis F. Is Multiparametric MRI Useful for Differentiating Oncocytomas From Chromophobe Renal Cell Carcinomas? AJR Am J Roentgenol 2017;208(2):343–350. doi: 10.2214/AJR.16.16832 [DOI] [PubMed] [Google Scholar]
- 18.Rosenkrantz AB, Hindman N, Fitzgerald EF, Niver BE, Melamed J, Babb JS. MRI features of renal oncocytoma and chromophobe renal cell carcinoma. AJR Am J Roentgenol 2010;195(6):W421–427. doi: 10.2214/AJR.10.4718195/6/W421 [pii] [DOI] [PubMed] [Google Scholar]
- 19.Young JR, Coy H, Kim HJ, Douek M, Lo P, Pantuck AJ, Raman SS. Performance of Relative Enhancement on Multiphasic MRI for the Differentiation of Clear Cell Renal Cell Carcinoma (RCC) From Papillary and Chromophobe RCC Subtypes and Oncocytoma. AJR Am J Roentgenol 2017;208(4):812–819. doi: 10.2214/AJR.16.17152 [DOI] [PubMed] [Google Scholar]
- 20.Silverman SG, Pedrosa I, Ellis JH, Hindman NM, Schieda N, Smith AD, Remer EM, Shinagare AB, Curci NE, Raman SS, Wells SA, Kaffenberger SD, Wang ZJ, Chandarana H, Davenport MS. Bosniak Classification of Cystic Renal Masses, Version 2019: An Update Proposal and Needs Assessment. Radiology 2019:182646. doi: 10.1148/radiol.2019182646 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kay FU, Canvasser NE, Xi Y, Pinho DF, Costa DN, Diaz de Leon A, Khatri G, Leyendecker JR, Yokoo T, Lay AH, Kavoussi N, Koseoglu E, Cadeddu JA, Pedrosa I. Diagnostic Performance and Interreader Agreement of a Standardized MR Imaging Approach in the Prediction of Small Renal Mass Histology. Radiology 2018;287(2):543–553. doi: 10.1148/radiol.2018171557 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Pedrosa I, Cadeddu JA. How I Do It: Managing the Indeterminate Renal Mass with the MRI Clear Cell Likelihood Score. Radiology in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Canvasser NE, Kay FU, Xi Y, Pinho DF, Costa D, de Leon AD, Khatri G, Leyendecker JR, Yokoo T, Lay A, Kavoussi N, Koseoglu E, Cadeddu JA, Pedrosa I. Diagnostic Accuracy of Multiparametric Magnetic Resonance Imaging to Identify Clear Cell Renal Cell Carcinoma in cT1a Renal Masses. J Urol 2017;198(4):780–786. doi: 10.1016/j.juro.2017.04.089 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Johnson BA, Kim S, Steinberg RL, de Leon AD, Pedrosa I, Cadeddu JA. Diagnostic performance of prospectively assigned clear cell Likelihood scores (ccLS) in small renal masses at multiparametric magnetic resonance imaging. Urologic oncology 2019;37(12):941–946. doi: 10.1016/j.urolonc.2019.07.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Wang ZJ, Davenport MS, Silverman SG, Chandarana H, Doshi A, Israel GM, Leyendecker JR, Pedrosa I, Raman S, Remer EM, Shinagare AB, Smith AD, Vikram R. MR renal mass protocols v1.0. https://abdominalradiologyorg/wp-content/uploads/2020/11/RCCMRIprotocolfinal-7-15-17pdf (accesed March 3, 2021). [Google Scholar]
- 26.Ramamurthy NK, Moosavi B, McInnes MD, Flood TA, Schieda N. Multiparametric MRI of solid renal masses: pearls and pitfalls. Clin Radiol 2015;70(3):304–316. doi: 10.1016/j.crad.2014.10.006 [DOI] [PubMed] [Google Scholar]
- 27.Krishna S, Schieda N, Flood TA, Shanbhogue AK, Ramanathan S, Siegelman E. Magnetic resonance imaging (MRI) of the renal sinus. Abdom Radiol (NY) 2018;43(11):3082–3100. doi: 10.1007/s00261-018-1593-1 [DOI] [PubMed] [Google Scholar]
- 28.Wang ZJ, Davenport MS, Silverman SG, Chandarana H, Doshi A, Israel GM, Leyendecker JR, Pedrosa I, Raman S, Remer EM, Shinagare AB, Smith AD, Vikram R. MRI renal mass protocol v1.0: Society of Abdominal Radiology Disease Focused Panel on Renal Cell Carcinoma. [online]. Available: https://cdn.ymaws.com/www.abdominalradiology.org/resource/resmgr/education_dfp/RCC/RCC.MRIprotocolfinal-7-15-17.pdf/ [Accessed 07-Apr-2020].
- 29.Nofuentes JAR, Porcel MdCO. Average kappa coefficient: a new measure to assess a binary test considering the losses associated with an erroneous classification. Journal of Statistical Computation and Simulation 2015;85(8):1601–1620. doi: 10.1080/00949655.2014.881816 [DOI] [Google Scholar]
- 30.Duchene DA, Lotan Y, Cadeddu JA, Sagalowsky AI, Koeneman KS. Histopathology of surgically managed renal tumors: analysis of a contemporary series. Urology 2003;62(5):827–830. doi: 10.1016/s0090-4295(03)00658-7 [DOI] [PubMed] [Google Scholar]
- 31.Frank I, Blute ML, Cheville JC, Lohse CM, Weaver AL, Zincke H. Solid renal tumors: an analysis of pathological features related to tumor size. J Urol 2003;170(6 Pt 1):2217–2220. doi: 10.1097/01.ju.0000095475.12515.5e [DOI] [PubMed] [Google Scholar]
- 32.Diaz de Leon A, Davenport MS, Silverman SG, Schieda N, Cadeddu JA, Pedrosa I. Role of Virtual Biopsy in the Management of Renal Masses. AJR Am J Roentgenol 2019:1–10. doi: 10.2214/AJR.19.21172 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Steinberg RL, Rasmussen RG, Johnson BA, Ghandour R, De Leon AD, Xi Y, Yokoo T, Kim S, Kapur P, Cadeddu JA, Pedrosa I. Prospective performance of clear cell likelihood scores (ccLS) in renal masses evaluated with multiparametric magnetic resonance imaging. Eur Radiol 2021;31(1):314–324. doi: 10.1007/s00330-020-07093-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Beck SD, Patel MI, Snyder ME, Kattan MW, Motzer RJ, Reuter VE, Russo P. Effect of papillary and chromophobe cell type on disease-free survival after nephrectomy for renal cell carcinoma. Ann Surg Oncol 2004;11(1):71–77. [DOI] [PubMed] [Google Scholar]
- 35.Rothman J, Egleston B, Wong YN, Iffrig K, Lebovitch S, Uzzo RG. Histopathological characteristics of localized renal cell carcinoma correlate with tumor size: a SEER analysis. J Urol 2009;181(1):29–33; discussion 33–24. doi: 10.1016/j.juro.2008.09.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Trpkov K, Hes O, Williamson SR, Adeniran AJ, Agaimy A, Alaghehbandan R, Amin MB, Argani P, Chen YB, Cheng L, Epstein JI, Cheville JC, Comperat E, da Cunha IW, Gordetsky JB, Gupta S, He H, Hirsch MS, Humphrey PA, Kapur P, Kojima F, Lopez JI, Maclean F, Magi-Galluzzi C, McKenney JK, Mehra R, Menon S, Netto GJ, Przybycin CG, Rao P, Rao Q, Reuter VE, Saleeb RM, Shah RB, Smith SC, Tickoo S, Tretiakova MS, True L, Verkarre V, Wobker SE, Zhou M, Gill AJ. New developments in existing WHO entities and evolving molecular concepts: The Genitourinary Pathology Society (GUPS) update on renal neoplasia. Mod Pathol 2021. doi: 10.1038/s41379-021-00779-w [DOI] [PubMed] [Google Scholar]
- 37.Schieda N KS, Pedrosa I, Davenport MS, Kaffenberger S, Silverman SG. Active Surveillance of Renal Masses : The Role of Radiology. Radiology 2021. [DOI] [PubMed] [Google Scholar]
- 38.Marin D, Soher BJ, Dale BM, Boll DT, Youngblood RS, Merkle EM. Characterization of adrenal lesions: comparison of 2D and 3D dual gradient-echo MR imaging at 3 T--preliminary results. Radiology 2010;254(1):179–187. doi: 10.1148/radiol.09090486 [DOI] [PubMed] [Google Scholar]
- 39.Greer MD, Shih JH, Lay N, Barrett T, Bittencourt L, Borofsky S, Kabakus I, Law YM, Marko J, Shebel H, Merino MJ, Wood BJ, Pinto PA, Summers RM, Choyke PL, Turkbey B. Interreader Variability of Prostate Imaging Reporting and Data System Version 2 in Detecting and Assessing Prostate Cancer Lesions at Prostate MRI. AJR Am J Roentgenol 2019:1–8. doi: 10.2214/AJR.18.20536 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Muller BG, Shih JH, Sankineni S, Marko J, Rais-Bahrami S, George AK, de la Rosette JJ, Merino MJ, Wood BJ, Pinto P, Choyke PL, Turkbey B. Prostate Cancer: Interobserver Agreement and Accuracy with the Revised Prostate Imaging Reporting and Data System at Multiparametric MR Imaging. Radiology 2015;277(3):741–750. doi: 10.1148/radiol.2015142818 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Mussi TC, Yamauchi FI, Tridente CF, Tachibana A, Tonso VM, Recchimuzzi DR, de Souza Leao LR, Luz DC, Martins T, Baroni RH. Interobserver Agreement and Positivity of PI-RADS Version 2 Among Radiologists with Different Levels of Experience. Acad Radiol 2019;26(8):1017–1022. doi: 10.1016/j.acra.2018.08.013 [DOI] [PubMed] [Google Scholar]
- 42.Bhindi B, Thompson RH, Lohse CM, Mason RJ, Frank I, Costello BA, Potretzke AM, Hartman RP, Potretzke TA, Boorjian SA, Cheville JC, Leibovich BC. The Probability of Aggressive Versus Indolent Histology Based on Renal Tumor Size: Implications for Surveillance and Treatment. Eur Urol 2018;74(4):489–497. doi: 10.1016/j.eururo.2018.06.003 [DOI] [PubMed] [Google Scholar]
- 43.Higgins JPT GS. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2011. [Google Scholar]
- 44.Moch H, Cubilla AL, Humphrey PA, Reuter VE, Ulbright TM. The 2016 WHO Classification of Tumours of the Urinary System and Male Genital Organs-Part A: Renal, Penile, and Testicular Tumours. Eur Urol 2016;70(1):93–105. doi: 10.1016/j.eururo.2016.02.029 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.