Diagnostic Accuracy of Magnetic Resonance Imaging (MRI) 5-Point Likert Scoring System Evaluated by The Result of MRI/Ultrasonography Image-fusion Targeted Biopsy of The Prostate

Abstract

Objective

To evaluate the accuracy of MRI based Likert scoring system in detection of clinically significant prostate cancer (CSPC) using MRI/Ultrasonography (US) image-fusion targeted biopsy (FTB) as a reference standard.

Patients and Methods

We retrospectively reviewed 1218 MRI-lesions in 629 patients who underwent subsequent MRI/US FTB between 10/2012 and 8/2015. 3-Tesla MRI was independently reported by 1 of 8 radiologists with varying levels of experience and scored on a 5-point Likert scale. All of lesions with Likert 1–5 were prospectively defined as targets for MRI/US FTB. CSPC was defined as Gleason score ≥7.

Results

Median patient age was 64 years, PSA level was 6.97ng/ml and estimated prostate volume was 52.2ml. Of 1218 lesions, 48% (n=581) were rated as Likert 1–2, 35% (n=428) were Likert 3 and 17% (n=209) were Likert 4–5. According to the Likert system of grading from 1 to 5, overall cancer detection rate were 12%, 13%, 22%, 50%, 59%, and CSPC detection rate were 4%, 4%, 12%, 33%, 48%, respectively. Grading of a 5-point scale showed strong positive correlation with overall cancer detection rate (r=0.949, p=0.05) and CSPC detection rate (r=0.944, p=0.05). In comparison between the more experienced radiologists for MRI-prostate and less experienced radiologists, statistical differences were noted in overall cancer detection rate (63% vs 35%, p=0.001) and CSPC detection rate (47% vs 29%, p=0.027) in Likert 4–5 lesions.

Conclusions

The detection rates of overall cancer and CSPC strongly correlated with a 5-point grading of the Likert scale. Among the radiologists with different levels of experience, there were significant differences in these cancer detection rates.

INTRODUCTION

MRI provides the best visualization of the prostate compared to other imaging modalities. Advances in MRI, such as a multi-parametric (mp) approach, show promise for improved detection and characterization of prostate cancer. Moreover, recent evidence suggests that mpMRI may better visualize clinically significant prostate cancer (CSPC) [1]. However, one major impediment to the promotion of mpMRI-prostate is a lack of standardization in the expression of results [2]. MpMRI data need to be presented to clinical colleagues in a simple but reliable standardized way, preferably using a structured reporting scheme [3]. Although a 5-point Likert scoring system and the MR Prostate Imaging-Reporting and Data System (PI-RADS) are often used to evaluate mpMRI of the prostate, controversy still exists on the best way to report [3, 4]. In addition, the interpretation capability of MRI-prostate is a critical clinical problem, which may depend on the experience of radiologist for MRI-prostate.

Recently, increasing evidence supports the use of MRI/Ultrasonography (US) image-fusion targeted biopsy (FTB) to improve the detection of CSPCs while limiting detection of indolent cancers compared to conventional systematic random biopsy [5–7].

However, data regarding the association between all of Likert scores (1–5) and the presence of CSPC and the use of MRI/US FTB to validate these scores are lacking.

Our purpose of this study was therefore to evaluate the diagnostic accuracy of a 5-point Likert scoring system rated by radiologists with varying levels of experience in detection of prostate cancer using MRI/US FTB as a reference standard and to determine the potential ability of mpMRI to identify CSPC.

PATIENTS AND METHODS

After verification from the Institutional Review Board, we retrospectively reviewed 762 patients who underwent mpMRI of the prostate and subsequent MRI/US FTB at Chesapeake Urology Associates between October 2012 and August 2015. A flow chart of the number of men who were suitable for study inclusion is presented in Figure 1. We excluded patients in whom MRI did not identify any visible lesions and patients who had received any prior treatments for prostate cancer. We also excluded patients whose MRI were reported by novice radiologists who had only minimum experience of MRI-prostate in less than 20 cases and patients whose biopsies were performed by novice urologists who had only minimum experience of MRI/US FTB in less than 20 cases during the study period. Finally, total 1218 MRI-lesions in 629 patients were included in this study.

Figure 1.

Schematic tree of study cohort.

MpMRI was performed using a 3-Tesla magnetic field strength and a pelvic phased-array coil. T1-weighted, T2-weighted, diffusion-weighted and dynamic gadolinium contrast-enhanced imaging sequences including the calculation of apparent diffusion co-efficient (ADC) maps were acquired. Each MRI was independently interpreted by 1 of 8 radiologists with varying levels of experience for mpMRI of the prostate in the clinical practice setting and not blinded to the clinical context. Each lesion was assigned a 5-point Likert scale score (Table 1) [4] by the interpreting radiologist and all of lesions with Likert 1–5 were defined as targets for MRI/US FTB. For detailed analysis, radiologists were grouped into experienced (3 radiologists who had read independently more than 80 mpMRIs of the prostate during the study period) and less experienced (3 radiologists who had read independently less than 50 mpMRIs) readers.

Table 1. Likert five-grade scoring system.

Likert score
1	Clinically significant cancer is highly unlikely to be present
2	Clinically significant cancer is unlikely to be present
3	Clinically significant cancer is equivocal
4	Clinically significant cancer is likely to be present
5	Clinically significant cancer is highly likely to be present

Standard random biopsy was performed with conventional systematic 10–12 cores per patient and targeted biopsy was performed with at least 1 core per lesion (all of lesions with Likert 1–5) under general or local anesthesia. Prostate biopsy for each patient was independently performed by 1 of 4 urologists with varying levels of experience for MRI/US FTB throughout the study period. 3D volume data of mpMRI and real-time trans-rectal US images were visualized on a screen of computer workstation, the UroStation (Koelis, Grenoble, France) and used for MRI/US FTB [8]. Urologists were grouped into experienced (2 urologists who had performed MRI/US FTB independently in more than 120 cases during the study period) and less experienced urologists.

The definition for CSPC was set at Gleason score equal to or greater than 3+4 in this study.

Statistical analyses were performed using SPSS Statistics version 23 (IBM Corporation, Armonk, New York) software. The results were considered significant at p<0.05.

RESULTS

Characteristics of the patients are shown in Table 2. In 629 patients, median age was 64.0 years, pre-biopsy PSA level was 6.97ng/ml and estimated prostate volume was 52.2ml. The median number of lesions on MRI was 2.0 per patient. The study population included the spectrum of men offered prostate biopsy, including 1) biopsy naïve men with a clinical suspicion of prostate cancer based on increased PSA and/or abnormal DRE (n=83, 13.2%), 2) men with previous negative biopsy with persistent clinical suspicion of prostate cancer based on increased PSA and/or abnormal DRE (n=274, 43.6%) and 3) men with histologically proven cancer on an active surveillance protocol (n=177, 28.1%) or previous atypical cells (n=85, 13.5%).

Table 2. Patient characteristics (n=629).

	Median	Range
Age (years)	64.0	30–87
Pre-biopsy PSA (ng/ml)	6.97	0.2–61.6
Estimated prostate volume (ml)	52.2	14.5–247.6
Number of lesions on MRI / patient	2.0	1–5
	n	%
Race
Caucasian	502	79.8%
African-American	106	16.9%
Asian	10	1.6%
Others or unknown	11	1.7%
Reason for biopsy
Rising PSA - previous negative biopsy	274	43.6%
Active surveillance - prostate cancer	177	28.1%
Elevated PSA - biopsy naïve	83	13.2%
Atypia on previous biopsy	85	13.5%
Others	10	1.6%

Of 1218 lesions on MRI, 48% (n=581) were rated as Likert 1–2 (lower suspicion), 35% (n=428) were Likert 3(equivocal suspicion) and 17% (n=209) were Likert 4–5 (higher suspicion) (Figure 2).

Figure 2.

Likert scores of 1218 lesions on MRI.

According to a 5-point Likert system of grading from 1 to 5, overall cancer detection rate were 12%, 13%, 22%, 50%, 59% (Figure 3(A)), and CSPC detection rate were 4%, 4%, 12%, 33%, 48% (Figure 3(B)), respectively. Grading of a 5-point scale showed strong positive correlation with overall cancer detection rate (r=0.949, p=0.05) and CSPC detection rate (r=0.944, p=0.05).

Figure 3 (A).

Overall cancer detection rate. Grading of a 5-point Likert scale on MRI showed strong positive correlation with overall cancer detection rate (r=0.949, p=0.05).

Figure 3 (B).

Clinically significant cancer detection rate. Grading of a 5-point Likert scale on MRI showed strong positive correlation with clinically significant cancer detection rate (r=0.944, p=0.05).

In addition, according to the Likert system of grading from 1 to 5, proportion of clinically insignificant cancer cores in all cancer positive cores were 66.7%, 74.1%, 46.8%, 34.6%, 19.4%, respectively (p<0.001).Thus, grading of a 5-point Likert scale showed strong negative correlation with the proportion of clinically insignificant cancers (r= −0.941, p=0.05).

In comparison between Likert 4–5 lesions (higher suspicion) and Likert 3 lesions (equivocal suspicion), the targeted biopsies from Likert 4–5 lesions apparently showed a higher overall cancer detection rate (55% vs 22%, p<0.001) and CSPC detection rate (40% vs 12%, p<0.001), respectively (Table 3).

Table 3. Overall cancer detection rate and clinically significant cancer detection rate in Likert 3 versus Likert 4–5 lesions.

Likert score	3 (n=428)	4 (n=104)	5 (n=105)
Overall cancer detection rate	22.0% (94/428)	54.5% (114/209)		p<0.001
Clinically significant cancer detection rate	11.7% (50/428)	40.2% (84/209)		p<0.001

Significant association between the radiologist’s experience for MRI-prostate and the detection rates of overall cancer and CSPC was identified in Likert 4–5 lesions. In comparison between 3 more experienced radiologists for mpMRI-prostate and 3 less experienced radiologists, statistical differences were noted in overall cancer detection rate (63% vs 35%, p=0.001) and CSPC detection rate (47% vs 29%, p=0.027), respectively (Table 4).

Table 4. Overall cancer detection rate and clinically significant cancer detection rate between 2 radiologist groups in Likert 4–5 lesions.

	3 more experienced radiologists	3 less experienced radiologists
Overall cancer detection rate	62.7% (69/110)	34.7% (17/49)	p=0.001
Significant cancer detection rate	47.3% (52/110)	28.6% (14/49)	p=0.027

In contrast, no significant association between the urologist’s experience for MRI/US FTB and the detection rates of overall cancer and CSPC was identified (Table 5).

Table 5. Overall cancer detection rate and clinically significant cancer detection rate between 2 urologist groups in Likert 4–5 lesions.

	2 more experienced urologists	2 less experienced urologists
Overall cancer detection rate	56.8% (92/162)	46.8% (22/47)	p=0.226
Significant cancer detection rate	39.5% (64/162)	42.6% (20/47)	p=0.708

Finally, Table 6 shows a comparison between conventional random biopsy and MRI/US FTB. A higher proportion of cores were positive for any cancer using MRI/US FTB than random biopsy (25.6% vs 8.5%, p<0.001). Regarding the proportion of clinically insignificant cancer cores in all cancer positive cores, there was a significant difference between random biopsy and MRI/US FTB (48.3% vs 31.9%, p<0.001). The median cancer core length with MRI/US FTB was significantly greater than that of random biopsy (5.0 mm vs 2.0 mm, p<0.001). A higher proportion of patients were positive for CSPC using MRI/US FTB than random biopsy (20.7% vs 14.9%, p=0.009). Moreover, random biopsy missed CSPC in 66 of 629 patients (10.5%), while MRI/US FTB missed it in only 1.6% (n=10) (p<0.001).

Table 6. Comparison between random biopsy and targeted biopsy.

	Random biopsy	Targeted biopsy for index lesion
Total cores	5825 cores	1666 cores
Median number of cores per patient (range)	10.0 cores (0–20)	2.0 cores (1–8)
Cancer positive cores	495 cores	426 cores
Positive for any cancer per core	8.5% (495/5825 cores)	25.6% (426/1666 cores)	p<0.001
Proportion of clinically insignificant cancer cores in all cancer positive cores	48.3% (239/495 cores)	31.9% (136/426 cores)	p<0.001
Median cancer core length (range)	2.0 mm (0.2–17)	5.0 mm (0.16–19)	p<0.001
Positive for clinically significant cancer per patient	14.9% (91/609 patients)	20.7% (130/629 patients)	p=0.009
	Positive only with random biopsy	Positive only with targeted biopsy
Clinically significant cancer	1.6% (10/629 patients)	10.5% (66/629 patients)	p<0.001

DISCUSSION

To our best of knowledge, the present study is the first to demonstrate that the experience level of radiologist for MRI-prostate interpretation using a Likert scale affected the detection of CSPC in trans-rectal MRI/US FTB. We used the number of cases in which each radiologist interpreted and reported mpMRI of the prostate during the study period as an indicator of the experience level for current mpMRI of the prostate. We recognize an important evolving role and a rapid progress of MRI methodologies in assessment of prostate cancer. We therefore believe the actual experience of the study period is more important than the previous life time experience and strongly reflects the ability to adjust to new advances in MRI technology and to learn new improved skills in interpretation of current mpMRI.

Recently, Gaziev et al. reported the accuracy of mpMRI during the learning curve of radiologists using transperineal MRI/US FTB for validation [9]. The cohorts were divided into groups representing five consecutive time intervals in the study. As a result, the overall cancer detection rates for Likert 4–5 lesions were 31.5% in the first cohort and 70.5% in the final cohort. In our present study, the overall cancer detection rates for Likert 4–5 lesions were 34.7% in less experienced radiologists and 62.7% in more experienced radiologists (p=0.001). This is comparable to the previous paper, although direct comparison is difficult because they used the different platform for image-fusion and transperineal approach.

In order to enhance the detection of CSPC in MRI/US FTB, proper interpretation of mpMRI is essential and the standardization, training and education of mpMRI-prostate interpretation are important [10]. As such, it was recently reported that the current PI-RADS version 2 may simplify and standardize the terminology and content of radiology reports, and also educate radiologists on MRI-prostate reporting and reduce variability in imaging interpretations [11]. In addition, a histopathological feedback to radiologists from urologists or pathologists may help in better standardization and improvement of mpMRI interpretation [12].

To express the result of mpMRI-prostate, the PREDICT (Prostate Diagnostic Imaging Consensus Meeting) panel recommended use of a 5-point Likert scale [2]. On the other hand, the European Society of Urogenital Radiology (ESUR) proposed use of the PI-RADS [3]. Noticeably the Likert scale approach does not recommend fixed criteria for interpretation, yet has been found to perform better than the PI-RADS for the detection of prostate cancer. Although the 5-point Likert scale scoring and PI-RADS are well-known reporting systems of MRI-prostate, controversy still exists on the best way to report [11]. Rosenkrantz AB et al. [13] reported that diagnostic accuracy was similar for the PI-RADS version 1 (v1) and Likert scales in the peripheral zone but was somewhat higher for the Likert scale than for the PI-RADS v1 scale in the transition zone. In contrast, Roethke MC et al. [14] reported that the aggregated PI-RADS v1 score was more valid than the Likert score.

In the present study, similar to the other reports [15], the 5-point Likert scale strongly correlated with overall cancer detection rate (r=0.949) and CSPC detection rate (r=0.944). Higher suspicious scores (Likert 4–5) on MRI correlated strongly with a higher likelihood of overall cancer (55%) and CSPC (40%). Thus, targeted biopsy should be considered for the Likert 4–5 lesions. In contrast, lower suspicion scores (Likert 1–2) on MRI may be useful in predicting low likelihood of high grade cancer, so targeted biopsy could potentially be avoided. Likert 3 is still equivocal and needs to be distinguished in further investigation.

In contrast, the present study showed no significant difference in detection rates of overall cancer and CSPC among the urologists with different levels of experience for MRI/US FTB. These findings may in part reflect that a software-based coregistration tool such as the UroStation may reduce issues of operator experience with visual targeting [8, 16].

A major concern related to prostate cancer screening and early detection is over-diagnosis and over-treatment of indolent disease [17]. Strategies to reduce over-diagnosis are necessary, as are strategies to differentiate indolent from aggressive tumors. In other words, an ideal biopsy strategy to identify men with prostate cancer would be detection of only CSPC and minimization of insignificant cancer detection and consequent over-treatment.

The potential of mpMRI and subsequent MRI/US FTB for improved detection of CSPC and reduction in unnecessary biopsies of insignificant or absent prostate cancer may be promising, but is still being explored [5, 16, 18]. The results of our present study showed that MRI/US FTB identified more CSPCs with much less biopsy cores compared to conventional random biopsy. A recent systematic review reported that MRI/US FTB found CSPCs (median 9.1%) missed by standard random biopsy alone [19]. In the present study, MRI/US FTB found 66 patients (10.5%) with CSPC missed by random biopsy alone.Although the detection rate has varied among previous studies, it was reported that the absolute difference in CSPC detection rate between MRI/US FTB and standard random biopsy was a median of 6.8% (range: 0.9 – 41.4%) [19], and the result of our present study was 5.8%. Moreover, median cancer core length of positive MRI/US FTB was 2.5 times greater than that of random biopsies (5 mm vs 2 mm). In addition, we have recently reported that MRI/US FTB allowed accurate identification of the index lesion, which was defined as a lesion with the highest Gleason score or the largest volume or extraprostatic extension [20].

These results may imply that mpMRI and subsequent MRI/US FTB efficiently differentiate aggressive from indolent cancers, and reduce over-diagnosis and over-treatment of indolent cancers. Furthermore, this procedure may also become a useful tool for precise cancer mapping in focal therapy of prostate cancer.

Our present study has a number of limitations.

First, analysis was retrospective and the study population was heterogeneous, comprising biopsy naïve men, men with prior negative biopsy and men with prior positive biopsy. Therefore, it suffers from potential for selection biases. However, the main objective of the present study was to evaluate the accuracy of the Likert scale itself scored by radiologists regardless of the reason for biopsy.

Second, our reference standard is a biopsy rather than a final prostatectomy specimen, so we cannot completely validate our scoring accuracy and determine the actual significance of a negative biopsy. On the other hand, when whole-mount radical prostatectomy has been used as the reference standard, the study only evaluates patients who underwent surgery and has a verification bias. We therefore chose to use MRI/US FTB as our reference standard.

Third, our analysis excluded patients without any visible lesions on MRI, so we cannot assess the detection of CSPC in such patients.

Fourth, the definition we used to indicate CSPC is open to debate because no universally accepted definition exists. Therefore, we performed a second analysis using other definition of CSPC (Gleason score ≥4+3 or a maximum cancer core length 6mm or longer), which was used in PROMIS study [21]. In the second analysis, according to the Likert system of grading from 1 to 5, CSPC detection rate were 3.4%, 3.7%, 9.1%, 33.7%, 42.9%, respectively (r=0.934, p=0.05). These results were very similar to our first analysis.

Lastly, the extra costs and efforts associated with mpMRI and MRI/US FTB should be further investigated with consideration of possible cost savings due to reductions in repeat biopsies.

In conclusion, the detection rates of overall cancer and CSPC strongly correlated with a 5-point grading of the Likert scale. Among the radiologists with different levels of experience for mpMRI-prostate, there were significant differences in these cancer detection rates. Further investigations and efforts are necessary for developing a better-standardized reporting system and enhancing the education system of mpMRI-prostate interpretation for radiologists including a feedback from urologists and pathologists. We also demonstrated that mpMRI of the prostate and subsequent MRI/US FTB resulted in better detection of CSPC than conventional random biopsy.