All Over the Map: An Interobserver Agreement Study of Tumor Location Based on the PI-RADSv2 Sector Map

Abstract

Background:

Prostate imaging reporting and data system version 2 (PI-RADSv2) recommends a sector map for reporting findings of prostate cancer mulitparametric MRI (mpMRI). Anecdotally, radiologists may demonstrate inconsistent reproducibility with this map.

Purpose:

To evaluate interobserver agreement in defining prostate tumor location on mpMRI using the PI-RADSv2 sector map.

Study Type:

Retrospective.

Population:

Thirty consecutive patients who underwent mpMRI between October, 2013 and March, 2015 and who subsequently underwent prostatectomy with whole-mount processing.

Field Strength:

3T mpMRI with T₂W, diffusion-weighted imaging (DWI) (apparent diffusion coefficient [ADC] and b-2000), dynamic contrast-enhanced (DCE).

Assessment:

Six radiologists (two high, two intermediate, and two low experience) from six institutions participated. Readers were blinded to lesion location and detected up to four lesions as per PI-RADSv2 guidelines. Readers marked the long-axis of lesions, saved screen-shots of each lesion, and then marked the lesion location on the PI-RADSv2 sector map. Whole-mount prostatectomy specimens registered to the MRI served as ground truth. Index lesions were defined as the highest grade lesion or largest lesion if grades were equivalent.

Statistical Test:

Agreement was calculated for the exact, overlap, and proportion of agreement.

Results:

Readers detected an average of 1.9 lesions per patient (range 1.6–2.3). 96.3% (335/348) of all lesions for all readers were scored PI-RADS ≥3. Readers defined a median of 2 (range 1–18) sectors per lesion. Agreement for detecting index lesions by screen shots was 83.7% (76.1%–89.9%) vs. 71.0% (63.1–78.3%) overlap agreement on the PI-RADS sector map (P < 0.001). Exact agreement for defining sectors of detected index lesions was only 21.2% (95% confidence interval [CI]: 14.4–27.7%) and rose to 49.0% (42.4–55.3%) when overlap was considered. Agreement on defining the same level of disease (ie, apex, mid, base) was 61.4% (95% CI 50.2–71.8%).

Data Conclusion:

Readers are highly likely to detect the same index lesion on mpMRI, but exhibit poor reproducibility when attempting to define tumor location on the PI-RADSv2 sector map. The poor agreement of the PI-RADSv2 sector map raises concerns its utility in clinical practice.

Level of Evidence:

3

Technical Efficacy:

Stage 2

Prostate multiparametric magnetic resonance imaging (mpMRI) and targeted biopsies have been shown to increase the detection of “clinically significant” cancer.^1–3 Variability in reporting,⁴ however, led to the development of the prostate imaging reporting and data system version 2 (PI-RADSv2) in an attempt to standardize the acquisition and reporting of prostate mpMRI.⁴

The PI-RADSv2 guidelines recommend outlining up to four lesions on a specifically provided standardized 36-sector template.^5,6 This sector map was proposed to “enable radiologists, urologists, pathologists, and others to localize findings described in MRI reports” and to facilitate “precise localization for MR-targeted biopsy and therapy, pathological correlation, and research” as well as surgical dissection.⁴

However, the application of a sector map for reporting mpMRI-detected lesions and guiding interventions has a number of potential complications. Prostate anatomy varies widely between patients.⁷ Prostate volumes in one report ranged from 22–106 cm³ and length from 3.1–6.0 cm.⁸ Therefore, use of the standardized map involves the radiologist subjectively morphing the actual MRI into the standardized MRI. Additionally, deformation in the prostate during MRI acquisition and differences in lesion’s appearance on different sequences may create challenges in adequately representing tumor burden on an ideal sector map.⁹

Anecdotally, we have experienced variation in reporting prostate cancer mpMRI findings between radiologists, particularly in mapping of lesion. This raised questions about the value and dangers of reliance on the sector map. To better understand the extent of this problem we conducted a multireader study on patients with mpMRI before prostatectomy to evaluate the interobserver agreement in defining mpMRI prostate tumor location on the PI-RADSv2 sector map versus identifying tumors with screen shots.

Materials and Methods

Patient Population

This single-center Health Insurance Portability and Accountability Act-compliant retrospective evaluation was approved by the local Ethics Committee and informed consent was obtained from participants. Study subjects were consecutively accrued patients who underwent prostate mpMRI with T₂W, diffusion-weighted imaging (DWI) (apparent diffusion coefficient [ADC] maps and b-2000 DWI), and dynamic contrast-enhanced (DCE) with endorectal coil (ERC) at 3T MRI followed by prostatectomy between October, 2013 and March, 2015 (n = 66). Patients were excluded if digital whole-mount pathology was not available (n = 34) or if patient had a hip prosthesis (n = 2), which hinders the quality of the mpMRI. The total study population was thus 30 patients. Patient characteristics are shown in Table 1.

TABLE 1.

Baseline Characteristics of Patients

	Patients	PZ	TZ
Number	30
Age (yr.)	60.2 (44.4–71.3)
PSA (ng/mL)	10.1 (1.7–44.8)
WP Volume (mL)	38.8 (20.0–81.3)
Total lesions (n)	87	59	28
GS 3+3	8	5	3
GS 3+4	57	41	16
GS 4+3	4	2	2
GS 4+ 4	14	8	6
GS> 4+4	4	3	1
Lesion volume (mL)	1.9 (0.1–20.1)	1.3 (0.1–5.6)	3.2 (0.1–20.1)
Index lesion volume (mL)	3.1 (0.4–20.1)	2.1 (0.4–5.6)	4.6 (0.4–20.1)

Averages with range reported in parentheses where appropriate. PSA = prostate-specific antigen, WP = whole prostate, PZ = peripheral zone, TZ = transition zone, GS = Gleason score.

Readers

Six radiologists from different worldwide institutions participated in this multireader study. Two readers were highly experienced in prostate MRI (>2000 cases, B.T. and T.B.), two were moderately experienced (500–1000 cases, Y.M.L. and H.S.), and two were relatively inexperienced (<500 cases, I.K. and S.B.). Methods for clinical reporting of mpMRI studies using PI-RADSv2 at each institution for all six readers are shown in Table 2. All had experience with PI-RADSv2 prior to this study.

TABLE 2.

Methods of Obtaining and Reporting Prostate mpMRI at Home Institutions for the Six Readers Participating in This Study

Reader	Experience	Country	Biopsy technique	How findings are reported	MRI sequences used
1	High	USA	mpMRI/TRUS Fusion	Physical description and screen shot	T2W/DWI/DCE
2	High	UK	mpMRI/TRUS Fusion	PIRADS sector location/physical description	T2W/DWI/DCE
3	Moderate	Egypt:	Cognitive TRUS	Physical description alone	T2W/DWI
4	Moderate	Singapore	mpMRI/TRUS Fusion	Physical description and screen shot	T2W/DWI/DCE
5	Low	Italy	mpMRI/TRUS Fusion	Physical description and screen shot	T2W/DWI/DCE
6	Low	Turkey	Cognitive TRUS	Physical description alone	T2W/DWI/DCE

mpMRI: multiparametric MRI, TRUS: transrectal ultrasound, T2W: T2-weighted imaging, DWI: diffusion weighted imaging, DCE: dynamic contrast-enhanced imaging.

MRI

All prostate mpMRI scans were acquired on a 3T scanner (Achieva 3.0T-TX, Philips Healthcare, Best, Netherlands) using an endorectal coil (BPX-30, Medrad, Pittsburgh, PA) filled with ~45 mL fluorinert (3M, Maplewood, MN) and the anterior half of a 32-channel cardiac SENSE coil (InVivo, Gainesville, FL). The sequences and MRI acquisition parameters used in this study are depicted in Table 3.

TABLE 3.

Multiparametric MR Imaging Sequence Parameters at 3T

Parameter	T2 Weighted	DWIa	High b-Value DWIb	DCE MR imagingc
Field of view (mm)	140 × 140	140 × 140	140 × 140	262 × 262
Acquisition matrix	304 × 234	112 × 109	76 × 78	188 × 96
Repetition time (msec)	4434	4986	6987	3.7
Echo time (msec)	120	54	52	2.3
Flip angle (degrees)	90	90	90	8.5
Section thickness (mm), no gaps	3	3	3	3
Image reconstruction matrix (pixels)	512 × 512	256 × 256	256 × 256	256 × 256
Reconstruction voxel imaging resolution (mm/pixel)	0.27 × 0.27 × 3.00	0.55 × 0.55 × 2.73	0.55 × 0.55 × 2.73	1.02 × 1.02 × 3.00
Time for acquisition (min:sec)	2:48	4:54	3:50	5:16

^aFor ADC map calculation. Five evenly-spaced b values (0–750 sec/mm²) were used.

^bb = 2000 sec/mm².

^cCE Images obtained, before, during, and after a single dose of gadopentetate dimeglumine 0.1mmol/kg at 3 mL/sec. Each sequence obtained at 5.6-sec intervals.

Image Interpretation

Readers were aware all patients carried a diagnosis of prostate cancer but were blinded to other clinical factors such as prostate-specific antigen (PSA), pre-mpMRI biopsy outcome, previous reads, and each other’s reads. Patient identifiable data was removed from images, and viewed with a single viewer, RadiAnt DICOM Viewer.¹⁰ All readers interpreted the MR datasets independently from each other at their respective sites using their own viewing equipment. Readers were instructed to detect up to four intraprostatic lesions that they would ordinarily include in a report if they were reading the case clinically as part of a PI-RADS assessment.⁴ Readers recorded findings with a Microsoft Access-based application. They marked each lesion by measuring the long axis of the lesion on the dominant sequence (ie, DWI for the peripheral zone [PZ] and T₂W for the transition zone [TZ]) and captured a screen shot of each marked lesion. They then opened the PI-RADSv2 map and digitally outlined the lesion/s on the sector map, indicating which lesion corresponded to which screen shot. Finally, they recorded the individual PI-RADSv2 T₂W, DWI, and DCE score for each lesion (Fig. 1). Readers received no feedback on their performance.

FIGURE 1:

Application to collect PI-RADS reports. Shown is a screen shot of the application coded in Microsoft Access used in this study. Readers were shown the fictional-MRN and name indicated to open a study. After detecting all lesion/s on a given prostate mpMRI study, readers uploaded screen shots of each lesion clearly demarcating the long axis on the dominant sequence. They then indicated the zone (TZ, PZ), level (apex, mid, base), and PI-RADS score for T₂W, DWI, and DCE imaging. After saving screen shots, readers opened the PI-RADS map and drew the borders of each lesion on the PI-RADS sector map.

Interreader Comparison

Comparisons between reader reports were made by a research fellow with experience evaluating >500 patients (M.D.G.). Interreader comparisons were performed based on screen shots and PI-RADS reports. Screen shots were classified as the same lesion based on axial slice, allowing no greater than two slices (each 3 mm) disparity to consider the lesion the same; lesion morphology; and lesion location relative to landmarks (ie, BPH nodules, urethra, internal capsule, etc.).

Each sector included in each lesion outline was recorded, with primary, secondary, and tertiary sectors assigned based on the percentage of each lesion included in a sector. For example, in Fig. 1 the primary sector for Lesion 1 is the right apex PZpm and the secondary sector is the right apex PZpl; conversely, Lesion 2 only had one sector assigned, the left mid PZpl. If a lesion extended over multiple levels of the prostate (ie, apex, mid, and base), the primary sector was defined for each level, with up to three primary sectors possible.

Histopathology Validation

All patients had whole-mount radical prostatectomy specimens processed with patient-specific MRI based 3D-printed molds.¹¹ Prostates were sliced in axial sections corresponding to axial MRI sections (Fig. 2). Lesion outlines and Gleason score (GS) on whole-mount specimens were determined by a genitourinary pathologist with >20 years experience (M.J.M.). Individual volumes of lesions were measured using Aperio Imagescope on digital pathology images.¹² Correlation to pathology was based on prostate landmarks and lesion morphology on screen shots. This was performed by a research fellow (M.D.G.). Index lesions were defined on pathology as the lesion with the highest grade. If lesions were of equivalent grade, the largest volume lesion was taken as the index lesion with up to two index lesions allowed per patient for lesion volumes within 0.1 mL of each other.

FIGURE 2:

Example of reader screen shot and whole-mount pathology. The patient was found to have a PSA of 8.4 ng/mL. mpMRI demonstrated a lesion in the left mid-base peripheral zone. Targeted biopsy found a Gleason 4+3 lesion and prostatectomy of the same grade lesion. Whole-mount pathology processed to correspond to mpMRI axial sections is shown alongside the screen shot of a reader who detected and marked the lesion in the left mid-base peripheral zone.

Statistical Analysis

Agreement was analyzed with respect to 1) lesion detection based on screen shots, 2) lesion detection based on the PI-RADS sector map, and 3) lesion location on the PI-RADS map among identified lesions.

Agreement on lesion detection based on screen shots was determined by the index of specific agreement (ISA). This is defined as the probability among a pairwise combination of readers that if one reader detects a lesion, the other will also detect that lesion. The denominator is the average number of lesions detected by each pairwise combination.¹³

Agreement on lesion detection based on the PI-RADS sector map was also determined by ISA. This analysis defined detection of the same lesion when at least one of the same sectors of a given lesion was marked by each reader. In other words, if two radiologists marked a lesion in any of the same sectors, that would be scored as overlap agreement. Statistical difference between lesion detection on screen shots vs. the sector map was based on the Z-test, with P < 0.05 considered statistically significant.

Agreement on sector map of identified lesions was evaluated with respect to tumor involvement (all sectors, primary sector, and level of disease) and type of agreement (exact, overlap, and proportion). Here, agreement was analyzed only with respect to lesions detected by at least two readers; lesions detected by only one reader were excluded. Exact agreement is defined as agreement in each sector/level involved with tumor. Overlap agreement is defined as agreement in overlap of sectors/levels involved with tumor. Proportion of agreement is defined as the proportion of overlap sectors/ levels among all sectors/levels indicated by two readers to have cancer. For all analyses, overall agreement was obtained by averaging pairwise agreement across all reader pairs.

Sensitivity was calculated for all lesions, index lesions, and by zone (PZ and TZ). Specificity was not calculated, as all patients had cancer and readers were not asked to define negative regions.

The bootstrap resampling procedure on a patient level was used to obtain standard errors and construct the 95% confidence intervals (CI) where the confidence limits were taken from 2.5% and 97.5% percentiles of 2000 bootstrap samples. Correlation between per lesion average tumor size and standard deviation and between pairwise proportion of agreement and average tumor size was evaluated by Spearman rank correlation.

The number of cases for this study was determined such that it had an adequate power to detect a clinically meaningful mean pairwise difference in reader agreement on location of all lesions based on screen-shots vs. sector map. From in-house data, the improvement of reader agreement by screen-shots over sector map was targeted at 10%, with the corresponding standard deviation set at 0.15. Twenty-six cases are required to detect the targeted 10% difference with 90% power using the Z-test at the two-sided 5% significance level. With 30 patients, the power is increased to 94%.

Results

Lesion Characteristics

A total of 130 unique lesions were detected by all readers in 30 patients (4.33 per patient). Each reader detected 1.93 lesions/patient on average (range 1.63–2.33). 76/130 (58.5%) of lesions were in the PZ. The average frequency of PI-RADSv2 scores for each reader was 3.8%, 29.6%, 44.8%, and 21.5% for PI-RADS 2, 3, 4, 5, respectively. Pathological characteristics of lesions in this patient population are shown in Table 1.

Readers defined a median of two (range 1–18) sectors per lesion. The median long axis length of a lesion was 11.0 mm (2.5–40.9), with a median of 9.7 mm (2.5–36.7) in the PZ and 13.0 mm (4.8–40.9) in the TZ.

Sensitivity

Average sensitivity was 50.4% for all lesions (range 42.5–63.2%) for all six readers. Sensitivity in the PZ was 50.3% (42.4–59.3%) and in the TZ 50.6% (21.4%–71.4%). For index lesions the average sensitivity was 72.9% (62.9–85.7%) with a sensitivity 67.9% (35.7–85.7%) in the PZ and 76.2% (71.4–85.7%) in the TZ.

Agreement by Screen Shots vs. Sector Map

The average agreement between readers for detecting all lesions based on screen shots was 61.1% (95% CI: 54.2–67.3%) vs. 48.3% (41.6–54.4%) overlap agreement for the PI-RADS sector map (P < 0.001). Similarly, agreement for index lesions by screen shots was 83.7% (76.1%-89.9%) vs. 71.0% (63.1–78.3%) overlap agreement on the PI-RADS sector map (P < 0.001). Agreement was comparable across reader experience, as demonstrated in Table 4.

TABLE 4.

Pairwise Agreement by Reader Experience for Detecting Index Lesions

Reader pair	Agreement
	Screen-shots	Sector Map
1–2	87.3%	80.0%
1–3	85.7%	67.9%
1–4	91.2%	77.2%
1–5	84.6%	73.1%
1–6	83.0%	75.5%
2–3	86.3%	66.7%
2–4	84.6%	76.9%
2–5	80.9%	68.1%
2–6	87.5%	75.0%
3–4	86.8%	67.9%
3–5	79.2%	58.3%
3–6	81.6%	53.1%
4–5	77.6%	73.5%
4–6	80.0%	76.0%
5–6	80.0%	75.6%
Average* (95% CI)	83.7% (76.1–89.9%)	71.0% (63.1–78.3%)

Experience level: 1,2: High. 3,4: Moderate. 5,6: Low.

^*P < 0.001.

Interreader Agreement on Sector Map

Agreement for indicating which sectors are involved with prostate cancer on the PI-RADSv2 map is shown in Fig. 3. A visual representation of variability between readers marking the same lesion is shown in Fig. 4. The average exact agreement for index lesions for all sectors involved was 21.2% (95% CI: 14.4–27.7%). Overlap agreement, ie, tumors in which any sectors were held in common, was 84.8% (78.2–90. 8%). The proportion of agreement, or proportion of sectors in common, was 49.0% (42.4–55.3%). For defining the primary sector of index lesion involvement, the exact agreement was 38.5% (28.6–48.2%), overlap agreement 69.4% (59.9–78.4%), and proportion of agreement 53.4% (45.3–61.3%). Agreement was comparable for all measures for all lesions, as shown in Table 5.

FIGURE 3:

Agreement on the PI-RADSv2 sector map for index lesions detected by at least two readers. Three measures of agreement are shown (exact, overlap, and proportion of agreement) for six radiologists defining sectors on the PI-RADSv2 map containing tumor. “All” is for agreement for all sectors marked to contain cancer. “Primary” is agreement for only the primary sector of a lesion. This demonstrates the variability in defining precise tumor location on a sector map. Error bars represent the 95% confidence interval.

FIGURE 4:

Example of interreader variability on sector map. On the top row are the screen shots of DWI images for six readers detecting the same lesion in the right mid-peripheral zone; this was the index lesion. On the second row are screen shots for all six readers detecting a lesion in the right mid-transition zone. The sector maps demonstrate the variability in how these two lesions were mapped. Both lesions were Gleason = 3+4 at prostatectomy.

TABLE 5.

Sector Map Agreement for All Lesions

Agreement	All sectors	Primary sector
Exact	20.1% (14.7–24.7%)	38.1% (29.3–46.6%)
Proportion	45.7% (39.8–50.4%)	49.9% (42.0–57.4%)
Overlap	79.2% (72.5–84.7%)	62.5% (54.1–70.8%)

All values reported are averages with 95% confidence interval.

Agreement in defining the level of disease (ie, apex, mid, base) was 61.4% (95% CI 50.2–71.8%). The proportion in which there was overlap among levels was 77.2% (70.3–83.9%). Agreement in overlap of levels was 92.8% (86.7–97.8%).

Agreement by Lesion Size

The relationship between mean tumor size and the standard deviation (SD) of tumor size for all lesions detected by at least two readers is shown in Fig. 5. The average standard deviation between readers measuring the same lesion was 2.66 mm (0.15–12.30). The median difference between the smallest and largest measurement by two readers was 4.15 mm (range 0.3–27.1 mm). There was a weak correlation (0.33) between increasing tumor size and improved agreement on localization. Larger tumor size did not improve agreement in sector assignment among readers, with a rank correlation of 0.07, as shown in Fig. 6.

FIGURE 5:

Plot of mean tumor size and standard deviation (SD) as measured by radiologists. Variability between radiologists tended to increase with increasing tumor size, but with a weak correlation.

FIGURE 6:

Correlation between reader-pair average tumor size and proportion of agreement. The Spearman rank coefficient is shown for each reader pair and proportion of agreement and tumor size. There was not an improvement in agreement on the sector map with increasing tumor size.

Discussion

Multiparametric MRI has gained increased utility in improving the detection rate of clinically significant cancer and decreasing the diagnosis of indolent disease.^2,14–16 A new system to communicate mpMRI findings among specialists, PI-RADSv2, has been proposed as a means of improving standardization of acquisition and reporting.¹⁷ PI-RADSv2 recommends the use of a complex 36 sector map for communicating graphically the location of a tumor. This is purportedly used to help guide the urologist to the right location for biopsy. Anecdotally, we found that different readers had very different interpretations of how to use this map, which could result in miscommunication and, therefore, inaccurate biopsies, especially if cognitive-guided biopsies were used.

We evaluated the variability among radiologists of varying experience levels for reporting lesion locations on this map. There was good agreement for detecting index lesions using screen shots, which is in line with previously published studies on PI-RADSv2.^18,19 However, by introducing the sector map, agreement in lesion detection decreased significantly, even when allowing for generous overlap in lesion outlines between readers. If constrained to agreement on the exact sectors involved, agreement on the sector map was poor (21%). There was poor agreement (39%) for indicating which sector was the primary sector involved in an index lesion and in only half of detected index lesions. Readers may have demonstrated poor agreement by differing in assessment of the levels of the lesions, but allowing for this difference, agreement only increased to 53% for the primary sector involved in an index lesion.

Some of the variability in using the sector map may be attributed to difficulties deciding to which level the tumor should be assigned, apex, mid, or base. This relates to the vague definition of these levels which are conceived as approximations of each third of the prostate in the sagittal or coronal planes.⁴ This presents challenges as it forces the 15–25 cross sectional axial mpMRI images to be compressed into three planes. We found only 61% agreement on the exact level, although this rose to 93% agreement when level overlap was permitted. This variability in describing the lesion level can translate into inaccuracies in guiding biopsies.

Lesion size did not seem to impact mapping variability. There was no association between interreader agreement and larger lesions. Junker et al²⁰ reported a prostate mapping system that demonstrated better agreement for lesions >0.5 cc than was found in our study. There are important differences in the map proposed by Junker et al, the “Prostate Interdisciplinary Communication and Mapping Algorithm for Biopsy and Pathology” (PIC-MABP) and the PI-RADS map. The PIC-MABP is an axial map with a variable number of planes dependent on the size of the prostate. This approach does not require a decision about which level the tumor is located in, since sectors are defined without regard to anatomical regions. One limitation of the PI-RADS sector map is that it presents an ideal case for prostate anatomy without regard to enlargement or other distortions, making it difficult for the reader to interpolate between the image he/she is looking at and an idealized map.

Admittedly, it is unclear what impact this interreader variability has on biopsy outcomes. Indeed, it is unclear how clinicians use the maps for guiding biopsy. For most TRUS/MRI fusion biopsies, it is necessary to define specific axial screen shots of lesions and the map is largely ignored. The sector map has the theoretical advantage that it can define the sagittal extent of the tumor, but the impact of this is unknown. A sector map remains essential for cognitive biopsies, but here there is concern that the radiologist’s depiction of the lesion may be misleading based on the high level of interreader variability. These findings should raise questions for clinicians performing cognitive biopsies and using the sector map to guide biopsies.

Reporting mpMRI findings on screen shots alone for fusion biopsies may not be the perfect solution. A single screen shot does not communicate the sagittal extent of a lesion. It seems logical that communicating both an axial and sagittal screen shot that show the craniocaudal extent of disease for cognitive biopsy might improve biopsy yield. In addition, text descriptions of radiographic findings remain essential, as often the only means of communicating with clinicians depending on working patterns. PI-RADSv2 advocates template-based reporting that prompts for report completeness.

There are a number of limitations to our study. All images originated from the same institution. This eliminates acquisition-related variability but introduces the possibility that the images produced were uniquely prone to interreader variability; however, we consider this unlikely. Despite having only one site supply images, we had six independent readers from different institutions and with different levels of experience, suggesting these data are indicative of what is occurring in general practice. Next, as readers were from different institutions, each had different reporting strategies (which regions to report, which PI-RADS score cutoff to use), which contributed to interreader variability. We tried to capture this variability in our analysis, but did not enforce a specified reporting mechanism beyond what is suggested in PI-RADS. This difference in agreement is important in interinstitutions, but may not be in intrainstitutions. Next, we included small pathological lesion (0.1 mL), which may inflate the sensitivity analysis, but would not influence agreement analysis. Next, reader-defined “dominant lesions” were not correlated to index lesions, but rather index lesions were defined on pathology. This may suggest that agreement on index lesions we report is higher than reality, as these lesions are likely more readily detected. Next, all of our patients had prostate cancer treated with prostatectomy and, therefore, may represent a subpopulation (most had 3+4 disease) with lesions that are more readily detected on mpMRI than the general population of men undergoing MRI. Finally, a single individual with access to prostatectomy specimens undertook the comparison between readers’ screen-shots and pathology. This could introduce an observer bias in the study. We attempted to control for this by having a second person cross-check findings.

In conclusion, radiologists reporting prostate cancer mpMRI findings on the 36-sector PI-RADSv2 map demonstrated poor agreement in assigning the location on the map. These data suggest that the PI-RADSv2 map may be adequate for indicating the general area of prostate cancer, but should be used very cautiously for actually guiding biopsies. If any two readers see the same lesion on mpMRI they will agree on only about one-half of sectors involved with a lesion on the sector map, when in reality they have complete agreement on the lesion location on screen shots. Difficulties were seen in consistently identifying the level (apex, mid, base) of the tumor as well as which sectors on the map were involved. Thus, we conclude that the PI-RADSv2 sector map may be useful to provide general guidance about lesion location, but should be used cautiously in isolation for actually directing biopsies, particularly with cognitive biopsies of the prostate. Actual screen shots of tumor location demonstrated much better agreement among readers.