“Pin the Tumor on the Kidney:” An Evaluation of how Surgeons Translate CT and MRI data to 3D Models

Abstract

Objective

To quantify how surgeons translate two-dimensional (2D) CT or MRI data to a three-dimensional (3D) model and evaluate if 3D printed models improve tumor localization.

Materials and Methods

Twenty patients with renal masses were randomly selected from our IRB approved prospective 3D modeling study. Three surgeons reviewed the clinically available CT or MRI data; and using computer-aided design (CAD) software, translated the renal tumor to the position on the kidney that corresponded with the image interpretation. The renal tumor location determined by each surgeon was compared to the true renal mass location determined by the segmented imaging data and the Dice Similarity Coefficient (DSC) was calculated to evaluate the spatial overlap accuracy. The exercise was repeated for a subset of patients with a 3D printed model.

Results

The mean DSC was 0.243 ± 0.236 for the entire cohort (n=60). There was no overlap between the actual renal tumor and renal tumor identified by the surgeons in 16/60 cases (26.67%). Seven cases were reviewed again by two surgeons in a different setting with a 3D printed renal cancer model. For these cases, the DSC improved from 0.277 ± 0.248 using imaging only to 0.796 ± 0.090 with the 3D printed model (p < 0.01).

Conclusions

In this study, cognitive renal tumor localization based on CT and MRI data was poor. This study demonstrates that experienced surgeons cannot always translate 2D imaging studies into 3D. Furthermore, 3D printed models can improve tumor localization and potentially assist with appropriate surgical approach.

Introduction

Improvements and increased utilization of imaging technology have made it possible to identify renal tumors at an early stage [1, 2]. Early detection has allowed for opportunities for utilization of nephron-sparing procedures. Nephron-sparing surgery minimizes a person’s risk of developing chronic renal disease and is the treatment of choice for localized small renal masses (cT1a) [3]. Given the increasing push towards minimally invasive organ-sparing partial nephrectomy procedures, it is imperative that the operating surgeon has a good working knowledge of the patient’s anatomy. The precise knowledge of the three-dimensional (3D) location of the renal mass has implications for surgical management decisions, which in turn may influence surgical, perioperative, oncological, and functional outcomes [4].

At our institution, computed tomography (CT) and magnetic resonance imaging (MRI) are the most common imaging modalities for the evaluation of renal tumors, but these are two-dimensional (2D) representations of complex 3D structures. Images are typically acquired in axial plane and viewed one cross section at a time or as multi-planar reformatted images in coronal and sagittal planes (MPR); therefore, surgeons must acquire the necessary skills to convert these 2D images into mental 3D reconstructions in order to adequately prepare for and perform the surgical procedure. While the process of translating 2D images into 3D objects seems straightforward in principle, previous research has shown that mental reconstruction strongly depends on the observer’s spatial abilities [5, 6].

Patient-specific 3D models created from CT or MRI are valuable tools that may be useful for pre-surgical planning, virtual-surgical simulations, intra-operative guidance, and patient consultation. 3D models can be viewed on a 2D computer screen, 3D printed, or viewed in virtual reality (VR) or augmented reality (AR) [7]. Compared to standard CTs or MRIs, 3D models enable better understanding of surgical anatomy [8, 9]. In addition, 3D printed kidney cancer models can influence pre-surgical planning decisions [4, 7] and can be useful for patient education [10–12]. Despite these prior studies, there is limited data quantifying the added value that these models may provide; and it is difficult to demonstrate the superiority of 3D models compared with traditional methods of medical image visualization. A recent study by Ebbing et al. attempted to investigate the reliability of how medical professionals with different levels of expertise (expert urologists and medical students) impart the location of prostate cancer onto a 2D diagram after 1) looking at MRI reports, 2) attending MRI presentations in multi-disciplinary team meetings, and 3) examining 3D printed prostate cancer models [13]. Interestingly, they found that 3D models did not help both expert urologists and medical students better identify tumor location; and they stated that this effect was due to a learning curve in a newly introduced source of information. We believe, however, that this study was limited since the interpretation of 3D anatomical information was performed by drawing in 2D.

The objectives of our study were twofold: (1) to determine how surgeons translate 2D CT or MRI data to a 3D model, (2) to evaluate if 3D printed models improve tumor localization in 3D. We believe that understanding how well the spatial information gained from a sequence of cross-sectional images is interpreted in 3D could give us useful insight as to why surgeons find the 3D models useful as compared to simply viewing the 2D images.

Methods

Subjects

A total of 20 patients with single renal tumors (age = 59.80 ± 10.9 years) were included from our IRB approved prospective study investigating advanced methods of data visualization for patients with kidney cancer. All human subjects provided written informed consent with guarantees of confidentiality. Out of the 20 patients, there were 11 males and 9 females, 14 who had pre-operative MRIs and six who had pre-operative CTs. In addition, seven patients had pre-operative 3D printed models created. The 3D printed models were multi-colored with the kidney in clear, tumor in purple, artery in pink, vein in blue, and collecting system in yellow (Stratasys J750, Eden Prairie, MN). Our workflow to create 3D printed models has been previously described [14]. The average nephrometry score was 7.4 (range = 6–10), with five ≥ 50% exophytic, ten < 50% exophytic, and five completely endophytic tumors.

Image Acquisition

Standard CT or MR images were acquired according to the clinical protocols described below. For patients undergoing CT, dual phase pre- and post-contrast CT data including the nephrogenic phase was acquired in the axial orientation with a 0.625 mm slice thickness (120kVp, 150mAs, 512 × 512 matrix) and reconstructed in sagittal and coronal planes with a slice thickness of 3–4 mm (Somatom Definition Edge (64-detector row, capable of reconstructing 384 slices) or Force (dual-source energy, capable of acquiring 384 (2 × 192) slices, Siemens, Erlangen, Germany). For patients who underwent MR, pre- and post- contrast axial T1-weighted fat-saturated MR images were acquired on a 1.5T MR system (Avanto, Siemens, Erlangen, Germany) using a phased array body coil with the following sequence parameters: TR = 3.58 ms, TE = 1.3 ms, FA = 12°, an interpolated spatial resolution of 1.4 mm × 1.4 mm × 2 mm, and breath-hold acquisition time ranged from 13 to 20 seconds.

Image Post-Processing and 2D to 3D Protocol

Image segmentation and the creation of 3D surface meshes was performed as previously described [7, 14]. Briefly, the kidney, renal tumor, renal artery, renal vein, and collecting system were segmented, the segmented regions of interest were converted to 3D surface meshes, and 3D kidney cancer models were viewed in computer-aided design (CAD) software (Mimics 20.0 and 3-matic 12.0, Materialise, Leuven, Belgium). In the 3-matic software, the tumor was first duplicated, next the tumor was hidden from its actual location, and finally the duplicate tumor was translated to a random location in the space outside the kidney.

Three experienced urologic oncology surgeons, fellowship trained in urologic oncology with 4, 5, and 11 years of experience from the finish of fellowship, reviewed the available multi-planar CT and/or MRI images for the twenty patients on a PACS workstation (Philips IntelliSpace, Koninklijke Philips N.V., Amsterdam, Netherlands). When analyzed by three surgeons, this generated 60 data points for analysis. Prior to review by the surgeon, these images were reviewed and interpreted by a radiologist and the written radiology report was available for the surgeon. Using the interactive translate tool which allows 3D objects to move in space in the X, Y, and Z directions, each surgeon placed the renal tumor on the 3D kidney without the original tumor in the position that corresponded with the interpretation of the CT or MRI. Figure 1 shows axial (1A) and coronal (1B) images for one representative patient along with the corresponding 3D kidney model shown with the tumor removed (1C) and shown with the tumor in its correct location (1D). During the tumor placement, the 3D kidney model could be rotated into any orientation in order to match what was seen in the 2D images. In addition, a user with five years of experience with the 3-matic software helped to facilitate tumor placement if necessary.

Figure 1.

Single case example showing A) axial and B) coronal images of a left endophytic renal mass. C) 3D kidney model with the kidney – pink, artery-red, vein- blue, and collecting system – yellow. The tumor (light blue) has been removed so the surgeon can translate the location where he believes it is after reviewing the images. D) 3D kidney tumor model with tumor in its correct location.

To determine if the 3D printed model would facilitate tumor placement, two surgeons repeated the exercise after reviewing the 3D printed renal cancer models. This exercise was performed in a different setting after a washout period to reduce any bias as to where the tumors were located from the imaging only portion of the trial.

Analysis

The renal tumor location selected for each surgeon was compared to the location of the actual renal tumor and the Dice similarity coefficient (DSC) was calculated to evaluate the spatial overlap accuracy (1= perfect similarity, 0 = no similarity). In regards to the DSC, the correlation is an effect size where 0–0.19 is very weak, 0.2–0.39 is weak, 0.4–0.59 is moderate, 0.6–0.79 is strong, and 0.8–1.0 is very strong. Here, the target score would be a strong correlation [15].

Patients were divided into cohorts based on Nephrometry Score (low complexity 4–6, moderate complexity 7–9, and high complexity 10–12), exophytic and endophytic properties (≥50% exophytic, <50% endophytic, and completely endophytic), and maximum lesion diameter (1–2cm diameter, 2–4cm diameter, and >4cm diameter). The DSC for each cohort were compared using the Mann-Whitney test. For the seven cases which were reviewed again with a 3D printed model, the DSC was compared using a paired t-test. Statistical analyses were performed in SPSS Statistics Version 23 (IBM Corp, Armonk, NY).

Results

All twenty cases were successfully reviewed with pre-operative imaging by the three surgeons. For this entire cohort (n=60, 3 surgeons analyzing 20 each), the mean DSC was 0.243 ± 0.236 (Range = 0 – 0.698), with the mean DSC for each surgeon being 0.291 ± 0.252, 0.238 ± 0.225, and 0.200 ± 0.234 respectively, p > 0.05. The highest DSC for the three surgeons ranged from 0.62–0.68. Supplementary Table 1 shows the DSC (mean ± standard deviation) for all groups evaluated.

There was no overlap between the actual renal tumor and renal tumor identified by the surgeons in 16/60 cases (26.67%). In three cases, all three surgeons incorrectly interpreted the tumor location with no correlation between the perceived tumor location and true location (Figure 2): one case with a completely endophytic renal mass measuring 1.5 × 1.2cm (Nephrometry Score 8p), one with a < 50% exophytic renal mass measuring 1.2 × 1.4cm (Nephrometry Score 7n), and one with a <50% exophytic renal mass measuring 2.6 × 2.8cm (Nephrometry Score 7p).

Figure 2:

Three cases with no overlap (zero correlation) between actual lesion location and cognitive localization by surgeon after reviewing available images. All 3D models have the following color scheme: kidney – light pink, actual segmented lesion – magenta, surgeon placed lesions -light blue, artery- red, vein- blue, collecting system – yellow. Case 1 is shown in the top row: (A), Sagittal Post VIBE MRI with lesion purple. (B) 3D model demonstrating the kidney in the same orientation as imaging slice, and (C) 3D model in axial orientation demonstrating that there is no overlap. Case 2 is in the middle row: (D) Coronal CT with the lesion –red arrow, (E) 3D model demonstrating the kidney in the same orientation as imaging slice, (F) 3D model in sagittal orientation demonstrating that there is no overlap. Case 3 is in the bottom row. (G) Axial Post VIBE MRI with lesion-red arrow and cysts – yellow arrows, (H) 3D model demonstrating the kidney in the same orientation as imaging slice, and (I) Posterior coronal view of 3D model demonstrating that there is no overlap.

Excluding the cases with no overlap, the overall DSC improved to 0.329 ± 0.212 with 0.388 ± 0.214 (n=15), 0.297 ± 0.213 (n=16), and 0.307 ±0.226 (n=13) for each surgeon respectively. The three cases with the highest overall correlation, with DSC measuring 0.563 ± 0.149, 0.503 ± 0.148, and 0.546 ±0.159 respectively, are shown in Figure 3.

Figure 3:

Three cases with the maximum overlap (highest correlation) between actual lesion location and cognitive localization by surgeon after reviewing available images. The same color scheme as Figure 2 is used here. Case 1 is shown in the top row: (A) Axial post VIBE MRI with the lesion –red arrow, (B) 3D model demonstrating the kidney in the coronal orientation (anterior) and (C) 3D model posterior coronal view. Case 2 is shown in the middle row: (D) Axial CT with the lesion –red arrow, (E) 3D model demonstrating the kidney in the same orientation as imaging slice, (F) 3D model in coronal orientation demonstrating overlap between surgeon placed lesions and true lesion. Case 3 is shown in the bottom row: (G) Axial CT with lesion-red arrow, (H) 3D model coronal orientation, and (I) Sagittal view of 3D model demonstrating the overlap.

Stratified by Nephrometry Score, with imaging only the overall DSC was 0.210 ± 0.252 for low complexity lesions (Nephrometry Score 4–6, n=4), 0.256 ± 0.232 for moderate complexity lesions (Nephrometry Score 7–9, n=15), and 0.180 ± 0.312 for one lesion with high complexity (Nephrometry Score 10–12), p>0.05. Grouped by endophytic/exophytic properties, the overall DSC was 0.254 ± 0.278 for ≥ 50% exophytic lesions (n=5), 0.258 ± 0.225 for < 50% exophytic lesions (n=10), and 0.202 ± 0.222 for endophytic lesions (n=5), p>0.05. Finally, the lowest DSC (0.156 ± 0.224) was seen for lesions 1–2cm in diameter (n=6) and the highest DSC (0.363 ± 0.222) was seen for lesions with diameter > 4cm (n=4), p=0.01.

The seven cases with 3D printed models were successfully reviewed by two of the surgeons. For these cases, the mean DSC improved from 0.277 ± 0.248 to 0.796 ± 0.090 (p < 0.01). Stratified by endophytic and exophytic properties, for ≥ 50% exophytic lesions (n=1) the mean DSC for the two surgeons was 0.835 ± 0.120, for < 50% exophytic lesions (n=4) the mean DSC was 0.792 ± 0.077, and for endophytic lesions (n=2), the mean DSC was 0.785 ± 0.125. In this group, there were three cases that had a DSC = 0 when reviewed with imaging only; and the DSC for these cases improved to 0.855, 0.749, and 0.638 with the 3D printed model. The improvement for each case is shown in Figure 4.

Figure 4:

Line graph demonstrating the improvement in DSC after reviewing the 3D printed model for the seven patients in which the exercise was completed in both scenarios. Results for surgeon 1 are shown in blue and for surgeon 2 in red.

Discussion

Methods of advanced medical image visualization including 3D printing, AR, and VR have all been rapidly expanding throughout the healthcare field [16–22]. Despite this expansion, there is limited data quantifying why 3D models are useful compared to traditional methods of image visualization and pre-operative planning. Specifically, how surgeons translate 2D images into 3D surgical planning is not well understood.

In this study, three surgeons reviewed CT and MRI datasets for twenty patients with renal cancer and translated the renal tumor location seen in those images to a 3D model. On average, a weak correlation was observed between the surgeon’s interpretation of renal tumor location based on CT or MRI and the actual location of the tumor (mean DSC = 0.24). These poor scores indicate that it is challenging for surgeons in our study to mentally translate 2D images into a 3D picture, and may help to validate why 3D models may be easier to understand than the current gold standard MRI and CT data. In addition, since each surgeon’s interpretation of the 2D images may be different, 3D models may also help to ensure that the entire surgical team has a similar understanding of the surgical anatomy which may allow for surgical procedures to be performed with greater efficiency.

For this group of 20 patients, there was no difference in how surgeons performed based on experience level. All surgeons performed best for the tumors that were > 4cm in maximum diameter and worst for tumors that were 1–2cm in maximum diameter suggesting that 3D models may be more helpful for smaller tumors. In addition, the surgeons performed worst for completely endophytic lesions as compared to those that had exophytic components. The DSC was lower for small, endophytic tumors, although this did not reach statistical significance. When stratified by nephrometry score, the DSC was lowest for the one case with a score of 10, highest for scores 7–9, and intermediary for scores 6–8. Small renal mass size makes nephrometry score low but results in poor DSC, while endophytic masses have higher nephrometry scores and lower DSC. These two features impact nephrometry score in different directions, but result in more difficult positioning of the tumor. This may explain, in part, why no significant difference between nephrometry score and DSC was observed, and may indicate that nephrometry is not the optimal indicator for whether or not it will be challenging for a surgeon to translate 2D imaging information into 3D, or find a 3D model useful.

In a small cohort of patients, we demonstrated that the DSC improved from weak to a strong correlation when the surgeons were provided 3D printed models, showing that comprehension of anatomy really does improve as compared to traditional cognitive localization methods. These findings are similar to studies evaluating cognitive registration accuracy versus ultrasound or MRI-guided prostate biopsy, which show that cognitive fusion does not work as well [23–25].

For minimally invasive partial nephrectomies, it is important that the tumor location is accurately identified pre-operatively, especially since poor pre-operative understanding of tumor location could result in a sub-optimal surgical approach, such as transperitoneal versus retroperitoneal, which could result in increased OR time, ischemia time, and blood loss. We believe that this exercise may help us identify what tumors are challenging to localize intra-operatively and to appropriately select for pre-operative 3D modeling. For example, pre-operative planning for a completely endophytic tumor located close to the collecting system and vasculature with a nephrometry score of 6, may benefit from a 3D model, while a large exophytic tumor with a nephrometry score of 6 may have limited benefit from a 3D model.

A key limitation of this study includes the unclear real-world significance of major errors (>1cm) or minor localization errors (0.5 – 1 cm). For example, a case in which there is small error, but the perceived tumor location is in the same general area as the actual tumor location (e.g. Figure 2, case 2) might not benefit from a 3D model since it may not alter the surgical approach. Alternatively, in a case in which there is a small error, but the perceived tumor location is on the opposite side of the kidney (e.g. Figure 2, case 1), a 3D model may be warranted since this is highly likely to result in a different surgical approach. Finally, a case in which there is large error, but the lesion is well visualized on intra-operative ultrasound may not warrant a 3D model.

Overall, the surgeons reported that this exercise was challenging particularly for smaller endophytic tumors due to the fact that the images were acquired in planes related to the position of the entire body as opposed to the kidney. Future studies may assess how a difference in the perceived location alters the surgical approach such as tumors located along Brodel’s line, completely endophytic lesions, and lower pole tumors which are challenging for the retroperitoneal approach. In addition, future studies with a multi-institutional experience and larger study cohort that includes quantitative outcome measures may help to identify which cases may warrant pre-operative 3D models.

Conclusions

Experienced surgeons sometimes have difficulty translating 2D imaging studies into 3D for renal tumor localization. This task is especially challenging for small and endophytic tumors. 3D printed models can improve tumor localization for all tumor types, therefore they can serve as an important tool to help surgeons better understand the anatomy and plan the surgical approach.