How specific are patient-specific simulations? Analyzing the accuracy of 3D-printing and modeling to create patient-specific rehearsals for complex urological procedures

Abstract

Purpose

In the field of urology, 3D printing and modeling are now regularly utilized to enhance pre-operative planning, surgical training, patient-specific rehearsals (PSR), and patient education and counseling. Widespread accessibility and affordability of such technologies necessitates development of quality control measures to confirm the anatomical accuracy of these tools. Herein, we present three methods utilized to evaluate the anatomical accuracy of hydrogel PSR, developed using 3D printing and molding for pre-operative surgical rehearsals, of robotic-assisted partial nephrectomy (RAPN) and percutaneous nephrolithotomy (PCNL).

Methods

Virtual computer-aided designs (CADs) of patient anatomy were created through segmentation of patient CT scan images. Ten patient-specific RAPN and PCNL hydrogel models were CT scanned and segmented to create a corresponding model CAD. The part compare tool (3-matic, Materialize), point-to-point measurements, and Dice similarity coefficient (DSC) analyzed surface geometry, alignment, and volumetric overlap of each model component.

Results

Geometries of the RAPN parenchyma, tumor, artery, vein, and pelvicalyceal system lay within an average deviation of 2.5 mm (DSC = 0.70) of the original patient geometry and 5 mm (DSC = 0.45) of the original patient alignment. Similarly, geometries of the PCNL pelvicalyceal system and stone lay within 2.5 mm (DSC = 0.6) and within 15 mm (16% deviation) in alignment. This process enabled the refinement of our modeling process to fabricate anatomically accurate RAPN and PCNL PSR.

Conclusion

As 3D printing and modeling continues to have a greater impact on patient care, confirming anatomical accuracy should be introduced as a quality control measure prior to use for patient care.

Introduction

Patient-specific anatomical modeling involves the development of virtual or physical phantoms of human pathophysiology that are individualized to patient-specific data [1]. Using a replica of the patient’s anatomy, a patient-specific rehearsal (PSR) can be performed prior to the real procedure [2, 3]. Recent technological advances in software and three-dimensional (3D) printing have reduced fabrication costs and complexity, increasing accessibility to the medical field and changing outlook of surgical education, minimal invasive approaches, and personalized medicine. In the field of urology, 3D printing has already been shown to enhance preoperative planning, surgical training, patient education and counseling for treatment of renal masses, prostate cancer, and nephrolithiasis [4, 5]. Several publications have developed PSR simulation platforms for robotic-assisted partial nephrectomy (RAPN) [6, 7]. Despite their direct impact on patient care, anatomical accuracy is rarely confirmed [8].

To create patient-specific anatomical models, the process starts with Digital Imaging and Communications in Medicine (DICOM) files of axial imaging, primarily MRI or CT scans. A segmentation program is used to isolate each structure of interest on every slice of the scan using a combination of algorithms and manual definition. The resulting 3D reconstruction, or computer-aided design (CAD), can be directly 3D printed to create a visual guide of the chosen components [9]. Currently, advanced 3D-printers are capable of printing objects on the order of tens of microns. Previous attempts to confirm the anatomical accuracy of 3D printed models for RAPN cases have found minimal error between the model and corresponding imaging, suggesting that the segmentation and printing process does not skew the anatomic relationships [10, 11].

The Simulation Innovation Lab in the University of Rochester Medical Center has developed PSR simulation platforms for RAPN [12, 13] and percutaneous nephrolithotomy (PCNL) [14]. These models are able to replicate the tissue properties of live organs by combining 3D printing with hydrogel molding [15, 16]. Briefly, negative molds are created for each component of the patient’s CAD, 3D printed, filled with hydrogel, and cured to produce a realistic anatomical structure. Each of the components are registered in successive molds to create the final model. The additional phases of this construction process, including registration of multiple components with different hydrogel formulations, could influence the anatomical accuracy of the final model.

To verify the accuracy of the elaborate registration and fabrication process required to build our PSR models, we implemented methods and protocol to ensure anatomical accuracy through analysis of the surface geometry, volume, and alignment.

Methods

Ten RAPN and ten PCNL hydrogel PSR models were scanned (CT scanner; Siemens Healthcare) using identical imaging protocol [8]. The segmentation process previously used to create the patient CAD was repeated to create a CAD of the PSR model. For the RAPN models, the parenchyma, tumor, renal artery, renal vein, and pelvicalyceal system (PCS) were segmented for inclusion in the model CAD. For the PCNL models, the spine, ribs, skin, stone, and PCS were segmented.

Three types of analysis quantified the error in surface geometry, volumetric overlap, and alignment between the model and original patient CAD.

Geometric similarity

To measure the accuracy of fabricating each individual component, each structure from the model CAD was aligned with the corresponding structure from the patient CAD using an optimizing function (global registration; Materialise, Belgium). Then the part comparison tool was used to measure the mean, standard deviation, minimum, maximum, and distribution of the distance between each point on the surface of the patient’s structure to the corresponding point of the model (approximately 20–30,000 points) (Fig. 1). Analysis was completed for each structure of the RAPN model (parenchyma, tumor, renal artery, renal vein, and PCS) and for the vital structures (PCS and stone) of the PCNL model.

Fig. 1.

Geometric similarity. The patient CT scan is segmented to form the patient CAD. The Patient CAD is then used to design and 3D print injection molds which are filed with hydrogel to create the patient specific rehearsal model. The model is CT scanned and segmented to create the model CAD. The model CAD is moved to align each component with its corresponding patient component (patient components—transparent). Part comparison is completed to analyze every point of the surface and their distribution (histogram displaying distribution of 98% of points, n = 22,158, range − 4.3622 to 4.4015) for every component (parenchyma, tumor, artery, vein, pelvicalyceal system). Scale for each component covers 98% of points and color scale indicates each point’s distance from patient anatomy

Volumetric overlap

Keeping the structures aligned as in the geometric similarity calculation, the Dice similarity coefficient (DSC) was calculated. DSC quantifies the intersect between the volume of the patient’s and model’s CAD components on a scale from 0 (no overlap) to 1 (perfect overlap) (Fig. 2) [17].

Fig. 2.

Volumetric overlap. The patient CAD and model CAD components were aligned (global registration, 3-Matic, Materialise) and intersect of the two volumes was calculated (boolean intersect). The volume of the patient calyx (yellow), model calyx (gray) and intersect (blue) were used to calculate the Dice similarity coefficient (DSC), a measure of volumetric overlap

Alignment

To measure the registration of the components during the fabrication process, all structures of the RAPN model CAD were moved until the parenchyma was aligned with the patient’s parenchyma (global registration). Then part comparison for each structure was completed to analyze the distance of the surface of each model component from its alignment in the patient model (Fig. 3). For the PCNL model, point-to-point measurements of the stone’s position with respect to the spine were obtained from each CAD and the average deviation was calculated (Fig. 3). Measurement points were chosen to analyze the model’s orientation, regardless of their clinical relevance.

Fig. 3.

Alignment. Alignment of the PCNL models (left) was analyzed by measuring the distance of several important relationships: stone to skin (red), stone to spine (blue), stone to rib (gray), stone to tip of rib (orange), and angle created by the end of rib, stone, and spine (green). Alignment of RAPN models was analyzed by completing a part comparison of each component after the model was moved as a whole to align the parenchyma of the patient and model CADs. Note the difference in orientation between the vein of the model in this example and in Fig. 1 (both with colored part comparison overlay) to that of the patient CAD (solid blue in Fig. 3, transparent blue in Fig. 1)

Using these methods, the model fabrication process was refined through additional model iterations. The following results represent the final construction process which yielded acceptable levels of discrepancy for our applications.

Results

RAPN

The average deviation of the surface geometry of the model parenchyma, tumor, renal artery, renal vein, and PCS were − 0.59 mm, − 0.43 mm, 2.42 mm, 1.06 mm, and 0.91 mm, respectively, with at least 80% of points within 3 mm of the corresponding patient anatomy (Table 1). Volumetric overlap was rated very strong for the parenchyma and tumor (DSC = 0.89 and 0.94), strong for the vein and PCS (DSC = 0.61 and 0.60) and moderate for the artery (DSC = 0.44).

Table 1.

Results of the analysis of surface geometry (Fig. 1), volumetric overlap (Fig. 2), and alignment (Fig. 3) between robotic-assisted partial nephrectomy (RAPN) models to the original patient anatomy

	Kidney	Tumor	Artery	Vein	Calyx
Surface geometry
Mean	− 0.6	− 0.4	2.4	1.0	0.9
Standard deviation	2.0	1.1	2.5	1.8	1.8
Minimum	− 7.3	− 3.8	− 2.7	− 4.5	− 3.4
Maximum	8.4	3.6	11.9	7.4	7.4
Volumetric overlap
DSC	0.9	0.9	0.4	0.6	0.6
Alignment
Mean		− 0.1	4.9	2.8	2.5
Standard deviation		2.4	4.1	3.4	3.0
Minimum		− 5.3	− 3.1	− 5.4	− 4.8
Maximum		6.2	15.6	12.7	10.8

Surface geometry and alignment were analyzed using the part compare tool (3-matic, Materialize) and volumetric overlap was analyzed using Dice’s similarity coefficient (DSC). Surface geometry and volumetric overlap were analyzed after aligning the model component to corresponding patient component while alignment was analyzed for each component in regards to the positioning of the parenchyma

The alignment of the tumor, artery, vein, and calyx deviated from the patient’s anatomy an average of − 0.1 mm, 4.93 mm, 2.82 mm, and 2.54 mm, respectively. Over 75% of the tumor points were within 3 mm. 42.6%, 55%, and 60.4% of the artery, vein and PCS, respectively, were within 3 mm while 62%, 74%, and 80% of were within 5 mm.

PCNL

Regarding the geometry, deviation of each point of the stone and PCS of the model CAD to the patient CAD averaged 0.96 mm and 2.20 mm, respectively (Table 2). Of the points on the stone, 94% were within 3 mm of the corresponding patients’ CAD while 59% of the of the PCS points were within 3 mm and 81% within 5 mm. Volumetric overlap was rated moderate for the PCS (DSC = 0.42) and strong for the stone (DSC = 0.79).

Table 2.

Results of the analysis of surface geometry (Fig. 1), volumetric overlap (Fig. 2), and alignment (Fig. 3) between percutaneous nephrolithotomy (PCNL) models to the original patient anatomy

	Stone	Calyx
Surface geometry
Mean	1.0	2.2
Standard deviation	1.0	3.0
Minimum	− 1.6	− 6.2
Maximum	3.9	10.6
Volume overlap
DSC	0.79	0.42
Alignment	Deviation	Percentage (%)
Stone to Skin (mm)	14.5	16
Stone to Rib (mm)	− 11.3	− 10
Stone to Spine (mm)	− 2.8	− 5
Stone to end of Rib (mm)	− 1.9	0.1
Angle between Rib and Spine (deg)	5.7	7

Surface geometry and alignment were analyzed using the part compare tool (3-matic, Materialize) and volumetric overlap was analyzed using Dice’s similarity coefficient (DSC). Surface geometry and volumetric overlap were analyzed after aligning the model component to corresponding patient component. Alignment was analyzed by comparing the average deviation of point-to-point measurements of patients and models

For the alignment, the most accurate measurement was from the stone to the tip of rib, averaging 6.7 cm in patients and 6.5 cm in the model, with an average deviation of − 1.91 mm (0.1%). The least accurate was from the stone to skin, averaging 120 cm in patients and 134 cm in the model, a deviation of 14.47 mm (15.6%).

Discussion

Any structure can be defined by its shape, volume, and 3D orientation in space. Utilizing a combination of the aforementioned three analytic methods, we comprehensively quantified each of these aspects. This is the first report to comprehensively quantify the anatomical accuracy of multiple PSR models.

Although there are no published reports analyzing the anatomical accuracy of RAPN PSR, previously work has determined the accuracy of 3D printed RAPN models. Wake et al. completed a calibration of their 3D printer and a comparison between two diameters of renal tumors from a CT scan to caliper measurements of a 3D printed model for seven patients with exophytic tumors. They reported high levels of correlation (< 1% difference) between measured diameters with a slight overestimation in size present in the calibration and final prints [10]. Michiels et al. measured the length of kidney and tumor axis, tumor volume, and measurements of three spots on the arterial branches reporting differences of 1.8%, 2.35%, 14.7%, and 1.8%, respectively [11]. Previous attempts to characterize the accuracy of PCNL PSR has rarely extended past visual confirmation [18]. The combination of methods presented in this work create a more detailed view of the accuracy of the model as it incorporates many more points into geometric analysis. The geometry of each component was able to be scrutinized separately from the possible errors introduced in the registration process. Our results demonstrate we can reconstruct the geometry of each component with more accuracy compared to alignment. However, the existing differences are still within our accepted levels of accuracy.

The two previous studies along with our presented data have prioritized anatomical accuracy, but this is far from standard practice in the field. Regulatory concerns, quality control, and quality assurance, are key components that will drive the successful adoption of 3D printing. The United States Food and Drug Administration (FDA) does not require oversight or approval for 3D printed models designated as “non-diagnostic” or made at hospitals for in-house use [19]. Thus, hospital physicians or engineers that are pursuing these types of 3D printing and modeling applications should consider implementing quality control systems. Methods for measurement of the consistency, repeatability, and reliability of any fabrication process are essential and have been previously investigated for 3D printing [20]. It is especially important that 3D printing applications with the potential to directly impact diagnosis, clinical treatment and surgical interventions, are frequently assessed. Widespread medical applications of 3D printing technology are projected to continuously increase as supported by the addition of four category III Current Procedural Terminology (CPT) codes for 3D printing models and guides, representing a step toward future reimbursement [21]. This work lays out several possible metrics for quality control of surgical simulation platforms. Standardization and validation of these emerging technologies remain an important consideration as the effects on patient outcomes and clinical applications continue to be explored.

This work comes with its limitations. While the process of obtaining further CT scans of the models may not be feasible at all centers, the emerging technology of 3D scanning may substitute CT scans. 3D scanning can produce higher resolution model CADs than CT scans, but recreates only the model’s outer surface. The establishment of an acceptable level of error can vary based on the importance of the structure within a surgical context and must be further quantified on as per structure and per measurement basis. For instance, the RAPN model artery had the largest surface geometry deviation and lowest volumetric overlap but considering the small diameter of the artery in relation to other structures, larger differences may be justified. In addition, the largest deviation in PCNL alignment was from the stone to skin which can change with patient positioning so the stone to rib distance may be a more consistent landmark. Lastly, it is still unclear which of these analysis types is the most pertinent to surgical rehearsal. Further studies will incorporate these quantifications prior to PSR to help define the clinical significance of variations from patient anatomy by collecting surgeon feedback and patient outcomes.

Conclusion

As the surgical applications of 3D printing and modeling expand, quality control measures must be developed. In the field of PSR, anatomical accuracy is especially important to ensure transfer from the simulated to live environments. The methods developed here will be continually applied to each model we fabricate.

Abbreviations

3D

Three-dimensional

CAD

Computer-aided design

CPT

Current procedural terminology

CT

Computed Tomography

DICOM

Digital Imaging and Communications in Medicine

DSC

Dice similarity coefficient

FDA

Food and Drug Administration

MRI

Magnetic Resonance Imaging

PCNL

Percutaneous Nephrolithotomy

PCS

Pelvicalyceal system

PSR

Patient-specific rehearsal

RAPN

Robotic-assisted Partial Nephrectomy

Availability of data and material

Not applicable.