Accuracy and precision of measurements performed on three-dimensional printed pelvises when compared to computed tomography measurements

Abstract

The preoperative contouring of plates decreases the duration of surgery and improves the quality of the reduction of pelvic fractures. Patient-tailored three-dimensionally printed pelvises might be an interesting tool for achieving that purpose. Currently, no study has evaluated the accuracy of measurements performed on three-dimensional printed models in comparison with computed tomography data for complex bones, such as the pelvis. This study examined whether the measurements obtained on pelvises printed using dual-material fused deposition modeling technology are not significantly different from those obtained on computed tomography images. The computed tomography images of the pelvic region from 10 dogs were used to produce three-dimensionally printed models with a dual-material fused deposition-modeling process. Four segments were measured on both three-dimensionally printed models and computed tomography images. The measurements were performed by three observers and repeated twice. Concordance correlation coefficients were used to assess the precision and accuracy of the measurements as well as evaluate the agreement between the methods. The accuracy of measurements between the methods was > 0.99 for all measurements. The precision was almost perfect for AE (0.996), substantial for BD and BC (0.963 and 0.958, respectively), and moderate for CD (0.912). These results indicate that, despite some minor variations, the measurements performed on printed models reproduced the computed tomography data reliably.

INTRODUCTION

Pelvic fractures in dogs are often complex and their reduction can be challenging [1]. In human surgery, preoperative planning and contouring of plates in complex fractures has been documented to improve the quality of the reduction as well as decrease the surgery time and complication rate [2,3].

The precise preoperative contouring of plates is possible only when a similarly sized anatomical model is available, which is challenging in veterinary patients of various sizes and breeds. The increasing availability of computed tomography (CT) and three-dimensional (3D) printing has paved the way for the use of 3D printed models to replicate a complex bone-surface anatomy. Several veterinary and human studies have reported the printing of bone models to plan surgery [4,5,6,7]. To be reliable, a 3D printed model should reproduce the morphology of the bone to a fidelity sufficient for surgical planning, but this has not been defined. The reliability of the 3D printing process for the femoral bone in dogs has been reported [8]. On the other hand, there is no literature evaluating the reliability of 3D printing of geometrically complex bone, such as a canine pelvis.

Several 3D printing methods are currently available [9,10]. Selective laser sintering uses a process of high-powered laser fusion with a powdered material [9]. Stereolithography, historically the first method of 3D printing, relies on the polymerization of a liquid resin using a UV low-powered laser [9,10]. Fused deposition modeling technology (FDM) is currently the most common printing method and relies on the sequential deposition of fused plastic layers [11,12]. Stereolithography and FDM depend on the design and printing of support parts to act as a scaffold of shallow slopes and overhangs. The support elements need to be trimmed after printing. The entire process can result in an irregularity of the surface of the printed model [9,13].

Dual-material FDM technology allows for the printing of models with different polymers [12]. Some of those polymers designed specifically to be used as a support material are either dissolvable (Polyvinyl alcohol PVA) or are made to breakaway (polyurethane and polylactic acid blend, Breakaway, Ultimaker, Netherlands) easily from other polymers. This allows for a 3D model with a smoother and more regular surface.

This study examined the accuracy and precision of measurements obtained on dual-material FDM (dm-FDM) 3D printed models by a comparison with the measurements of anatomical images obtained by CT. The hypothesis was that measurements of the anatomic segments on a dm-FDM printed model and CT data would be in agreement, indicating that the 3D printed models reproduce the CT data reliably, which is essential for orthopedic use.

MATERIALS AND METHODS

Ten dogs of different sizes were selected opportunistically among patients requiring a CT of the pelvic region for reasons unrelated to the study. The owners' consent was obtained, and the study was performed following the ethical guidelines of the research institution (IACUC 20170501). The procedure was performed under general anesthesia (acepromazine 0.5 mg/kg, Calmivet, Vetoquinol, France; buprenorphine 10 µg/kg, Buprevet, Virbac, France; Alfaxalone 1 mg/kg, Alfaxan, Jurox, United Kingdom; plus isoflurane relay, Vetfluran, Virbac), and the anatomical region of the pelvis was imaged with a four slice CT scan Lightspeed plus (General Electric, General Electric Medical System, France). Image acquisition was performed according to the patient size, following routine protocols selected to limit both the length of anesthesia and the X-ray dose delivered to the patient. The conditions were as follows: FOV 50 cm, slice thickness 2.5 mm, 120 kV, and 153 mA for dogs weighing less than 15 kg; and FOV 50 cm, slice thickness 2.5 mm, 120 kV, and 240 to 383 mA for dogs weighing more than 15 kg [8]. The acquisitions were performed with a 1.5 pitch and proprietary helical interpolation algorithms (Bone, General Electric, General Electric Medical System) were used without edge enhancement. The reconstruction interval was set to 2.5 mm. The pelvises were evaluated for any abnormalities, including but not limited to signs of degenerative joint disease, and were excluded if any were found.

File and 3D printing

Standard tessellation language (STL) files were generated from the DICOM images. Segmentation, thresholding, and exportation of the models were performed within the CT-scan software (Advantage Workstation 4.3, General Electric Medical System). The osseous structures of the pelvic area were isolated from the surrounding soft tissues based on their density measured in Hounsfield units (superior to HU 226). Further individualization of the pelvis from the sacrum, coccygeal vertebrae, and femora was done automatically using the software based on their anatomical separation (i.e., sacro-iliac joints and coxo-femoral joints). A visual inspection of the segmentation was performed to confirm its accuracy. Two hemipelvises (right and left) were printed for each dog, but only the right ones were used in this study. The 3D models were printed by a professional company (3D ArcWest, France) using a professional grade 3D printer (Stratasys 1200 sst, Eden prairie, USA). The printing materials used were ABS (acrylonitrile butadiene styrene) for the model and hydrosoluble PVA for the support with a layer height of 200 microns. All pelvises were printed in a similar orientation with the long axis parallel to the printing surface. Upon printing, the dissolvable PVA was washed away with water and the pelvis was left to dry.

Measurements on printing model and geometric coordinate acquisitions

Five anatomical landmarks (Fig. 1) [14] were chosen based on the observers' ability to localize them on both the 3D printed model and CT scan images, and were assigned a letter as follows: A, ischiatic tuberosity at its most distal aspect; B, ischiatic spine at its most dorsal aspect; C, insertion site of the rectus femoris, as characterized by its raised area on the lateral surface of the body of the ilium; D, caudal dorsal iliac spine, also known as the eminence of the iliac crest; and E, cranial dorsal iliac spine, which is defined anatomically as the obtuse angle between the cranial and dorsal border of the iliac spine [15]. All three observers had prior clinical experience in measuring 3D rendered models on a computer and gross measurements on the anatomic specimen. The comparison of the 3D printed models with the CT scan images was based on the length of four segments: BC, AE, BD, and CD. Prior to the beginning of the study, the observers came to an agreement on a standardized methodology for the measurements. The three different observers repeated each series of measurements twice.

Fig. 1. Anatomic landmarks used for the measurements on a gross specimen. (A) ischiatic tuberosity at its most distal aspect; (B) ischiatic spine at its most dorsal aspect; (C) insertion site of the rectus femoris, as characterized by its raised area on the lateral surface of the body of the ilium; (D) caudal dorsal iliac spine also known as eminence of the iliac crest; and € cranial dorsal iliac spine, defined anatomically as the obtuse angle between the cranial and dorsal border of the iliac spine.

On the CT-scan images using the CT-scan software, the automatically calculated (x, y, z) coordinates of the anatomical landmarks were recorded and used to calculate the length between the landmarks using the formula, l_ab = √ [(x_a−x_b)²+(y_a−y_b)²+(z_a−z_b)²], which is derived from the Pythagorean theorem (Fig. 2).

Fig. 2. Illustration of how the coordinates of the anatomical landmarks in a three plane Cartesian system were used to calculate the distance between the aforementioned points.

Direct measurements of the lengths of the four segments were performed on the 3D models with a scientific digital caliper with a precision of 0.1 mm on each printed model. Using this method and the concept of geometrical analysis, a limited evaluation of the pelvic morphology was also performed [16,17].

Statistical analysis

All measurements were recorded in centimeters to two decimals (Tables 1 and 2); the measurement differences between the averages of the measurements of both methods were calculated and expressed as absolute values (Table 3). The interobserver agreements and comparison between the measurements obtained were evaluated using the concordance correlation coefficients (CCC) [18,19,20]. Statistical analyses were performed using the software R 3.2.3 and its agreement package (R Foundation for Statistical Computing, Vienna, Austria). The statistical tests allowed an assessment of the precision (i.e., index of variability of the measurement) and accuracy (i.e. are all measurements centered on the same value). The precision and accuracy were evaluated according to a currently published scale [21], with agreement considered perfect, almost perfect, substantial, moderate, or poor when= 1.0, > 0.99, 0.95–0.99, 0.90–0.95, or < 0.90, respectively [21].

Table 1. Measurements for each anatomic segment, as obtained on the three-dimensional printed models.

Dogs	BC						AE						BD						CD
	A1	A2	B1	B2	C1	C2	A1	A2	B1	B2	C1	C2	A1	A2	B1	B2	C1	C2	A1	A2	B1	B2	C1	C2
Dog 1	2.80	2.70	2.75	2.80	3.00	2.90	8.10	8.00	8.10	8.10	8.10	8.10	3.90	4.10	3.80	3.90	3.90	3.80	3.50	3.40	3.05	2.85	3.40	3.20
Dog 2	4.70	4.60	4.40	4.50	4.70	4.40	13.40	13.80	13.40	13.40	13.80	13.70	6.30	6.10	6.70	6.50	6.30	6.20	5.40	5.30	5.20	5.00	5.00	4.90
Dog 3	3.50	3.40	3.40	3.40	3.40	3.60	10.60	10.70	10.50	10.40	10.30	10.50	5.70	5.70	5.70	5.50	5.40	5.30	5.10	5.20	4.30	4.20	4.20	4.50
Dog 4	2.80	2.90	2.90	2.70	2.90	2.70	9.60	9.60	9.50	9.60	9.40	9.60	4.30	4.30	4.40	4.50	4.40	4.50	3.80	3.90	3.50	3.55	3.40	3.40
Dog 5	2.20	2.20	2.10	2.10	2.40	2.00	7.70	7.70	7.50	7.70	7.70	7.80	3.80	3.70	3.60	3.60	3.70	3.60	3.00	3.10	2.80	2.90	2.80	3.10
Dog 6	3.10	3.20	3.30	3.20	3.80	3.40	10.10	10.20	10.20	10.30	10.30	10.30	5.10	5.20	5.00	5.20	5.10	4.80	4.10	4.20	3.70	3.80	3.70	3.70
Dog 7	4.00	3.90	3.80	4.00	4.30	4.20	12.90	13.00	13.10	13.00	12.90	13.00	5.80	5.60	5.50	5.50	5.90	5.70	4.80	4.90	4.50	4.40	4.30	4.20
Dog 8	4.60	4.40	4.20	4.10	4.40	4.20	11.60	11.40	11.40	11.20	11.70	11.60	5.40	5.40	5.30	5.20	5.70	5.30	4.20	4.30	4.20	4.20	3.90	4.10
Dog 9	3.30	3.30	2.85	2.90	3.60	3.30	10.20	10.30	10.20	10.15	10.30	10.30	4.50	4.40	4.30	4.40	4.50	4.40	3.70	3.70	3.50	3.50	3.30	3.40
Dog 10	1.80	1.90	1.80	1.80	2.10	2.00	5.40	5.40	5.40	5.30	5.60	5.50	2.80	2.70	2.85	2.70	2.70	2.80	2.60	2.60	2.40	2.30	2.30	2.40

All measurements are reported in centimeters to two decimals. Each observer and the occurrence of their observation is recorded as A1, A2, B1, B2, C1, and C2. Each observer was assigned a letter (A, B and C), and the observation time was recorded as either 1 or 2.

Table 2. Measurements of each anatomic segment, as obtained on the computed tomography data based on the coordinates of the anatomical landmark and the formula, l_ab = √ [(x_a−x_b)²+(y_a−y_b)²+(z_a−z_b)²].

Dogs	BC						AE						BD						CD
	A1	A2	B1	B2	C1	C2	A1	A2	B1	B2	C1	C2	A1	A2	B1	B2	C1	C2	A1	A2	B1	B2	C1	C2
Dog 1	2.87	2.75	2.65	2.75	2.91	2.83	8.42	8.08	7.96	7.94	7.97	8.02	3.94	3.88	3.88	3.75	3.80	3.77	3.47	3.54	2.99	2.83	3.21	3.15
Dog 2	4.55	4.54	4.57	4.24	4.52	4.50	13.82	13.89	13.53	13.53	13.41	13.69	6.11	6.31	6.31	6.08	5.90	6.11	5.17	5.20	5.40	4.88	4.85	5.01
Dog 3	3.30	3.38	3.22	3.21	3.13	3.55	10.28	10.62	10.40	10.37	10.64	10.45	4.72	4.88	4.88	5.08	4.96	4.85	4.27	4.33	4.46	3.94	4.01	4.23
Dog 4	2.67	2.88	2.69	2.77	2.85	2.91	9.65	9.71	9.44	9.62	9.78	9.60	4.17	4.43	4.43	4.63	4.38	4.28	3.87	3.72	3.39	3.46	3.61	3.33
Dog 5	2.29	2.18	2.25	2.22	2.28	2.17	8.11	7.96	7.52	7.99	7.62	7.95	3.65	3.60	3.60	3.45	3.37	3.38	2.91	3.08	2.64	2.79	2.54	2.85
Dog 6	3.32	3.31	3.20	3.40	3.68	3.36	10.55	10.47	10.21	10.50	10.12	10.33	5.06	5.18	5.18	5.25	4.83	4.73	4.23	4.12	3.62	4.00	3.49	3.82
Dog 7	4.03	3.80	3.52	3.90	4.40	4.18	13.07	12.81	13.06	13.10	13.09	13.06	5.79	5.80	5.80	5.54	5.71	5.50	4.93	5.24	4.25	4.13	4.22	4.11
Dog 8	4.11	4.12	4.06	3.91	4.45	4.22	11.89	11.78	11.47	11.33	11.96	11.93	5.23	5.28	5.28	5.19	5.48	5.24	4.36	4.28	4.03	4.15	4.04	4.23
Dog 9	3.22	3.14	2.89	3.22	3.64	3.55	10.07	10.27	10.10	10.12	10.28	10.19	4.42	4.38	4.38	4.34	4.65	4.56	3.79	3.70	3.47	3.32	3.33	3.46
Dog 10	1.80	1.91	1.88	1.92	2.15	1.98	5.55	5.65	5.47	5.54	5.51	5.53	2.76	2.82	2.82	2.72	2.87	2.71	2.66	2.53	2.23	2.32	2.38	2.38

All measurements are reported in centimeters with two decimals. Each observer and the occurrence of their observation is recorded as A1, A2, B1, B2, C1, and C2.

Table 3. Average of the measurements obtained among all three observers and both observations separated between measurements based on the 3D printed models and the measurements obtained on the CT data.

Dogs	Average of the observed values on the anatomical data (cm)				Average of the measurement on the CT data (cm)				Difference between the measurement between 3D model and CT (cm)
	BC	AE	BD	CD	BC	AE	BD	CD	BC	AE	BD	CD
Dog 1	2.83	8.08	3.90	3.23	2.79	8.06	3.84	3.20	0.03	0.02	0.06	0.03
Dog 2	4.55	13.58	6.35	5.13	4.49	13.64	6.24	5.08	0.06	−0.06	0.11	0.05
Dog 3	3.45	10.50	5.55	4.58	3.30	10.46	5.00	4.21	0.15	0.04	0.55	0.38
Dog 4	2.82	9.55	4.40	3.59	2.80	9.63	4.39	3.56	0.02	−0.08	0.01	0.03
Dog 5	2.17	7.68	3.67	2.95	2.23	7.86	3.50	2.80	−0.06	−0.17	0.17	0.15
Dog 6	3.33	10.23	5.07	3.87	3.38	10.36	5.00	3.88	−0.05	−0.13	0.07	−0.01
Dog 7	4.03	12.98	5.67	4.52	3.97	13.03	5.59	4.48	0.06	−0.05	0.08	0.04
Dog 8	4.32	11.48	5.38	4.15	4.14	11.73	5.25	4.18	0.17	−0.24	0.13	−0.03
Dog 9	3.21	10.24	4.42	3.52	3.27	10.17	4.43	3.51	−0.07	0.07	−0.01	0.01
Dog 10	1.90	5.43	2.76	2.43	1.94	5.54	2.77	2.42	−0.04	−0.11	−0.01	0.01

All measurements are reported in cm with two decimals. The difference between the averages was calculated and expressed in absolute value ranging from 0.01 cm to 0.38 cm.

3D, three-dimensional; CT, computed tomography.

RESULTS

The precision and accuracy of repeated measurements using the caliper on the 3D printed pelvises by the three observers were almost perfect for the AE anatomical line (precision: 0.998; accuracy: 1; Table 4). The caliper measurements of BD, BC, and CD showed substantial precision (0.982, 0.969 and 0.962 for BD, BC and CD, respectively). The accuracy of the measurements of BD and BC were perfect and almost perfect, respectively (accuracy: 1.00 and 0.984, respectively). On the other hand, the caliper measurements of CD showed only moderate accuracy (0.934).

Table 4. Intra-observer, inter-observer (k = 3), and total agreement between repeated measurement (m = 2) of anatomical lines on 3D-printed pelvises and on CT-scan volume reconstruction (n = 10).

Variables		BC (CT data)	BC (3D model)	AE (CT data)	AE (3D model)	BD (CT data)	BD (3D model)	CD (CT data)	CD (3D model)
Intra-observer
	CCC	0.972	0.979	0.997	0.998	0.971	0.987	0.960	0.986
	Precision	0.972	0.979	0.997	0.998	0.971	0.987	0.960	0.986
Inter-observer
	CCC	0.960	0.964	0.996	0.998	0.971	0.988	0.900	0.904
	Precision	0.983	0.980	0.997	0.998	0.972	0.988	0.961	0.969
	Accuracy	0.976	0.984	0.999	1.000	0.999	1.000	0.937	0.933
Total
	CCC	0.947	0.954	0.994	0.997	0.957	0.981	0.883	0.898
	Precision	0.969	0.969	0.996	0.998	0.958	0.982	0.942	0.962
	Accuracy	0.977	0.984	0.999	1.000	0.999	1.000	0.938	0.934

3D, three-dimensional; CT, computed tomography; CCC, concordance correlation coefficient.

The precision of repeated measurements on the CT data by the three observers was almost perfect for AE (0.996), substantial for BC and BD, and moderate for CD (0.969, 0.958 and 0.942, respectively). The accuracy was almost perfect for AE and BD (0.999 for both measurements), substantial for BC, and moderate for CD (0.977 and 0.938, respectively; Table 1).

The accuracy of measurements between both methods was > 0.99 for all measurements (Table 5). The precision was almost perfect for AE (0.996) and substantial to moderate for the other measurements (0.963, 0.958, and 0.912 for BD, BC, and CD, respectively). 
Although the measurements did not show perfect precision between methods, they were very accurate. This indicates that, on average, the length of each segment measured was almost identical, irrespective of the method used.

Table 5. Intra-methods, inter-methods (k = 2), and total agreement between repeated measurements (m = 6) of anatomical lines on the three-dimensional-printed pelvis (n = 10).

Variables		Anatomic distances
		BC	AE	BD	CD
Intra-methods
	CCC	0.955	0.996	0.972	0.906
	Precision	0.955	0.996	0.972	0.906
Inter-methods
	CCC	0.994	0.999	0.980	0.987
	Precision	0.995	0.999	0.986	0.990
	Accuracy	0.999	1.000	0.994	0.998
Total
	CCC	0.957	0.996	0.957	0.910
	Precision	0.958	0.996	0.963	0.912
	Accuracy	0.999	1.000	0.994	0.998

CCC, concordance correlation coefficient.

DISCUSSION

The current results indicate that a dual-FDM 3D printer reliably and accurately reproduces the size and geometry of the pelvis, as measured on CT-scan. The pitfalls of this study were fivefold. First, being a prospective and opportunistic study, the CT parameters varied between patients and were chosen on a clinical basis and not optimized for 3D image printing [22,23]. Ideally, repeating the CT-scan with a smaller FOV and smaller slice thickness (submillimeter) is needed. This would have resulted in an increased dose delivered to the patient and anesthesia time, both of those being considered questionable ethically by the authors. On the other hand, as these results indicate a high degree of accuracy between the 3D models and the CT data, despite the use of thicker slices, it would appear intuitive that thinner slices, 1mm slices, having been described as having higher accuracy [23], would have improved the accuracy of the models further and strengthened the study results.

Second, as this was an opportunistic design, none of the dogs used in the study underwent a necropsy, which prevented a comparison between the data and the gold standard of gross pathology. Despite this, CT is an accurate way to evaluate and represent the anatomy [24,25,26,27,28]. Therefore, one could logically assume that if the printed pelvises match the CT images, they would also match the gross anatomy.

Third, in the current study, the scale designed by McBride [21] was used to interpret and define the agreement between the observers and methods. Although used widely, it may not reflect a relevant evaluation of the precision and accuracy required for clinical purposes. Designing a scale adapted to this question would have required a definition of what a clinically acceptable difference in size between a bone and its 3D printed model is, but this was beyond the scope of the present study and would likely be controversial.

Fourth, the morphology comparison of the 3D printed models and CT data relied on the analytic geometry. By comparing the distances between multiple points located on 3D different planes, a limited comparison of the geometry was also performed. This is most apparent when the measurements BC, BD, and CD are compared to one another as they produce a pyramidal shape. If the geometry of the pelvis between the 3D model and the CT data were to be discordant, there would be further discrepancies among the three measurements [17]. More advanced and thorough methods, such as optical 3D scanning [8] and surface shape analysis would have provided additional information on the accuracy between the model and the CT data but this was unavailable to the authors.

Lastly, there is considerable variation in the methods available to produce the 3D models from the CT data and further additional parameters to print the models, which could have influenced the study results. On the other hand, the technologies, parameters and methods (four slice CT, standard bone kernel protocol, 2mm slice thickness, FDM printer, 200 microns resolution, use of common polymers) used in this research were chosen as being readily available and clinically reproducible [23]. Hence, this is likely to be clinically applicable to veterinary surgeons in the future.

FDM printers are readily available, cost effective, and do not require extensive maintenance [23]. Being the most popular of consumer grade 3D printers [23], they will most likely be the type found in veterinary hospitals and clinics in the near future. In human medicine, several studies have reported strong accuracy between CT scans combined with 3D printing techniques to produce different bone models, such as a skull or mandible [29,30]. The veterinary scientific literature currently available lacks thorough studies evaluating this new technology. Several veterinary publications have described the use of 3D printing in the planning of angular limb correction surgery without confirming whether the printed model represents the shape of the bone reliably [31,32,33]. A single veterinary publication validated both steps by studying the concordance between the shape of the femur and printed models after different acquisition and printing methods, but it did not evaluate the use of dm-FDM [8] nor did it evaluate their methods for a more geometrically complex bone, such as the pelvis.

The minor differences in measurements between the CT data and 3D printed models were due most likely to a combination of the described above pitfalls.

When printing shallow slopes and complex shapes with FDM, a support material acting as a scaffolding is often required. The remnant of the supports after their removal can affect the smoothness, regularity, and accuracy of the object printed [9,13]. This can be overcome easily with dual-material FDM because it prints two types of material simultaneously, such as one soluble for the support material and another polymer for the 3D model.

Similar to the research conducted in human medicine, the models evaluated in the current study were clinically aimed at improving the planning of pelvic and acetabular surgery [7,34]. The use of 3D printed models to measure and choose surgical implants can be supported scientifically by demonstrating that on average, the length of each segment measured was almost identical between the two methods using the reported parameters. Additional study evaluating the surface contour of the pelvis more thoroughly would be useful in the future to further support the practice of contouring the surgical implants. The use of 3D printing in the preoperative planning of the surgery has been documented previously to decrease the length, morbidity, mortality, and cost of canine limb deformity treatment [35]. Whether this is also true for the treatment of pelvic fractures has yet to be determined.

In conclusion, this study lays the groundwork in validating the accuracy of 3D printed models when compared to CT-data and therefore to the anatomy in general [25,28,36,37,38,39]. Additional studies comparing 3D printed models with anatomical specimens would be beneficial to the field because 3D models are being used more commonly in veterinary medicine to practice surgery, contour implants, as well as to improve communication with clients and students.

Demonstrating that the measurements made on the 3D models are true to those obtained from the CT data is the first step in scientifically validating the use of 3D models for presurgical planning. This opens the pathway for additional studies evaluating the geometry of 3D printed models and determining if the preoperative contouring of plates on 3D models allows a better reduction of the bone when compared to perioperative plate contouring.