Three-Dimensional Surface Imaging and Printing in Anatomic Pathology

Abstract

Three-dimensional (3D) imaging is increasingly being incorporated into a variety of medical specialties: surgery and radiology being but two prominent examples. Image-intensive disciplines, such as anatomic pathology (AP), represent excellent potential candidates for further exploration of this innovative technology. Multiple potential use cases exist within AP, involving patient care, education, and research. These use cases broadly include direct utilization of the 3D digital assets for viewing on a 2D screen, populating 3D extended reality platforms (virtual reality, augmented reality, and mixed reality) as well as generation of 3D printed photorealistic specimen models. Herein, these use cases are explored with specific regard to our experiences and yet unrealized potential. Future directions and considerations are also discussed.

INTRODUCTION

A picture is worth a thousand words. This familiar adage drives home the point that complex and esoteric concepts may be readily conveyed by a simple image. Perhaps nowhere is this truer than medicine, wherein medical imaging and illustration provide important diagnostic and documentary information.

While generally applicable to all medical specialties, the emphasis on imagery is of specific relevance to the field of anatomic pathology (AP). This specialty, dedicated to recognition of the macroscopic (“gross”) and microscopic findings of disease, has long been reliant on imaging to document and convey information to other pathologists, the treating team, and even patients themselves.

Drawings, sketches, and paintings were commonly used for documentation and communication in medicine dating back to 300 B. C., allowing for important collaboration to advance our understanding of disease.[1] Dorothy Reed's drawings, for instance, of the atypical cells she noted in cases of Hodgkin lymphoma while working as a fellow at The Johns Hopkins Hospital, were noted to be similar to those demonstrated by Carl Sternberg several years prior in 1898 [Figure 1].[2] The detailed illustrations, combined with astute observation, allowed for recognition of the eponymous Reed–Sternberg cell, a pathognomonic finding still useful today.

Figure 1.

Sketches as medical illustrations. Dorothy Reed documented the atypical cells present in Hodgkin lymphoma via hand-drawn sketches (a, reproduced with permissions),[3] which were later compared to the near contemporaneous drawings by Carl Sternberg (b, public domain).[4] The striking similarities between the two observations resulted in moniker of the Reed–Sternberg cell, still used today

Photography represented a major technologic advancement that would further advance the field of medical illustration. In 1839, Alfred Donné and Louis Daguerrein first documented slide preparations, using this new technique, to teach medical students [Figure 2a].[5] It is interesting to note that one of the first applications of photography in medicine was photographing objects seen under a microscope. Rudimentary black and white photography advanced to the near-ubiquitous access to full color, high-resolution, digital photography and photomicrography seen today [Figure 2b].

Figure 2.

Evolution of medical documentation via the photomicrograph. One of the first applications of photography was that of the photomicrograph, as illustrated in Alfred Donne's Atlas du Cours de Microscopie, and entitled “Epidermal cells of normal vaginal mucosa” (translated) in 1845 (a, public domain).[6] Today, photomicrographs are still utilized to document and convey vital concepts in anatomic pathology (b, ThinPrep with Papanicolaou stain, ×600 original magnification)

Transformation to digital platforms, along with advances in data storage, has allowed for rapid accumulation of massive medical image datasets. Increasingly, whole slide imaging, artificial intelligence, and machine learning algorithms are being used to facilitate diagnosis and enhance interpretation. Moreover, images are extensively used in pathology education to help train the next generation of pathologists to visually recognize the spectrum of disease.

While unquestionably valuable, traditional medical illustration has been somewhat limited by its two-dimensional (2D) nature. Three-dimensional image acquisition has, however, come to the fore in a number of areas, including medicine, offering unique and interesting opportunities to push the field forward [Table 1].[7] These 3D medical datasets have myriad application in the arenas of clinical practice, education, and research. Several different technologies contribute to obtaining these 3D data sets including 3D reconstructions of radiology cross-sectional imaging, medical models/sculptures, 3D illustration, 3D animation, 3D reconstructions of electron microscopy, 3D printing, 3D surface scanning, and photogrammetry. Several surface scanning methods exist to capture 3D geometry of pathologic specimens including photogrammetry, structured light scanning, and laser scanning. Herein, we describe the application of 3D image acquisition to AP, using structured light scanning and 3D physical reproduction using 3D printing. This has the potential to usher in a new era in medical illustration and imaging, helping to revolutionize AP practice.

Table 1.

Advantages of three-dimensional scans/prints over specimens

Nonbiohazardous

Durability

Portability

Accessibility

Reproducibility

Space conscious

TECHNICAL METHODS

Three-dimensional image acquisition

A handheld 3D scanner (Space Spider, Artec 3D [Santa Clara, CA, USA]) was used for all image acquisition, including both the geometry (shape) and texture (color and surface pattern) data [Figure 3] by projecting a pattern of alternating light on the item while simultaneously photographing the specimen. 3D point accuracy was 0.05 mm with a resolution of 0.1 mm in all scans. Ultimate texture resolution was captured at 1.3 megapixels with 24 bits per pixel. Acquisition was performed at 7.5 frames per second with a 0.0002 second 3D exposure time, illuminated by onboard blue and white light-emitting diodes.

Figure 3.

Equipment utilized in three-dimensional image acquisition. A handheld device (Space Spider, Artec 3D [Santa Clara, CA, USA]) with onboard blue and white light-emitting diodes (LEDs) was utilized to obtain specimen geometry (shape) and texture (color and surface pattern) data (a). Innovations such as mounting of the acquisition device on a tripod, utilization of a turntable with a nonreflective, black surface, and the use of color-coded locator pins increased both scan quality and ease of postscan processing (b)

Specimens were formalin fixed prior to scanning and dissected for optimum demonstration of key anatomic or pathologic features. Two sequential scans were prepared (top and bottom) of each specimen while rotating the specimen on a turntable, coated with a nonreflective, black, acrylic surface. Three unique (color-coded) locator pins were placed in specimens to facilitate alignment of the resulting point cloud of both the top and bottom scans. Depth of field was maintained between 223 and 330 mm for orientation and acquisition. Texture brightness was adjusted for optimal acquisition depending on the nature of the tissue to be scanned.

Artec Studio (versions 9.0–14.0 Hamm, Luxembourg) modeling software was used during scanning and processing [Figure 4]. Specimen geometry and texture were used as algorithmic reference during acquisition. Image alignment was done manually and then registrations (rough, fine, and global) were performed sequentially by software algorithms based primarily on specimen geometry. The image base (including any residual scanned portion of the turntable surface) was manually eliminated with the cutoff plane selection tool. Undesired portions of the specimens were manually abstracted using the 3D cutoff tool on an ad hoc basis. Outlier removal algorithms were applied to eliminate superfluous objects acquired during the scanning process, performed at a 0.3 mm resolution (identical to the sharp fusion resolution applied during final image construction). Digital objects were hole filled (watertight) and mesh simplified (generally to <150,000 triangles) to facilitate image transfer and printing.

Figure 4.

Image processing progression. (a) Point cloud. (b) Three-dimensional mesh (simple geometry). (c) Three-dimensional mesh (complex, smoothed, geometry). (d) Texture-mapped three-dimensional image

Final texture was applied to the scan (JPG format) by the modeling software, and the final files were saved in virtual reality modeling language (VRML 2.0) format. Final file sizes were ~15 MB per specimen.

Three-dimensional printing

All specimens were printed on either binder jetting on a Projet 660 (3D Systems, Littleton, CO, USA) or material jetting on a Mimaki 3D UJ-553 (Mimaki, Nagano, Japan) and Stratasys (Eden Prairie, MN, USA) Objet 500 or J55. Of note, the benefits of binder jetting are printer speed, ease of support material removal, and low cost while the disadvantages are fragility, reduced detail, need of infiltration, and reduced color palette (1 million colors). The material jetting printers, Mimaki and Stratasys, have a greater color range (10 million colors) and hence more photorealism but the materials are up to ×10 more expensive to print and need specialize postprocessing units including lye baths for soaking and water jetting for removal of photopolymetric support material [Table 2]. These photorealistic printers are a great advancement over single color or multitone printers, particularly for anatomic specimens [Figure 5].

Table 2.

Three-dimensional printer comparisons for pathologic specimens

	3D printers
	Binder jetting Projet (660)	Material jetting
		Stratasys (J750)	Mimaki 3D (UJ-553)
Specifications
Build volume	254 mm×381 mm×203 mm	490 mm×390 mm×200 mm	508 mm×508 mm×305 mm
Resolution	600 dpi×540 dpi	600 dpi×1800 dpi	600 dpi×1200 dpi
Layer thickness	0.1 mm	0.014 mm	0.02 mm
Build speed	28 mm/h	80 g/h	7 mm/h
Specimen examples
Heart
Whole
Cost	$87.34	$170.11	$325.60
Print time	3 h 52 min	10 h 52 min	24 h 1 min
Postprocess time	2 h	1 h	5 h
Short-axis section
Cost	$21.00	$30.18	$32.36
Print time	33 min	1 h 50 min	2 h 18 min
Postprocess time	1 h	30 min	2 h
Lung
Whole
Cost	$147.13	$367.59	$454.97
Print time	4 h 38 min	15 h 27 min	32 h 14 min
Postprocess time	1 h	1 h	3 h
Parasagittal section
Cost	$51.24	$91.60	$137.99
Print time	1 h 19 min	3 h 42 min	5 h 39 min
Postprocess time	1 h	30 min	2 h
Brain
Whole
Cost	$227.13	$647.44	$853.22
Print time	7 h 6 min	26 h 37 min	30 h 32 min
Postprocess time	1 h	30 min	6 h
Coronal section
Cost	$17.10	$23.25	$22.49
Print time	40 min	2 h 6 min	2 h 57 min
Postprocess time	1 h	30 min	2 h

3D: Three dimensional

Figure 5.

Specimen and evolution of material jetting three-dimensional printed models. (a) Original specimen. (b) Early three-dimensional photorealistic printed model in grayscale. (c) Contemporary three-dimensional photorealistic model printed in full color

All specimens printed on the Projet 660 were infiltrated with cyanoacrylate and then clear coated during the postprocessing step. All prints were clear coated after the support material was removed. All printed 3D pathologic models were postprocessed using the standard techniques as supplied by the manufacturer. Quality control steps to assure accurate manufacturing included caliper measurements with comparison to know triangulated data on CAD file, visual inspection, and color matching. If the model was deemed fragile based on fine structures as well as size and print modality, a 3D printed support cradle for holding the specimen was printed at the same time.

USE OF THREE-DIMENSIONAL IMAGES

The use of 3D imagery spans the three major domains of medical practice: clinical, education, and research, with numerous use cases in each. This includes utilization of both acquired digital images and physical (printed) specimens. Digital images can be used as independent assets (either as 3D PDFs or VRMLs) or as incorporated elements in broader digital mediums, such as 3D on 2D mobile and fixed screens or in mixed reality (MR) content. Examples of the utility of 3D specimen capture are described below, but by no means is it intended to be a comprehensive list of all the possibilities unleashed by this technology. Rather, the focus is the areas with which we have direct experience and have active protocols to study the utility of the technology.

Clinical

The practice of AP is broad, encompassing general areas of surgical pathology, autopsy pathology, and cytopathology. Gross pathologic examination is an integral component of these disciplines, so much so that the task of evaluation and dissection is restricted by accreditation to high-complexity testing personnel. Narratives describing gross findings and employed dissection techniques are required elements of surgical pathology reports. Increasingly, these descriptions are enhanced by referenced photographs of the specimens themselves to capture and convey important gross information.

Such photographs are, however, inherently limited by what is believed to be important at the time of acquisition, when critical information (such as diagnosis) may not yet be available. As additional diagnostic or clinical information comes to light, As additional diagnostic or clinical information come to light, different dissection techniques may be required than that which was originally performed. Routine 3D surface scanning of specimens, both prior to and following dissection, is a reliable solution. The digitally captured 3D specimen offers opportunity for hindsight manipulation and can be a valuable asset when discussing the case with colleagues both within and outside pathology.

Certain surgical pathology specimens, explanted organs in particular, are common targets of questions from patients and their families. We have found that patients from our solid-organ transplant practice, in particular, are often deeply invested in their disease process and commonly express curiosity about their explanted organs. This curiosity may involve a desire to see the organ in order to better understand their disease and achieve some closure around their experience. Consequently, requests for organ viewings have, over the years, become relatively commonplace at our institution. Issues arise, however, relating to biohazardous materials and finding appropriate accommodation to allow for these viewings. Having the digital image or a 3D printout of a patient's explant removes these barriers and not only allows the patient to achieve closure and learn about their disease in a safe way but also facilitates the development of a relationship between pathologist and patient as the disease is discussed and questions are addressed.

Multidisciplinary conferences or tumor boards are another forum in which questions about gross specimens commonly arise. Having access to high-fidelity 3D images that recapitulate the specimen as it was procured from the patient can be an invaluable asset to highlight salient points regarding margin status or invasion. Further yet, having a 3D print of the specimen that can be held and passed around the room, can also be helpful in advancing the clinical discussion.

Similar to opportunities in surgical pathology, autopsy pathology offers additional important clinical opportunities. The ability to scan and reproduce 3D renderings of postmortem findings can be invaluable at morbidity and mortality conferences, routinely used for institutional quality assurance and control. Perhaps more specific to autopsy is the application of this technology to forensic pathology wherein 3D capture of gross findings can serve as the basis for reproducing important findings for purposes of litigation. For example, seeing a 2D image of a broken lateral horn of the hyoid, secondary to manual strangulation, may not be as compelling as holding the 3D printout of such in one's hand which relays victim specific size, photorealistic color, and provides a deeper understanding through haptic perception [Figure 6]. Ultimately, being able to capture these findings also may reduce the need for specimen retention and ensuring integrity of the findings in perpetuity, as digital images are not subject to the same deterioration as tissue samples and can be reprinted from the file data at any time.

Figure 6.

Patient–pathologist meeting to review a three-dimensional model of lung explant. Three-dimensional models have enhanced and facilitated patient–pathologist relationships, as demonstrated by regular requests to review explanted organs by our transplant patients. These models allow for safe handling of otherwise biohazardous specimens, and augment patient understanding of his/her disease process

Education

Medical education is replete with opportunities for incorporation of 3D technologies to enhance learner experience and performance. Printed specimens in the classroom or at the microscope offer many of the benefits of traditional gross teaching such as the ability to rotate and orient the specimen allowing the learner to better understand the 3D relationships without the restrictions imposed by biohazardous specimens [Figure 7]. Indeed, there is a growing body of literature to support the notion that 3D models may enhance learning.[8,9] In addition, some parity has been established between 3D models and cadaveric specimens.[10] The use of a high-quality printer that allows for accurate color and texture reproduction yields excellent results when comparing specimen and model [Figure 8].

Figure 7.

Three-dimensional printed models used in education. Integration of three-dimensional models into the classroom has resulted in improved safety and appreciation of anatomic relationships. Hazards requisite to tissue handing, such as the handling of formalin and biologic materials, had previously been circumvented at our institution by suspending specimens in clear plastic boxes filled with formalin (as seen on the table in the image). However, this solution was suboptimal, as it limits tactile specimen interaction and therefore appreciation of important anatomic relationship. Utilization of photorealistic three-dimensional models obviates the need for these accommodations, allowing for another step forward in medical education

Figure 8.

Specimen and three-dimensional printed models. Scanning of the original specimens (a and c) can lead to high-quality, photorealistic three-dimensional models (b and d) that can be utilized, for example, in the classroom with respect to (b) congenital heart disease, and (d) in forensic proceedings in the case of a fracture of the superior horn of the thyroid cartilage during manual strangulation (arrow)

This portability opens additional opportunities, such as being able to easily transport gross specimens to medical conferences or share these invaluable and unique assets with colleagues worldwide. Sending 3D printed specimens or digital files (for local printing, if the resources exist) allows for wide dissemination of rare specimens that offer up important teaching points. It affords more opportunity for democratizing valuable educational content, particularly to underserved areas.

In addition to specimen sharing, 3D specimen image acquisition also creates content that can be incorporated into MR learning environments. Having a photorealistic representation of a Whipple specimen, for example, with which a user is virtually interacting via headset, enables safe and on-demand education outside of the high-pressure situation of a gross room. An entire virtual grossing manual could be created with some ease, allowing trainees to hone their skills with unique specimens before encountering the real thing. Creation of assessments can help ensure baseline competence of a trainee prior to rotation in a busy laboratory, thus allowing for maintenance of laboratory efficiency and optimal use of in-person time between trainer and trainee. Immersive, on-demand, experiences such as this can truly revolutionize our approach to teaching trainees and are increasingly desired by learners.

These same digital assets can be useful adjuncts during didactics or small group educational sessions. They allow for real-time placement of virtual specimens anytime and anywhere on any device, allowing a new level of in-classroom engagement.

Research

Routine 3D imaging increases the amount of data captured for cases which can be invaluable for research. Research in pathology frequently involves the retrospective pooling of samples for study. One of the limitations of retrospective studies is the fact that certain aspects may not have been routinely evaluated at the time of grossing. Having a complete 3D rendering of the specimen allows for more comprehensive retrospective specimen analysis.

Acquired images can also be incorporated into larger datasets, introducing more complex gross variables into machine learning and artificial intelligence-based algorithms.

Finally, having accompanying 3D renderings of specimens can make for tremendously valuable assets during research poster or platform presentations. In such forums, they can take presenter-audience engagement to the next level.

FUTURE DIRECTIONS

3D imaging and printing has begun to make its way into various areas of medicine, led by a handful of specialties such as radiology and surgery.[11] Uses have included physical reproduction of radiologic images to aid in surgical planning and patient education, creation of prosthetic devices, custom patient-matched osteotomy guides, and reproduction of medical instruments and safety equipment.[12] Creation of MR content is increasingly commonplace in the arenas of clinical care and education. These MR platforms are beginning to be incorporated into pathology as well and thus there will be a need to generate assets and content that can feed these new and exciting platforms.[13]

The work herein describes and further expands the use of 3D technologies within AP.

Laying out the potential use cases for this new and exciting technology is but a first step. Broader implementation and eventual study of the outcomes of incorporating 3D imaging and printing into routine practice will undoubtedly be needed before widespread adoption. The case for investment in this technology is strong, with use cases touching many aspects of our practice in the key domains of clinical practice, education, and research.

Cost and time are important factors that will limit how quickly technology such as this can be deployed. Newer technologies that allow for more rapid image acquisition will be necessary. Photogrammetry is likely the next evolution in this space.[14] In contrast to manual scanning via a handheld device, photogrammetry offers the ability to turn a static specimen into a digital 3D rendering in a more automated fashion. Specimens could be placed within a photogrammetry unit, and with the push of a button, a complete rendering would be available within minutes without deep technologic knowledge of 3D file formats and coding. This could allow for such devices to be readily deployed in the laboratory, with rapid, reliable image acquisition and vastly improved documentation of gross findings.

In addition, the opportunity for enhanced pathologist–patient interaction that this technology offers should not be understated. In the era of rapidly changing reimbursement, specialties seen as essential and value added by patients will likely be in the best position to lobby for their share of bundled payments. Opportunities such as this can help to educate the public on the importance of pathology and the critical role of the pathologist in patient care.

This will likely necessitate research assessing whether pathologist–patient interactions are value added and can affect outcomes. Study of educational use cases will be important as well. Consequently, we anticipate a need for formal pedagogical study on the results of using 3D technology to improve learning.

Pictures can indeed convey significant information in an efficient way. If, as suggested in the adage mentioned at the outset of this manuscript, the picture is worth a thousand words, the key will be to quantify the value we can glean from a 3D version of such.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.