Mixed reality and three dimensional printed models for resection of maxillary tumor: a case report

Abstract

In the field of oral and maxillofacial surgery, many institutions have recently begun using three-dimensional printers to create three-dimensional models and mixed reality in a variety of diseases. Here, we report the actual situation model which we made using three-dimensional printer from virtual operation data and the resection that was performed while grasping a maxillary benign tumor and neighboring three-dimensional structure by designing an application for Microsoft® HoloLens, and using Mixed Reality surgery support during the procedure.

Introduction

The preparation of a three-dimensional (3D) model from medical images using a 3D printer improves the safety of techniques, supports preoperative planning, and helps in education and training (1,2). Establishment of a surgical guide using computer-aided design/computer-aided manufacturing technology is linked to increased precision of the techniques used, improved safety, and reduced operating time (3). Image-guided surgery utilizing augmented reality (AR) based on medical imaging is used in several medical fields, such as plastic surgery (4), orthopedic surgery (5), neurosurgery (6), and oral surgery (7), and has been reported to be useful. However, AR requires a monitor, wherein the surgeon has to look away from the operative field. Advances in AR technology, which use head-mounted displays, have resolved this issue, by projecting 2D and 3D images in a real space without the need for looking away from the operative field. Among such technologies, the Microsoft^®︎ HoloLens (Microsoft Corp., Redmond, WA) has a head mounted sensor camera, and spatial mapping of the surrounding environment enables holographic image positioning, thus, achieving mixed reality (MR) with speech and/or gesture control. To date, several reports have been published on the use of Microsoft^® HoloLens (8,9). MR surgical support with superimposed holographic images on the patient enhances safety and reduces postoperative complications. Moreover, there are reports on the feasibility of surgical training and remote surgical support (10), but this technique is yet to be widely used in oral surgery.

In this work, we focused on a case of a patient where MR surgical support using Microsoft^® HoloLens was used to treat a maxillary calcifying odontogenic cyst that had progressed to the maxillary sinus and nasal cavity. This technique enabled the safe performance of surgery, in accordance with the preoperative planning. Details regarding the procedure, including the preoperative preparation protocol, are provided.

Case report

We examined the case of a 27-year-old woman who became aware of gingival swelling in the left anterior maxillary region in the year 2010; as she felt no pain, she left it untreated. When she underwent another dental treatment by a local dentist in April 2016, panoramic X-radiography showed a radiolucent area with unclear margins, in the left maxilla, extending from the anterior to the premolar region. Complete examination and treatment were recommended. However, the patient left the condition untreated, as she felt no pain. From March 2017, the patient became aware of the left anterior maxillary teeth movement and she was referred to our department for a complete examination.

The facial findings at the initial examination were deformation of the left nasal alar basal region and swelling of the left upper lip (Figure 1A). The intraoral findings were swelling and parchment feeling in the left maxillary region, from the lateral incisor to the buccal mucosa near the second premolar, and movement of the teeth in the same region (Figure 1B). There was no spontaneous pain. The electric pulp test showed a vital reaction in the left maxillary region, from the lateral incisor to the second premolar; however, there was no pain on pressure during swelling on the buccal side.

Figure 1.

Preoperative (A, face; B, oral) photographs and (C) panoramic image.

At the initial examination, panoramic X-radiography showed a unilocular radiolucent area with unclear margins, in the left maxillary region, from the anterior to the premolar region (Figure 1C), with several calcified areas in the interior. Computed tomography (CT) showed an approximately circular, low-density region with unclear margins that reached to the maxillary sinus and nasal cavity, containing several high-density, calcification-like structures of varying size (Figure 2A). Magnetic resonance imaging (MRI) showed a T2-weighted, unilocular lesion (Figure 2B).

Figure 2.

Preoperative computed tomography (A) and magnetic resonance image (B). (A) Computed tomography finding shows an approximately circular, low-density region with unclear margins, reaching the maxillary sinus and nasal cavity, in the interior of which there are high-density, calcification-like structures of varying size. (B) Magnetic resonance image showing a T2-weighted, unilocular lesion.

Then, a biopsy was performed under local anesthesia, which showed proliferation of tumor cells resembling the odontogenic epithelium, with a densely proliferative region, a plexiform region, and a region showing a luminal structure. In addition, massive and diffuse amyloid-like deposits and probable calcified tissues were observed. In the regions showing substantial tumor cell proliferation, rosette-like structures were observed, and the presence of an adenomatous odontogenic tumor was diagnosed.

The approach selected was radical treatment. We planned to perform partial resection of the left maxilla from the lateral incisor to the second premolar region and, then, to graft the iliac particulate cancellous bone and marrow into the post-resection void.

Written informed consent was obtained from the patient for publication of this study and any accompanying images.

Preoperative preparation (Virtual operation and Microsoft^® HoloLens application)

Using CT data obtained preoperatively with SOMATOM Definition AS (Siemens, Forchheim, Germany), a virtual operation was performed using Mimics (Materialise, Leuven, Belgium), and the tumor resection domain was decided upon. To determine the quantity of the iliac particulate cancellous bone and marrow needed after tumor resection, a mirror image of the post-resection void was superimposed on an image of the healthy side of the jaw, and the shape of the post-resection void was restored, enabling its volume to be measured. Next, using Materialise Mimics, the CT data were segmented, and standard triangulated language data were prepared for the jawbone, tumor, nasal cavity, and maxillary sinus. Region extraction was performed by region growing based on the CT values.

Further, these data were used to prepare a Microsoft^® HoloLens application with UNITY (April 17, 2017 flavor) and Visual Studio Community (2017/Version 15.9.4) (Figure 3). The anatomic structures that required special attention during surgery are depicted with the tumor, nasal cavity, and maxillary sinus marked as green, yellow, and pink, respectively (Figure 4A).

Figure 3.

Extended reality for medical imaging. Microsoft^® HoloLens application is prepared.

Figure 4.

(A) Microsoft^® HoloLens application. The tumor, nasal cavity, and maxillary sinus marked as green, yellow, and pink, respectively. (B) The resection domain is set, and the 3D model is prepared. 3D, three-dimensional.

In addition, using a 3D printer (Objet 260 Connex, Stratasys, Ltd., Rehovot, Israel), a 3D model detailing the tumor resection domain was prepared, and resection-domain and post-resection images were shared preoperatively between all surgery personnel (Figure 4B).

Throughout the operation, the three surgery personnel wore the Microsoft^® HoloLens, and surgery was performed while using virtual reality (VR) to ascertain the anatomic morphology of the maxillary sinus and nasal cavity surrounding the tumor (Figure 5A,B). The VR application provided the same 3D image to all surgeons and adjusted the transparency of the colors, thus, enabling surgery with superimposition on the patient (Figure 5C).

Figure 5.

A view of surgery in progress. (A) The surgeon and all assistants wear head-mounted displays and share VR images. (B) Intraoperative photographs. (C) VR is superimposed on biological samples. VR, virtual reality.

We easily separated the lesion from the mucosal periosteal flap. The left maxillary teeth from the lateral incisor to the second premolar were included in the resection domain (Figure 6A,B). After resection, communication was found between the left maxillary sinus and nasal cavity (Figure 6C). Once the maxillary sinus and nasal cavity were covered with the artificial dermis, a fragment of the iliac cancellous bone was grafted into the post-resection void (Figure 6D). The lesion resected as a single piece together with the teeth and its size was 30 mm × 30 mm. The surgery duration was 109 min and the hemorrhage volume was 118 g. The time needed for the preparation of the Microsoft^® HoloLens before surgery was approximately 10 min.

Figure 6.

(A-D) Surgery is performed without separating the operating and visual fields, with 3D ascertainment of the communication region between the maxillary sinus and nasal cavity by MR. After resection, communication was found between the left maxillary sinus and nasal cavity. 3D, three-dimensional; MR, mixed reality.

The resected tissue had a cystic structure and was composed of a fibrous connective tissue inside the jawbone, with tumor epithelium covering the inner surface. The tumor epithelium consisted of spindle-shaped tumor cells resembling odontogenic epithelium and had an abundant quantity of dispersed calcified material resembling ghost cells. The main cystic structure body consisted of a calcified odontogenic cyst, but a part of it was morphologically similar to an adenomatous odontogenic tumor, as observed in the biopsy (Figure 7).

Figure 7.

The tumor is composed of spindle-shaped tumor cells resembling the odontogenic epithelium and contains large quantities of dispersed calcified matter resembling ghost cells. The main body of the cystic structure is a calcified odontogenic cyst, but a part of it was morphologically similar to an adenomatous odontogenic tumor. HE; scale bar =50 μm.

At present, 18 months after surgery, the swelling of the left upper lip and buccal mucosa has disappeared, the alveolar morphology planned during virtual operation has been achieved, and there has been no recurrence (Figure 8).

Figure 8.

Postoperative (A, face; B, oral) photographs and (C) panoramic image.

Discussion

In the field of oral and maxillofacial surgery, many institutions have recently begun using 3D printers to create 3D models of several diseases. The first medical fabrication laboratory in Japan was established at Tokyo Dental College—the “Fab Lab TDC”—in December 2013 (11). Techniques to construct full-scale 3D models, including those of the jaw, based on CT and MRI modalities have been reported recently (11-13). Our team has created preoperative 3D-printed models for tumors in the maxilla and mandible and jaw deformities and used them primarily for patient education and preoperative simulations. Most of the commercially available systems require the operator to possess a certain level of expertise; however, it is difficult for an inexperienced operator to add the anatomical structure image. Indeed, younger surgeons have a deeper understanding, operate the technology easily, and utilize the 3D model simulation for the surgery safely and without difficulty. The use of full-scale 3D models facilitates the performance of shorter in time and more precise operations, as a result of the realistic pre-operative simulation, enables a better grasp of the bone movement extent, and illustrates ideal occlusion. Furthermore, 3D models would also enable the construction of temporary crowns and the fabrication of operation plates before surgery.

Visualization of anatomic structures adjacent to the tumor, using MR surgical support, and performing the surgery with these superimposed models in the operative field enabled its safe completion. Moreover, VR sharing by personnel wearing the HoloLens enabled preoperative discussion on a 3D basis and allowed the surgeons to perform surgery with a shared perception. Furthermore, improving the visualization of anatomic structures provides experience, which enhances surgical safety (14,15). The combination of a 3D model and HoloLens was previously found to be effective for patient education and counseling. However, in our case, it enabled us to perform surgery without separating the operative and visual fields, while maintaining a 3D understanding of the communication region between the maxillary sinus and nasal cavity. Moreover, the procedure was safer and completed in less time.

Calcified odontogenic tumors account for 2–7% of odontogenic tumors. In histological terms, eosinophilic, anucleate ghost cells are found within the epithelium, and although the epithelial morphology could be presented in various forms, the tumors resemble enamel epithelioma. The tumors contain calcified material and occur frequently in the anterior tooth region. They are especially frequent in people aged 20–40 years. Surgeries should be performed to treat such cases, involving fenestration, tumor excision, and/or bone resection (16-18). As there have been reports of recurrence and deterioration when only fenestration and/or tumor excision were performed, we performed bone resection in our patient. Today, 18 months after surgery, the postoperative progression has been favorable.

A limitation of the method used in this patient was that VR superimposition on the patient was performed manually. A new technique is currently being developed, in which specific makers are recognized by HoloLens, and, then, used to automatically superimpose the VR on the patient. Martinez et al. reported the use of a 3D printer to create a single unit that combines a surgical guide and a tracking marker for sarcoma resection, registering the patients and VR, and achieving higher accuracy with an error of only 2–3 mm (15). Verification of the precision of insertion of pedicle screws guided by the HoloLens presented an error of 2.5±0.44 mm; however, it is expected that navigation of operations will be achieved in future (14).

In our work, we used markers for superimposition. The operating field and the application could be automatically aligned by fabricating the markers used for alignment with the 3D printer, and the HoloLens recognized the markers attached to the splint in our application. The surgeon could confirm the marker, and the system was designed with two surfaces (surface size, 10 cm × 10 cm) that did not interfere with the operating field. We hope that the superimposition method will be established in the future, and precision will be increased.

Pratt P et al. reported that head-mount displays with installed MR techniques can confirm the location of the vascular pedunculated flaps, thus, minimizing the risk of hemorrhage from small veins (10). Sato et al. reported that holographic image-guided thoracoscopic esophagectomy provided better spatial recognition of vascular variation, safer lymph node dissection (19), and enabled a safe Glissonean pedicle approach for cases of hepato-cellular carcinoma with a hilar anatomical anomaly (20). Philip et al. reported that HoloLens enabled to perform surgery safely and reduced postoperative complications and, therefore, it can be used for surgical training and remote surgical support. Additionally, it is also effective for answering patient queries. In our case, performing surgery without separation of the operative and visual fields, while performing the 3D determination of the communication region between the maxillary sinus and nasal cavity by MR resulted in operating time reduction and safer completion of the procedure. However, this method suggested that intraoperative MR techniques contributed to last-minute simulation not to real navigation.

Authors have utilized MR surgical support in various types of oral surgery. Viewing salivary calculi and blood vessels in VR when resecting submandibular gland sialolithiasis, and using HoloLens to ascertain intraoperative progression, could result in improved safety and provision of surgical support, education, and training in inexperienced surgery resident. Technology could be also useful for blood vessel imaging during orthognathic surgery and intraoperative verification by exporting superimposing virtual-operation data at the time of jawbone movement. These techniques are expected to have various applications in oral surgery in the near future.

In conclusion, incorporation of HoloLens in the preoperative planning and intraoperative visualization of the oral and maxillofacial regions resulted in a precise and safe surgery performance. This MR surgical guide could have an immense potential in performing safe and effective surgical procedures if extended to other conditions, which we intend to accomplish in the near future.

Ethical Statement: Written informed consent was obtained from the patient for publication of this study and any accompanying images.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.