Augmented Reality Technology for the Positioning of the Auricle in the Treatment of Microtia

Supplemental Digital Content is available in the text.

Abstract

Background:

The positioning of the auricle is a key factor in successful ear reconstruction. However, the position of the ear is usually determined by transferring the auricle image of the nonaffected side to the affected side using a transparent film. Augmented reality (AR) is becoming useful in the surgical field allowing computer-generated images to be superimposed on patients. In this report, we would like to introduce an application of AR technology in ear reconstruction.

Methods:

AR technology was used to determine the position of the reconstructed ear of a 10-year-old male with right microtia. Preoperative 3-dimensional photographs of the nonaffected side were taken using VECTRAH1. Then, the image was horizontally inverted and superimposed on the three-dimensional image of the affected side with reference to the anatomical landmarks of the patient’s face. These images were projected onto the patient in the operation room using Microsoft’s HoloLens. The design and positioning of the auricle was done with reference to the AR image. To confirm the accuracy of the AR technique, we compared it to the original transparent film technique. After the insertion of the cartilage framework into the skin pocket, the position and shape of the reconstructed ear was confirmed using the AR technology.

Results:

The positioning of the reconstructed ear was successfully performed. The deviation between the 2 designated positions using the AR and the transparent film was within 2 mm.

Conclusion:

The AR technology is a promising option in the surgical treatment of microtia.

INTRODUCTION

In auricular reconstruction, placing of the reconstructed auricle in the proper position will greatly affect the result of the procedure. Although anthropometric analysis has revealed detailed data, ie, the shape, size, and angle of the human ear,^1,2 it is still challenging to use these data in surgery. Due to previous works of several surgeons, some parts of auricular reconstruction have become routinary.^3,4 However, deciding the auricular position is still performed manually, eg, transferring the nonaffected side image using transparent film. Recently, augmented reality (AR) technologies, which make it possible to overlay computer-generated images onto patients’ bodies, have become an operative auxiliary tool.^5–7 In this report, we describe an application of the AR technology in auricular reconstruction.

PATIENT AND METHODS

The clinical application of the AR technology is approved by the Ethical Committee of Osaka Medical College (No. 2316).

We used an AR device to position the auricle of a 10-year-old male with right congenital small concha type microtia. A day before the surgery, 3-dimensional (3D) photographs were taken, while the patient was in a supine position. Before taking the photographs, 3 dots were drawn around the affected right ear. Then, 3D photographs of the right and left face including the auricles were taken using VECTRAH1 (Canfield Scientific, Parsippany, N.J.). The 3D images were exported to Blender, a free- and open-source 3D software. Then, the 3D image of the nonaffected side of the face was inverted to the right and left. This image was superimposed to the image of the affected side with reference to the anatomical landmarks on the face, eg, ala of nose, lateral canthus, eyebrow, angle of mandibular, and facial contour. Finally, the image of the facial outline of the affected side with the 3 dots was created using the software. As a result, the following images were obtained: 1) auricular image of the affected side, 2) reversed left and right nonaffected auricular image aligned on the affected side, and 3) image of the outline of the face of the affected side with the 3 dots (see Figure, Supplemental Digital Content 1, which displays a 3D photograph of the face of the affected side with the 3 dots indicated by a, b, and c (left) and a 3D photograph of the nonaffected side which was inverted to right and left was superimposed with reference to the anatomical landmarks on the face, eg, ala of nose, lateral canthus, eyebrow, angle of mandibula, and facial contour (middle); image of the outline of the face of the affected side with the 3 dots was prepared (right); http://links.lww.com/PRSGO/B307.). These images were exposed to HoloLens (Microsoft Corp., Redmond, Wash.), a head-mounted mixed reality device.

The operation was performed under general anesthesia with the patient in a supine position. The prepared 3D images were projected onto the surgical field using HoloLens (Fig. 1). By aligning the 3 dots on the 3D image of the facial outline to the 3 dots on the patient’s face, it can be superimposed on the surgical site⁵ (Fig. 2). Then, the 3D image of the affected side was displayed and the position was confirmed and finely adjusted. Finally, the reversed 3D image of the right and left nonaffected sides was displayed and the outline of the opposite auricle was transferred (Fig. 3) (see Video, [online], which displays the surgeon projecting the image of nonaffected ear on the operative field and drawing the position of reconstructed ear via HoloLens). Positioning of the auricle was also done using a transparent film. The outline of the normal auricle and facial landmarks, eg, ala of nose, lateral canthus, eyebrow, and sideburns, were marked on the film then it was flipped and the position was transferred to the affected side.

Fig. 1.

The prepared 3D images were projected onto the surgical field using HoloLens. The surgeon can confirm the position of the auricle through the device.

Fig. 2.

The surgical site seen through HoloLens. Constructed 3D images were superimposed guided by the 3 dots (white arrows) on the 3D image of the facial outline (yellow arrows) to the 3 dots on the patient’s face.

Fig. 3.

The surgical site seen through HoloLens. The right and left reversed 3D image of the nonaffected auricle was displayed and the outline of the opposite auricle was transferred.

After the skin incision, the remnant cartilage above the concha was removed. The three-dimensional costal cartilage framework of the upper two-thirds of the auricle was placed into the skin pocket. Then, the frame was draped with skin flap and suction was applied to simulate the draping of the skin. The shape and position of the reconstructed auricle was compared to the normal auricle by using HoloLens.

RESULTS

The positioning of the reconstructed auricle was successfully performed. Real-time comparison of the reconstructed auricle with the normal was made possible using the AR device. The 2 designated positions using the AR device and the transparent film were almost identical in the longitudinal direction. On the other hand, the auricular design using the AR device was 1–2 mm shorter in the lateral direction. The time it took to align the 3D images and draw the design was less than 10 minutes.

DISCUSSION

In auricular reconstruction, the AR technology can effectively support the positioning and confirm the shape of the reconstructed auricle after the costal cartilage framework was transferred.

To determine the position of the auricle, level, axis, and distance from the orbit should be considered.^1,2 In our technique, the image of the nonaffected side and the affected side can be superimposed finely using computer software with reference to not only the anatomical landmarks on the face but also the facial contour. Historically, transparent film has been used to transfer the outline of the nonaffected auricle to the affected side.^8,9 To execute the design correctly, it is necessary to set the film in the appropriate position. Patients often have low hairline, and available anatomical landmarks are limited to the lateral canthus, eyebrow, and ala of the nose. These landmarks are away from the auricle; therefore, slight shifts in alignment can greatly affect the position of the auricle. Furthermore, these right and left landmarks may not be exactly the same. In hemifacial microsomia patients, it is challenging to apply the transparent film technique. Therefore, an AR device can help project a detailed simulated image onto the surgical site with less error. In this case, the auricular design was 1–2 mm shorter than the design using transparent film in the anterior–posterior axis. It may be because the 2 mm error must be due to the image of the standing ear which did not perfectly reflect the image of the auricle after the first stage of operation. There are also reports that some error may occur in the alignment method.^7,10 At this early stage, these kinds of errors can be easily optimized with improvements in the system and the device. The AR technique is costly and it time-consuming to prepare the images. However, we hypothesized that an AR device has a potential to reduce the time taken for the positioning of the reconstructed auricle in the operation room and can help improve the accuracy.

CONCLUSION

An AR device can effectively overlay computer-generated images onto the surgical site. This technology can be a promising tool in auricular reconstruction.

Video 1. Video showing the surgeon projecting the image of nonaffected ear on the operative field and drawing the position of reconstructed ear via HoloLens.

Download video file ^{(20.8MB, mp4)}