Volume estimation of tonsil phantoms using an oral camera with 3D imaging

Abstract

Three-dimensional (3D) visualization of oral cavity and oropharyngeal anatomy may play an important role in the evaluation for obstructive sleep apnea (OSA). Although computed tomography (CT) and magnetic resonance (MRI) imaging are capable of providing 3D anatomical descriptions, this type of technology is not readily available in a clinic setting. Current imaging of the oropharynx is performed using a light source and tongue depressors. For better assessment of the inferior pole of the tonsils and tongue base flexible laryngoscopes are required which only provide a two dimensional (2D) rendering. As a result, clinical diagnosis is generally subjective in tonsillar hypertrophy where current physical examination has limitations. In this report, we designed a hand held portable oral camera with 3D imaging capability to reconstruct the anatomy of the oropharynx in tonsillar hypertrophy where the tonsils get enlarged and can lead to increased airway resistance. We were able to precisely reconstruct the 3D shape of the tonsils and from that estimate airway obstruction percentage and volume of the tonsils in 3D printed realistic models. Our results correlate well with Brodsky’s classification of tonsillar hypertrophy as well as intraoperative volume estimations.

1. Introduction

Optical 3D imaging has been a very important tool for accurate shape detection for manufacturing, quality control, modeling and visualization [1]. Although it has the advantages of being non-contact, fast and accurate, industrial 3D imaging has been complex and expensive to implement. It is only recently that improvements in sensors, projectors and light sources have led to portable devices that have been utilized for a variety of applications. Currently, portable 3D imaging devices are used extensively in gesture recognition and gaming. An emerging area in the space of portable 3D imaging is visualizing and modeling the human body [2, 3]. This has several applications in biomedical imaging, diagnostics and training. For instance, 3D imaging has been used to imaging the ear [4, 5], in endoscopy [6] and dental imaging [2]. Reports have shown that it can improve diagnostic accuracy by minimizing subjectivity in many assessments.

An interesting application where 3D imaging can be very useful is in imaging the oral cavity and oropharynx. Typically, examination of this area is carried out using a tongue depressor and a light source. However, to adequately inspect the inferior tonsillar pole and its relation to the tongue base and epiglottis a flexible laryngoscope is required. Conventional laryngoscopes only offer 2D visualization, which can often be limiting in many cases. The oral cavity and oropharynx has a complex anatomy and capturing the 3D structure of the entire oropharynx can provide important information that may assist decision making and surgical planning. There have been reports of 3D imaging of the oropharynx using CT, laser scanning [7], optical coherence tomography [8, 9] and MRI [10]. CT imaging is a reliable method to carry out 3D imaging of the oral cavity and oropharynx but has the significant downside of ionizing radiation exposure, which can be limiting in the pediatric population. MRI on the other hand, can provide high resolution information without the risk of exposure to ionizing radiation [10]. A recent report demonstrated OCT for ‘in office’ imaging but the dynamic range of OCT is small and can only be used for very fine measurements [8]. Most of these reports have imaged the vocal cords, and other laryngeal structures but this work has not been extrapolated to evaluate the oropharynx and in specific, the tonsillar imaging.

Palatine tonsils are lymphoid tissue present in the posterior aspect of the oropharynx and play an important role in defending infection. They can get infected and lead to conditions like tonsillitis and peritonsillar abscesses. Most commonly, they can grow and occupy significant space in the oropharynx leading to tonsillar hypertrophy, which is the enlargement of tonsils. This is a common condition that occurs in children and can lead to snoring and obstructive sleep disorders [11–15]. It is very important to assess the size and extension of the tonsils, to determine the proper treatment. Currently, the assessment is subjective even though there are grading guidelines available such as the Brodsky’s classification [16]. This grading scale classifies tonsil enlargements into four categories depending upon the size and obstruction percentage on a horizontal axis. Conventionally, the four categories are assessed without any measuring tools [17]. A recent report demonstrated the use of a digital camera to carry out pediatric hypertrophy grading [11]. However, 2D size alone is not adequate and establishing the volume of the tonsils can provide valuable information, which may enhance current examination techniques and correlate better with polysomnography (PSG) [18]. 3D imaging of the tonsils and oropharynx can provide an accurate assessment of the anatomy and volume calculation, thereby helping physicians to make more accurate diagnosis and surgical decision making.

In this report, we present for the first time, a method for potential in vivo 3D imaging of tonsils using a novel oral camera arrangement based on structured illumination method. This allows for hand held 3D imaging, which can be easily utilized in a clinical setting and can rapidly provide crucial quantitative information that may be complementary to current diagnostic techniques and may assist in surgical planning. We have been able to accurately estimate volume of tonsils and the airway obstruction percentage for the different grades of tonsillar hypertrophy on realistic 3D printed models.

2. Materials and methods

2.1 Optical design

Programmable structured light projection was carried out using a portable, wireless DLP LED projector (Altec Lansing). The projector was coupled to a modified otoscope head and the light was directed to the throat region using a front surface mirror as shown in Fig. 1(a).

Fig. 1.

(a) Schematic of the optical arrangement of the oral camera. (b) Photograph of the prototype used for the study.

The front surface mirror ensured only a single set of patterns are projected on the scene, thereby eliminating multiple reflections caused by the two reflecting surfaces of conventional mirrors. Reflected light from the scene was collected using a 12 mm lens arrangement with a variable focus and aperture (f/1.4-f/8). A high definition sensor with a resolution of 1024x786 pixels was coupled to the lens arrangement to capture the set of images. The distance between the projector and the scene was set at 21.5 cm and the x-y calibration of the sensor that distance was 0.06 mm for pixel to pixel spacing. The focal length of the optical assembly was 16.5 cm and could be varied depending on the scene. This optical setup was well suited to image the tonsils providing an optimal field of view and magnification. Triangulation was achieved by adjusting the angle between the optical axis of the camera and the projection system as shown in Fig. 1(a). 3D printed components were utilized to house the projector and lens assembly as shown in Fig. 1(b).

2.2 Tonsil models and 3D printing

The 3D printing test model consisted of the uvula, which is a fleshy extension that is present on the roof of the mouth, flanked by two tonsils on either side as shown in Fig. 2(a). In order to test the imaging device on the tonsil phantoms we used two methods for developing the tonsillar models: direct 3D printing and casting into 3D printed molds. Idealized computer aided design (CAD) models of tonsils featuring four different sizes representing the Brodsky classification were obtained from Netter’s atlas and re-created using SolidWorks (DassaultSystemes) as shown in Fig. 2(a)-2(d). 3D printing was carried out using fused deposition (FDM) 3D printers (MakerBot Replicator 2, 2X and Ultimaker 2) as shown in Fig. 2(e)-2(h). A viscous silicone rubber (Dragon Skin) was poured into the assembled mold directly and cured for up to 40 minutes. The cured silicone was then released from the mold, which resulted in the completed tonsillar model.

Fig. 2.

CAD and 3D printed models of the four stages of tonsils in accordance with the Brodsky classification. (a) Grade 1 + , (b) Grade 2 + , (c) Grade 3 + and (d) Grade 4 + . Corresponding 3D printed models for the respective grades (e) Grade 1 + , (f) Grade 2 + , (g) Grade 3 + and (4) Grade 4 + . (i-j). Photographs of the measurement setup with the mouth model and the optical arrangement.

The four grades of Brodsky’s classification are denoted by 1 + , 2 + , 3 + and 4 + , respectively [16]. Stage 1 + indicates an obstruction <25%, Stage 2 + between 25 and 50%, Stage 3 + between 50 and 75% and Stage 4 + >75%.

2.3 Mouth phantom arrangement

Testing of the device was carried out using a realistic mouth model (Nissin) that was mounted on an optical post as shown in Fig. 2(i), 2(j). The entire assembly was mounted on a calibrated rotational stage in order accurately determine the angle of the tonsils with respect to the projector optical axis. Experiments were carried out at 5 different angles i.e. −10, −5, 0, + 5, + 10 degrees outside which the visibility of tonsils was reduced. The 0-degree arrangement was useful to arrive at 2 dimensional features like the inter-tonsillar distance and airway obstruction whereas larger angles allowed for volume estimation. The error in the angular measurement was ± 0.5°.

2.4 Height reconstruction-structured light and phase shifting

For 3D shape estimation, we used phase shifting structured illumination method where five phase-shifted fringe patterns were projected on to the scene and the reflected images were captured. Every frame was projected for 0.3 s making the total integration time 1.5 s for one measurement. The intensity of the captured images can be modeled as:

$$I_n\left(x,y\right)=a\left(x,y\right)+b\left(x,y\right)\cos (\varphi \left(x,y\right)+{\omega }_{x}x+{\omega }_{y}y+\frac{\pi }{2}n).$$ (1)

for $n=1,2,3,4,5$; where $a\left(x,y\right)$ is the background intensity, $b\left(x,y\right)$ the contrast, $\varphi \left(x,y\right)$ the modulated range and $({\omega }_x{\omega }_y)$ the fringe pattern frequency and the phase shift at each stage is $\pi /2$ radians [19]. Automatic segmentation was applied to the images to select the region where fringes were visible. Since the sensor captured an image area larger than the illuminated region, this step was crucial for eliminating noise from non-illuminated or regions with stray light. The images from Eq. (1) were combined as follows:

$$f\left(x,y\right)=2I_3-I_1-I_5+i[I-I_4],$$ (2)

where, $i=\sqrt{-1}$. Equation (2) can be expressed as a complex output of a 5-step quadrature filter that is tuned at $\pi /2$ radians [19]. The complex field $f(x,y)$ was used it to find the region of the visible fringes and the magnitude of Eq. (2) was taken as,

$$p\left(x,y\right)=f(x,y)$$ (3)

where, $p\left(x,y\right)$ is the response of Eq. (2) and indicates pixels in Eq. (1) that are well correlated. Thresholding of this magnitude yields pixels that have a good response and the threshold was automatically calculated using the method described in (20). This method chooses the threshold that minimizes the interclass variance of the black and white pixels. In cases where the visibility is good, this method is effective. For cases, where the visibility is not good, it leaves holes. This is a realistic situation as tissues diffuse light and the visibility is not good in many cases. These holes can be removed by the application of erode and dilate operators with a mask of $3\times 3$ pixels [19]. This procedure for this operation is presented as follows:

• 1.$m\left(x,y\right)=threshold [p\left(x,y\right)]$
• 2.For $n=1$ until 5: $m\left(x,y\right)=dilate [m\left(x,y\right)]$
• 3.For $n=1$ until 5: $m\left(x,y\right)=erode[m\left(x,y\right)]$

For our system, 5 iterations were sufficient to obtain the segmentation. A typical example of this process is shown in Fig. 3(b)-3(d). Using the binary $m\left(x,y\right)$ from the previous procedure, region of interest was segmented and the local frequencies $({\omega }_x{\omega }_y)$ were estimated. The phase of the 5-step quadrature filter combination in Eq. (2) was taken as,

$$\text{Φ}\left(x,y\right)=\angle f(x,y).$$ (4)

Using Eq. (4), the frequencies along $x$ and $y$ were estimated by the summing the product of ${\text{ΔΦ}}_{x,y}(x,y)$ and $m\left(x,y\right)$ normalized by the $L_1$ norm, where ${\text{Δ}}_{x,y}$ is the first order difference operator. Since the recovered phase was wrapped, phase difference modulo $2\pi$ was carried out. After the frequencies $({\omega }_x{\omega }_y)$ were estimated, the 3D shape (phase units) of the object under test was obtained as,

$$\varphi \left(x,y\right)=\angle \left\{e^{\text{Φ}(x,y)}{e}^{-i({\hat{\omega }}_{x}x+{\hat{\omega }}_{y}y)}\right\}$$ (5)

In order to obtain the range accurately, we applied a phase unwrapping algorithm that was demonstrated by Estrada et al. [20]. The procedure and the sequence of phase estimation from the capturing of images to phase unwrapping are summarized in Fig. 3(a)-3(d). A final step of low pass Gaussian filtering ensured that the effect of fringes was minimized on the reconstructed surface.

Fig. 3.

(a) Procedure for calculating height of an object using phase shifting fringe projection method. (b) Respective image after thresholding and segmention. The corresponding phase image (c) and reconstruced 3D map (d).

2.5 Height estimation from phase and calibration

The height $h\left(x,y\right)$ can be calibrated using the standard linear calibration procedure used in phase profilometry given by [21],

$$h\left(x,y\right)=\frac{L_0\Delta \varphi (x,y)}{\Delta \varphi \left(x,y\right)+2\pi f_{0}d}$$ (6)

where, $L_0$ is the object camera distance, d is the distance between the camera and projector, $f_0$ is the spatial frequency of the projected pattern and $\Delta \varphi (x,y)$ is the relative phase change. For our system, the measured parameters are shown in Table 1.

Table 1. Geometric parameters for height conversion.

Parameter	Value
$L_0$	165 ± 0.5 mm
$d$	10 ± 0.5 mm
$f_0$	1.3 fringe/mm

3. Results

Experiments were performed on the 3D printed models as shown in Figs. 2(e)-2(h) with varying viewing angles (0°, ± 5°, ± 10°). A viewing angle of 0° provided a complete description of the tonsils and uvula and was useful in estimating inter-tonsillar distance. Whereas, higher viewing angles were useful in estimating volume of tonsils. Viewing angles of ± 10 degrees gave the best results for volume estimation. Intermediate viewing angles did not yield any additional information.

3.1 3D printed models-inter-tonsillar distance estimation

An important parameter to classify tonsillar enlargement is the airway obstruction percentage (AOP), which is the ratio obstruction to the total lateral dimension of the throat without any obstruction. AOP can be calculated in our experiments by measuring the relative distance of the tonsils with $(D_t)$ and without obstruction $(D_o)$, and is given by $[(D_o-D_t)/D_o\times 100]$. The inter-tonsillar distance $(D_t)$ can be estimated from the inner boundary of the profiles as shown in the Figs. 4 and 5. The non-obstruction distance $(D_o)$ can be evaluated from the firstminima at the extremities of the profiles. Results on the four classes of tonsils are shown in Figs. 4 and 5. For the case of Grade 1 + , we observed a $D_t$ = 27.7 mm and $D_o$ = 38.4 mm, as shown in Fig. 4(a), 4(b). AOP for Grade 1 + case turns out to be 27.8% which is close to the upper range of 25%. $D_t$ was also measured using a Vernier calipers and the result was 31.5 mm with an error of 12%. For Grade 1 + , the enlargement is small and hence the error rate can be higher given that the inner boundary of the tonsil can be difficult to extract. This case is characterized by a mild enlargement and generally occurs early on during an infection. In the case of Grade 2 + tonsils, $D_t$ = 19.7 mm, $D_o$ = 39.3 mm and AOP = 49.8% were obtained. This falls in the range of Grade 2 + which according to the Brodsky classification is 25-50% as seen Fig. 4(c), 4(d). A $D_t$ = 19.3 mm was measured using a calipers yielding a 2% error.

Fig. 4.

(a,c) 3D reconstruction for the cases of Grade 1 + and 2 + tonsillar hypertrophy. (b,d) Profile maps of Grade 1 + and Grade 2 + tonsils corresponding to the blue marker as indicated in the (a,c).

Fig. 5.

(a,c) 3D reconstruction for the cases of Grade 3 + and 4 + tonsillar hypertrophy. (b,d) Profile maps of Grade 3 + and Grade 4 + tonsils corresponding to the blue marker as indicated in the (a,c).

For the case of Grade 3 + progression of tonsillar hypertrophy we observed a $D_t$ = 11.1 mm and $D_o$ = 38 mm, which corresponds to an AOP of 70% (Fig. 5(a), 5(b)). This accurately falls into 50-75% class of tonsil progression. An actual $D_t$ was measured to be 11.4 mm with an error of 2%. Finally, for Grade 4 + tonsils (Fig. 5(c), 5(d)) we observed $D_t$ = 6.7 mm and $D_o$ = 39.9 mm which corresponds to an AOP of 83.2%, which agree well with the Brodsky classification for Grade 4 + with an AOP>75%. Actual measurement of $D_t$ was 6.3 mm which resulted in an error of 6%. Hence, we observe a good agreement with the AOP derived from profiles of 3D measurements with the Brodsky classification.

3.2 Viewing angle 5°-10°: Tonsillar volume estimation

Experiments carried out with larger viewing angles provided a better view of the tonsils and were utilized to arrive at volumes. As can be seen from Fig. 6(a)-6(d), at a viewing angle of 10° only a single tonsil was observable. This provided an accurate estimation of its volume. We applied segmentation to extract on the tonsillar region from the surface plot and carried out two-dimensional numerical integration to estimate the volume under the surface of interest. In some cases a finer segmentation was necessary to select the region of interest. As can be observed from Fig. 6(a)-6(d) we estimated a volume of 0.5 mL/tonsil for the case of Grade 1 + or a total volume of 1 mL for two tonsils. Similarly, we obtained 0.7 mL/tonsil for Grade 2 + , 1.4 mL/tonsil for Grade 3 + and 3.9 mL/tonsil for Grade 4 + progression of tonsils. Hence, the range of total tonsillar volume obtained for the different stages of tonsillar hypertrophy was 1 ml - 7.8 mL. These results are reasonable considering the lateral dimensions of the tonsils are in the range of 10-20 mm and the height is in the range of 10 mm. Our volume estimation is comparable to intraoperative measurements from earlier studies where mean tonsil volumes were measured in the range of 5.6-6.8 mL in a study conducted in adults [13]. The phantoms in our study were chosen for pediatric tonsil sizes which could also account for the small discrepancy in the volume. Hence, combining the AOP with volume estimation provides a clear distinction of the stage of progression of tonsils and can be used to arrive at the grade.

Fig. 6.

Volume estimation of tonsils when viewed from an angle of 5-10° for (a) Grade 1 + , (b) Grade 2 + , (c) Grade 3 + and (d) Grade 4 + . Shaded regions in red indicate the region of interested selected for volume estimation.

3.3 Volume comparison with simulated and displacement based methods

A comparison study was carried out to ascertain whether the volume estimation obtained from optical measurements corroborate with actual volume measurements done using standard displacement methods. Two types of comparisons have been carried out in this study; 1) Using tonsils extracted from the model, as they would naturally be observed in vivo and 2) using tonsils completely deconstructed and isolated from the structure, resembling in vitro post-operative tonsils. In both cases tonsils were 3D printed separately and volume measurements were carried out using standard methods like liquid displacement. A comparison was also carried out using volume estimations from the software used to construct these models. A satisfactory agreement was found between optical and displacement based volume measurements as summarized in Table 2. We observe that there was close correlation between volume measurements using the 3D camera and the extracted tonsils (resembling in vivo). Completely dislodged tonsils (resembling in vitro) yielded larger volumes as they resemble a tonsil that has been operated and removed. In the case of extracted tonsils there is a possibility of incomplete imaging the entire tonsils due to obstructions leading to smaller volume estimations. The error in displacement based volume measurements was ± 0.5 mL and hence the volume estimations from the 3D camera are reasonable within experimental limits.

Table 2. Volume comparison using simulated, optical and fluid displacement methods.

Brodsky Grade	3D Camera method (mL/tonsil)	Actual volume from extracted model- like in vivo (mL/tonsil)	Actual volume from dislodged model- like in vitro (ml/tonsil)	Displacement Method-like in vivo (mL/tonsil) (3D printed)	Displacement Method- like in vitro (mL/tonsil) (3D printed)
1 +	0.5	0.3	1.2	0.6	0.8
2 +	0.7	0.9	1.5	0.8	1.0
3 +	1.4	1.6	2.2	1.7	2.0
4 +	3.9	2.5	2.9	3.0	3.5

3.4 Soft tissue-like models: inter-tonsillar distance and volume estimation

Soft models that resemble tissue in terms of mechanical and optical properties were fabricated using a combination of 3D printing and molding techniques as mentioned in Section 2.2. The soft model had a lower reflectivity as compared to the 3D printed phantoms. Additionally, the reflection was diffuse in nature which provided less defined fringes. The reflectivity issue was solved by increasing the exposure of the camera. Diffuse reflections contributed to the system noise but results from the fringe analysis were results comparable to the 3D printed phantoms as long as the fringes were visible. Important parameters like AOP and the volume could still be calculated from the 3D estimation. Experiments carried out on a representative Grade 3 + phantom yielded $D_t$ = 17.5 mm, $D_o$ = 37.8 mm and an AOP of 53.7% corresponding to Grade 3 + tonsils as shown in Fig. 7(a), 7(b). Volume measurements on soft models were estimated to be 1.4 mL/tonsil, which agree well with earlier measurements of Grade 3 + tonsils as seen in Fig. 7(c).

Fig. 7.

Measurements on soft tissue-like models. (a) 3D shape map for the case of Grade 3 + soft model. (b) Line profile of the shaded blue line in (a). (c) Volume estimation of tonsils (red shaded region) when viewed from an angle of 5-10° for Grade 3 + soft model.

We would like to note that in realistic situations, the presence of saliva or other translucent secretions may increase specular reflections locally. It was observed that this factor does not significantly alter the reconstruction characteristics. Experiments were carried out in which a soft tonsil was imaged after being rinsed with water. The 3D reconstruction was not affected by the presence of specular reflections.

4. Discussion and clinical relevance

Tonsillectomy and adenoidectomy has long been considered the standard of care for pediatric obstructive sleep apnea as well for sleep disorder breathing in patients with adenotonsillar hypertrophy [22]. Current assessment of tonsillar hypertrophy is made on a subjective basis describing tonsil size in a 2D perspective applying most commonly the Brodsky scale. This however, does not take into account the whole dimension of the tonsils or more importantly the overall volume it occupies in a three dimensional space. To this day the most common and accurate way of calculating tonsil volume is following surgical excision.

Adenotonsillar hypertrophy is one of the major factors that can lead to OSA in pediatric patients. OSA is the result of upper airway collapse during sleep [23]. Various comorbidities have been associated with OSA when untreated including cardiovascular, neurocognitive and behavioral among others [24, 25]. Tonsillectomy and adenoidectomy is widely accepted as the first line of therapy for pediatric OSA. Most studies have shown poor association between tonsil size and objective OSA severity but most of these studies have used the 0- + 4 Brodsky classification, which does not take volume or the adjacent anatomy into consideration [18]. However studies that have used tonsil volume or magnetic resonance to assess the tonsil size have correlated adequately to OSA severity [26, 27].

3D volume estimation of tonsils in an in vivo manner has several advantages over current diagnostic methods, which are highly subjective. This is especially true in pediatric patients where physical examination may involve gagging the patient and having only an instant to make the assessment. This novel application offers an objective assessment of tonsillar volume while obtaining visualization of the base of tongue and adjacent structures. Earlier reports have either measured 2D features or carried out volume measurements on surgically removed specimens, with both modalities lacking the ability to gauge the entirety of the oropharynx. It has been suggested by Nolan et al. [18] that to achieve better correlation between the tonsils clinically would be easy if the method could capture the tonsils in a 3D volumetric framework. We have shown in this study the ability to capture a three dimensional view of the tonsils and the ability to calculate volume. This measurement of lateral dimensions, AOP and volume can provide an objective assessment of the level of airway compromise caused by the tonsils. The volumetric approach can also be applied for situations of OSA where tonsillar hypertrophy is not present providing important information for potential areas of obstruction [28]. More importantly calculating tonsillar volume opens the door to partial tonsillectomies in which energy must be delivered to the tonsillar tissue without extending into the adjacent structures [29, 30]. Knowing the tonsillar volume will allow to make energy calculations without difficulty. We are currently adapting our device to evaluate the adenoids to in the future obtain a more complete upper airway evaluation. More tests will need to be carried to validate this approach in vivo and most importantly if it correlates with OSA severity as established with polysomnography.

5. Conclusion

We demonstrated a hand held, portable oral camera that can accurately estimate tonsillar size, separation and volume in life-like models. This arrangement can be realized in a relatively simplified manner and has the potential to be made clinically translatable for pediatric and adult oropharyngeal imaging.