3D superimposition of craniofacial imaging—The utility of multicentre collaborations

Structured Abstract

Clinical applications of 3D image registration and superimposition have contributed to better understanding growth changes and clinical outcomes. The use of 3D dental and craniofacial imaging in dentistry requires validate image analysis methods for improved diagnosis, treatment planning, navigation and assessment of treatment response. Volumetric 3D images, such as cone-beam computed tomography, can now be superimposed by voxels, surfaces or landmarks. Regardless of the image modality or the software tools, the concepts of regions or points of reference affect all quantitative of qualitative assessments. This study reviews current state of the art in 3D image analysis including 3D superimpositions relative to the cranial base and different regional superimpositions, the development of open source and commercial tools for 3D analysis, how this technology has increased clinical research collaborations from centres all around the globe, some insight on how to incorporate artificial intelligence for big data analysis and progress towards personalized orthodontics.

1 |. INTRODUCTION

Development of a method to obtain standardized cephalometric head films in 1931 by Broadbent¹ changed the course of dentistry for the first time. This led to the development of innumerous cephalometric image analyses, being the first one by Downs in 1948, with the attempt to measure the face. At first, the goal was to determine an average value of a perfect face and use it as a reference for all treatment plans. Steiner, in 1950s, suggested measurements to diagnose and treatment plan each patient individually by predicting growth changes, as well as treatment outcomes.² Even though cephalometric values are still used as a normality reference in orthodontics, it is known that variability can occur from each patient, for many different reasons. Innumerous studies have shown that cephalometric numbers differ according to race, genetics backgrounds or orofacial anomalies.^3–5

The use of cone-beam computed tomography (CBCT) in dentistry has the advantage of having a similar imaging quality as spiral multi-slice CT, with a significantly lower radiation dose, and being able to focus with higher resolution only into a specific anatomic area instead of having the exam of the full face.^6,7 It overcomes some inherent flaws of 2D images, such as patient’s head position, superimposition of different anatomical structures, image magnification and distortions.^8–11 The broad applicability of the CBCT scans includes assessing skeletal asymmetries, skeletal treatment outcomes, surgical planning, and regional anatomical remodelling. It enables volumetric measurements and allows a detailed assessment of the maxillofacial structures in variable thickness of the axial, coronal and sagittal slices, providing real measurements with no magnification.^11,12

Even though CBCT images add information to diagnosis and treatment planning, caution should be taken not to expose the patient to excessive radiation. ALARA, the acronym for “As Low As Reasonably Achievable,” is a fundamental principle for diagnostic radiology. Dose minimization can be achieved by following appropriate radiograph selection criteria after taking a history from the patient, then clinical evaluation by an appropriate healthcare professional.¹³ Over the past 10 years, studies have reported advantages/disadvantages of the CBCT in relation to regular radiographs and concluded that CBCT should be indicated on specific situations where radiographs only would not be enough to diagnose cleft lip and palate, craniofacial anomalies and impacted canines.¹⁴ The overall purpose of this critical review was to explore the literature on three-dimensional analysis.

1.1 |. Three-dimensional superimposition

The different methods to superimpose 3D images are voxel-based, landmark-based and surface-based registration. A good superimposition method should be able to register precisely and aid understanding of the changes as a result of growth and/or treatment relative to the structure of reference. According to previous studies, landmark superimposition (Figure 1) is similar to 2D cephalometric but more complex, since identifying the landmarks in 3D images can be challenging.^15–17 Surface-based superimposition (Figure 2) uses only the “surface” of the 3D structure and requires high-quality surface for an accurate superimposition.¹⁸ voxel-based registration (Figure 3) matches the greyscale values of approximately 300 000 voxels when, for example, the cranial base is used as the structure of reference. It is a completely automated superimposition to avoid observer-dependent technique.^{15,16,19–21} Studies comparing the three methods suggested that landmark superimposition is the least reliable. Surface registration and voxel-based registration were validated, being the voxel-based registration associated with less variability.^17,22,23 It was also suggested that initial approximation of the images is an important step to reduce registration time of the programme and improve the precision of the superimposition.¹⁸

FIGURE 1.

Example of superimposition of surface models based on landmarks placed in teeth cusps

FIGURE 2.

Example of surface superimposition of surface models. Computer tries to superimpose the 2 surface models based on its meshes

FIGURE 3.

Example of a voxel-based superimposition at the cranial base. A, Cone-beam computed tomographys overlaid before and (B) after superimposing at the cranial base

For 2D superimposition, the American Board of Orthodontics suggests using the anterior cranial base to assess the total superimposition; the anterior surface of zygomatic proves for maxillary superimposition; and the inner contour of the chin and the alveolar canal for mandibular superimposition.¹⁷ The translation from 2D to 3D superimposition can be challenging. Studies have attempted to determine the best anatomical structures for 3D superimposition, considering that anatomical structures reported to be stable on lateral cephalograms may not be reliable, because 3D superimposition also includes transverse dimensional changes.²⁴

Cranial base (CB) superimposition (Figure 3) – CB superimposition has been suggested as a reference to assess overall facial growth and treatment outcomes²⁵ considering that the CB grows fast in early postnatal life, reaching 90% of its final size by 4–5 years of age.²⁶ Even though four different methods of CB superimposition have been suggested, studies have concluded that the best fit of the anterior CB was the best superimposition method.^27–29 They highlighted that interpretation of the overall facial changes, especially in growing patients, should be made in reference to the CB superimpositions, that means, such superimpositions yield information of facial displacements relative to the CB.

Maxillary superimposition (Figure 4): Superimposition of stable maxillary structures can be used to assess growth and treatment changes within the maxillary dentoalveolar complex. Different regions have been suggested as stable structures, for example, palatal plane, internal palatal structures or different regions of the maxilla, being the gold standard the “implant method” superimposition described by Bjork in 1964.³⁰ Ruellas et al³¹ tested 2 different areas of the maxilla to be used as reference and concluded that they showed similar results.

FIGURE 4.

Example of maxillary regional superimposition. A, cone-beam computed tomographys overlaid before and (B) after regional superimposition

Mandibular superimposition (Figure 5) – It has been a consensus that the most stable structure in the mandible is the symphysis. Two-dimensional analysis had also suggested that the contour of the mandibular canal and the lower contour of the developing third molar germ before root development could also be stable,³² but recent 3D studies showed that mandibular canal actually relocates laterally during growth,³³ and therefore, it would not be a stable anatomic reference for 3D superimposition. The stables regions suggested in the literature is the Mandibular body without teeth, alveolar bone and rami,³⁴ and the chin and symphysis.³⁵

FIGURE 5.

Example of mandibular regional superimposition. A, cone-beam computed tomographys overlaid before and (B) after regional superimposition

1.2 |. Software development

A number of software for image analysis have tools to superimpose different time-point scans. Open-source software have been developed and improved by computer scientists in order to fulfil clinical and researcher’s need. Most commonly used in combination, ITK-Snap and 3DSlicer, allow users to segment volumetric models, approximate and register CBCT scans or surface models, and allow 3D analysis and quantification. Examples of commercial software are as follows: Dolphin Imaging (Dolphin Imaging and Management Solutions, Chatswoth, CA, USA) uses a semiautomatic landmark-based registration process that also allows users to manually adjust the position of CBCT images and visualize the changes between 2 images. InVivo Dental (Anatomage, San Jose, CA, USA) and Maxilim Software (Medicim NV, Mechelen, Belgium) perform a voxel-based registration. The 3dMDvultus (3dMD, Atlanta, Georgia, USA) software uses a surface-based registration process where users manually select the stable anatomic regions to be register when superimposing images taken at different time points. OnDemand3D (Cybermed, Seoul, Korea) software performs a fast superimposition (10–15 seconds), has a user-friendly interface, does not require extensive training and does not require previous segmentation.^36,37 So far, our group has worked closely with 3D Slicer Software on developing modules that allow voxel-based superimposition of CBCT scans of growing and non-growing patients, landmark-based superimposition and surface-based superimposition (both with surface landmarks or regions of interest), determine linear and angular measurements in the 3D space and its directional components (antero-posterior, infero-superior and medial-lateral) and create surface models with parametric and non-parametric meshes.

In addition to superimposing CBCT images, one of the goals of image analysis software development is to limit the human errors in image analysis, in an attempt to automate the processes of segmentation and land-mark identification, for example. Automatic landmark detection can be performed with different approaches: training-based, model-based and knowledge-based.³⁸ However, biological variations can make it difficult to detect the precise location of landmark with training or model-based approach, and those authors concluded that even though automated landmark identification would eliminate human error and reduce time and effort involved in manual plotting, until there is a reliable technology, the gold standard will still be human eye of an expert.³⁸

1.3 |. 3D cephalometrics

Quantification is an integral and important part of orthodontic diagnosis and planning. Two-dimensional cephalometric analysis is a reliable tool and has been used for over a century. Even with basic flaws, it is still considered a gold standard in image analysis. Translation from 2D to 3D perspective is still a challenge. For start, CBCT images can be assessed in different ways: reformatted conventional radiographs, 2D measurements out of different visualizations (axial, coronal, sagittal or parasagittal slices), and more recently, 3D linear measurements that can be decomposed in the following 3 planes of the space: antero-posterior, infero-superior and medial-lateral; or angular measurements, which are decomposed in pitch, roll and yaw (Figure 6). There are also different ways to place landmarks on the 3D image: on multiplanar views with or without cross-verification on the volume-rendered image, or on surface models,^39–45 or even measure angles by moving preexisting planes.

FIGURE 6.

Illustration of (A) roll, (B) pitch, (C) yaw and (D) schematic view of the rotations

3D linear differences between two superimposed time points (T1 and T2) can be measured in each plane of the space. The 3D measurements correspond to the Euclidean distances between the T1 and T2 landmarks.⁴⁶ Such measurements can be based on thousands of points in triangular meshes automatically defined in the surface models, and calculated in different ways:

1. Colour-coded maps:

Closest point 3D linear distances measure the distances between the closest vertices of the triangular meshes in two surfaces, and not corresponding distances between anatomical points on two or more longitudinally models. The computation of the surface distances can be stored as colour-coded 3D linear distances within.obj,.ply, or.vtk file formats in the user’s software of choice, such as Paraview (http://www.paraview.org; Accessed Aug 03, 2018) or within 3DSlicerCMF https://www.youtube.com/user/DCBIA: Accessed Aug 03, 2018). Even though this standard analysis currently used by most commercial and academic software does not map corresponding surfaces based in anatomical geometry, if used in conjunction with semi-transparent overlays, the graphic display of the colour-coded maps can help clinicians and researchers understand the complex overall surface changes. Localized measurements in closest point colour-coded maps can be made at one point or at any radius defined around the landmark.

2. Shape correspondence:

The SPHARM-PDM module⁴⁷ in SlicerCMF computes point-based surface models, where all models have the same number of triangular meshes and vertices in corresponding (homologous) locations. Corresponding surface distances and vectors can then be calculated and graphically displayed in SlicerCMF. Localized measurements in closest point colour-coded maps can be made at one point or at any radius defined around the landmark.

3. Quantification of directional changes in each plane of the 3D space:

The components in each plane of the space can be measured based on observer defined landmarks or automatically defined points. The distances between landmarks can be quantified in the transversal (x axis), antero-posterior (y axis) and vertical (z axis) directions using SlicerCMF.

4. 3D Angular Measurements:

3D angular measurements between lines or planes defined in a common 3D coordinate system can be used to quantify pitch, roll and yaw of the model. Evaluations using each of the three views (sagittal, coronal or axial) allow these angles to be measured from different time points. Such angular measurements can be accomplished in SlicerCMF for either characterization of facial morphology at any time point or for comparison of rotational changes between time points. Positive and negative values can be used to indicate rotations in different directions, such clockwise or counterclockwise rotation. The choice of which landmarks or planes should be selected depends on which kind of evaluation researchers would perform to answer their aims. For example, mandibular rotations between two time points can be assessed in the following ways: pitch can be measured as the angle obtained in the sagittal view; roll is measured in the coronal view; and yaw is measured in the axial view.

2 |. APPLICATIONS OF OPEN-SOURCE SOFT WARE TO COLL ABOR ATIVE CLINICAL RESEARCH

No doubt that 3D technology marked the development of Dentistry, and while some centres have the expertise to further develop tools that can be applied to clinical and research purposes, other centres have the eager to learn and be up to date. This exchange of knowledge leads to multicentres collaboration that can only benefit scientific advances from all parts involved. Dissemination of open-source software not only helps clinicians and researchers access to image analysis tools, but also brings feedback to the biomedical computer scientists. The Dental and Craniofacial Bionetwork for Image Analysis (DCBIA-https://sites.google.com/a/umich.edu/dentistry-image-computing/) group is composed by researchers, clinicians, computer scientists and engineers with the aims to (a) develop image analysis tools specifically to answer dentistry-related clinical questions; (b) train researchers interested in those tools. These efforts resulted in international collaborations with South America, Europe, Asia and Oceania, not to mention other centres within the United States, with a large variety of application of the tools as follows: craniofacial anomalies, asymmetry, surgical outcomes and growth changes.^12,48–56 Strengthening the relationship with other centres may eventually facilitate the development of big data that require secure but easily accessible de-identified databases, with storage, mining and analytics capabilities.

3 |. CONCLUSIONS AND FUTURE DIRECTIONS

Artificial intelligence, big data analysis and personalized orthodontics—Artificial Intelligence (AI) is the next step on diagnostic technology. A newly developed neural network-based classification of temporomandibular joint osteoarthritis (TMJ OA) was based on 3D surface meshes of over 250 condyles with different degrees of TMJ OA, as the training dataset for a deep learning algorithm. Ideally, once a new patient condyle surface model is uploaded to the programme, the Slicer ShapeVariation Analyzer (SVA) module can classify the severity of arthritic degeneration of that condyle within five groups of morphological variability.⁵⁷ The technology behind such classification is a machine learning technique (deep learning). Up to now, we have trained and optimized nine different supervised machine learning algorithms to compare to our current deep neural network: Nearest Neighbors, Linear Support Vector Machine, Radial Basis Function Kernel Support Vector Machine, Gaussian Process, Decision Tree, Random Forest, AdaBoost, Naive Bayes and Quadratic Discriminant Analysis. We chose a deep neural network for classification of 3D meshes shape features, because it’s more modulable, innovative and robust to allow addition of larger data-sets as this work progresses with future collaboration with other clinical centres. With continued enrolment of subjects, neural networks have shown a better performance to build a predictive tool to assess disease progression and explain variability over time. The architecture of the current neural network was trained with high-dimensional imaging data of geometric shape features from 1002 3D meshes vertices of the condyles to classify a given condylar morphology. However, even though patterns of condylar degeneration can be quantified, the bone remodelling pattern obtained from SVA is not as a strong predictor of treatment outcomes neither pinpoint new treatment strategies by itself.

Under this perspective, it is expected that computers using machine learning may, in the future, automatically classify different malocclusions, skeletal discrepancies or dismorphologies of anatomical structures, and even perform automated image cephalometrics. One of the most important aspects on the development of the neural network-based classification is the sample size. Considering the biological variability, the larger the sample, the more accurate the neural network algorithms can become. In order to overcome that, national and international collaboration between centres/researchers is important.

Challenges for such diagnostic advances utilizing big data and artificial intelligence include the need for robust secure web-based systems with storage, mining and analytics capabilities (Figure 7). Additionally, standardization and integration of diagnostic records from clinical information, biological samples and imaging protocols and modalities (such as digital scans, photographs, X-rays, CBCT, CT, MRI and/or ultrasound) are crucial to derive more precise diagnosis. Once biological variability and genetic diverse background are determined or identified by computers, it can lead to a more personalized orthodontics, instead of “one size fits all type” of orthodontics.

FIGURE 7.

Schematic representation of a Data Storage for Computational Integration that is capable to store, mine and analyse data online