Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Apr 7.
Published in final edited form as: Orthod Craniofac Res. 2021 May 24;24(Suppl 2):26–36. doi: 10.1111/ocr.12492

Clinical decision support systems in orthodontics: A narrative review of data science approaches

Najla Al Turkestani 1,2, Jonas Bianchi 1,3, Romain Deleat-Besson 1, Celia Le 1, Li Tengfei 4, Juan Carlos Prieto 5, Marcela Gurgel 1, Antonio C O Ruellas 1,6, Camila Massaro 1,7, Aron Aliaga Del Castillo 1,7, Karine Evangelista 1,8, Marilia Yatabe 1, Erika Benavides 9, Fabiana Soki 9, Winston Zhang 10, Kayvan Najarian 10, Jonathan Gryak 10, Martin Styner 11, Jean-Christophe Fillion-Robin 12, Beatriz Paniagua 12, Reza Soroushmehr 10, Lucia H S Cevidanes 1
PMCID: PMC8988880  NIHMSID: NIHMS1793852  PMID: 33973362

Abstract

Advancements in technology and data collection generated immense amounts of information from various sources such as health records, clinical examination, imaging, medical devices, as well as experimental and biological data. Proper management and analysis of these data via high-end computing solutions, artificial intelligence and machine learning approaches can assist in extracting meaningful information that enhances population health and well-being. Furthermore, the extracted knowledge can provide new avenues for modern healthcare delivery via clinical decision support systems. This manuscript presents a narrative review of data science approaches for clinical decision support systems in orthodontics. We describe the fundamental components of data science approaches including (a) Data collection, storage and management; (b) Data processing; (c) In-depth data analysis; and (d) Data communication. Then, we introduce a web-based data management platform, the Data Storage for Computation and Integration, for temporomandibular joint and dental clinical decision support systems.

Keywords: artificial intelligence, decision support systems, machine learning, orthodontics

1 |. INTRODUCTION

Over the past decades, the digital revolution has transformed every facet of our world, including dental practice. Adopting modern technology in orthodontics has not only provided us with advanced diagnostic and treatment tools, but also given us the chance to shift clinical practice from a ‘malocclusion-centred’ to a ‘patient-centred’ model.1 Establishing optimum and personalized orthodontic care requires (a) analysis of large and complex data sets derived from different sources, such as clinical examination, diagnostic images, biological and genetic data,2,3 and (b) identification of patterns/associations that turn the individual’s big data into knowledge for precise decisions and outcomes prediction.4 Big data analysis is associated with numerous challenges that necessitate the use of high-end computing solutions, advanced analytical methods5 (e.g. artificial intelligence and machine-learning algorithms) and data science approaches before communicating the acquired knowledge to healthcare providers via decision support systems.6

Clinical Decision Support Systems (CDSSs) are computer programs developed to provide expert support for healthcare providers in making decisions regarding prevention, diagnosis and treatment of health diseases.7 Lusted and Ledley were pioneers in explaining how the reasoning behind the foundation of diagnosis and treatment in medicine can be investigated and solved accurately using mathematical tools.8 Since then, researchers have been using different methods to supply clinical applications with knowledge. In orthodontics, CDSSs are being developed to reduce subjectivity, decrease errors, save time and increase the efficiency of diagnosis and treatment planning among clinicians.9 Examples of CDSSs include systems that aid in detecting cephalometric landmarks, determining the need for extractions, identifying the maturation stage of cervical vertebrae and predicting facial soft tissue changes following treatment.10 Over the last 5 years, our clinical research team, the Dental and Craniofacial Bionetwork for Image Analysis (DCBIA), has addressed knowledge gaps in dentistry that require CDSSs (Figure 1). In this manuscript, we present a narrative review of the data science approaches that are required to develop clinical decision support systems in orthodontics. We also describe a web-based data management platform for the temporomandibular joint and the dental clinical decision support systems.

FIGURE 1.

FIGURE 1

Open in a new tab

Gaps of knowledge in (A) TMJ and (B) Dental diagnosis and prognosis assessment tools

2 |. DEVELOPMENT OF CLINICAL DECISION SUPPORT SYSTEMS (CDSSs)

2.1 |. Types of CDSSs

Clinical decision support systems have been categorized according to (a) function of the support: provision of alerts (active) or reaction to patients’ information/providers’ inputs (passive); (b) time of support delivery: before, during or following decision-making and (c) method of development: knowledge or non-knowledge-based.

Knowledge-based CDSSs contain compiled data entered directly by users or extracted from patients’ electronic records, phone apps11 and data about medications12 or clinical protocols and guidelines based on the intent of using a clinical decision system.3,12 Such systems will deliver support to users, usually following the IF-THEN rule. For instance, in a system for detection of medication interactions, the rule will be IF drug X and IF drug Y are used, THEN alert the clinician.12 Non-knowledge-based CDSSs also require a data source; however, decisions are made with statistical pattern recognition or machine-learning (ML) approaches. Consequently, computers learn from previous experiences, discover patterns within the data and eliminate the need for expert inputs or rules. Artificial neural networks and genetic algorithms compose the main types of non-knowledge-based CDSSs.13,14

2.2 |. Data science approaches for the development of CDSSs

Clinical decision support systems development is a highly challenging and multidisciplinary task that integrates clinical knowledge with decision science to adapt clinical practice and workflow with the decision system.15 Here, we provide a simplified overview of steps involved in creating a clinical decision support system (Figure 2A).

FIGURE 2.

FIGURE 2

Open in a new tab

Overview of steps involved in developing clinical decision support systems (CDSSs). A, General spectrum of data science approaches in the CDSSs. B, Implementation of robust data management, Dental Storage for Computation and Integration, in the CDSSs

2.2.1 |. Data collection, storage and management

The advancement of information technology promoted exponential growth of health data and generation of big data15; that is, large-volume data that are produced at high speed and integrates different types of data, such as: (a) clinical data (e.g. orthodontic diagnosis and treatment progress notes, imaging, health records), (b) omics data (e.g. genomics, proteomics, metabolomics), (c) patient-generated data (e.g. wearable devices and scanners, social media) and (d) normative data sets (e.g. data collected in clinical trials or nationwide surveys).2,16,17 Current evolution of data capturing, storage and analytical methods will allow us to transform the wealth of knowledge in big data into actionable plans to overcome challenges in clinical decision, deliver personalized care and improve the population’s health.18,19

Collection of patient diagnosis and treatment progress data is considered valuable when it is done in a systematic way following interlinkable and coherent data standards that produce high-quality information.17–20 Clinical data constitutes an essential resource for medical and health research—electronic health records (EHR) are one of the major types of clinical data.21 The use of EHR is highly encouraged to improve clinicians’ compliance with documentation and enable data sharing among different members of the healthcare team.16 In 2009, the Health Information Technology for Economic and Clinical Health (HITECH) Act was created to support the adoption and meaningful use of health information technology, including a provision of financial incentives for using EHR systems.22 By 2015, the adoption of certified EHR programs by hospitals and office-based physicians reached 96% and 78%, respectively.23,24 Nevertheless, the quality of data within EHR was affected by duplication, missing information, fragmentation and inconsistent organization. The accuracy and reliability of data can influence the development and use of CDSSs, even in the presence of sophisticated advanced technologies. Thus, there is a need to standardize the terminology within the dental and orthodontic fields using well-structured forms and templates to assist in ensuring consistency of the collected data.16 There have been several governmental efforts to standardize the data within different EHR systems; however, at this time no federal or professional association program has been able to produce universally accepted high-quality data. Indeed, data governance requires policies for care providers and auxiliary staff, hands-on training, a culture of responsibility and the right tools to improve and monitor data quality.25

Multicentre data collection is commonly performed to construct clinical prediction models. Although such data structures create additional challenges for data analysis, they cover a broader population and can improve the generalizability of the artificial intelligence models.26 Digital data repositories provide web-based platforms that enable researchers from multiple institutions to access and manage their data securely. Data repositories are designed to store large amounts of data, ranging from thousands of data set repositories, supported by funding and government agencies, to small data sets, supported by a research team for a certain study.27

‘BigMouth’ is an example of an oral health data repository that contains EHR from 11 dental schools across the United States. It provides access to the demographic, dental and medical data of over 3 million patients to facilitate the advancement of research and patient care outcomes.28,29 Defining and establishing a code of conduct is important for big data collaborations to guide the ethical and meaningful use of shared data. When preparing data for sharing, it is essential to strike a balance between privacy protection (e.g. de-identification & anonymization of patients’ data, security measures, controlled data sharing) and maintenance of data utility.16,30

2.2.2 |. Data processing

After defining the proper source for data collection, it is essential to verify the quality of data and prepare a final data set for analysis or machine training.31

In 1955, a mathematician named John McCarthy coined the term artificial intelligence (AI) to describe the ability of machines to conduct tasks that lie within the range of intelligent activities. Afterwards, Richard Bellman defined AI as the ability to automate activities with human thinking capabilities; for example, problem-solving, learning and decision-making.32 Machine learning (ML) is a subfield of AI, whereby algorithms are utilized to find structures and patterns within data. Consequently, machines will learn to predict similar patterns on unseen data, and their actions will improve each time new data are introduced without human inputs.9,33,34

High-quality data sets are essential for developing effective machine learning models. To build an ML model, three non-overlapping data sets are utilized for training, validation and testing. The training data set is used to develop the ML model. Then, model parameters are adjusted with the validation data set. The last step involves testing the model performance on unseen data, that is, test data set.35 In ML, raw data is not usually suitable for learning; features/variables should be identified and extracted from the raw data via data mining. However, raw data should be cleaned and prepared for data mining; noise and errors within raw data might confuse the data mining process and thus result in faulty detection of patterns.36 The aims of cleaning and enriching data are primarily data normalization, elimination of redundant features, data standardization, removal of duplicates, resolution of inconsistent data, management of missing data and data matching across multiple sources.35,37 Images normalization is an example of a data preparation procedure required before applying statistical or ML algorithms. Putting several images in a common statistical distribution based on the size and pixel values (spatial normalization) will enable reliable detection of changes between several individuals or within the same individual at different time points.38–40 Additionally, normalization of images’ intensity should be performed when planning to develop a model that classifies a disease, to avoid biasing the results, or building a model for image synthesis, for example, segmentation and transformation. Evidence demonstrated the accuracy of the image synthesis model is greatly affected by standardizing the intensity features across the training’s input images.41,42

Data mining involves analysing large data sets to extract unknown patterns and comprehensible information from large data sets.43 Several functionalities or tasks can specify the knowledge found in data mining, such as regression, clustering and classification.36 These findings can summarize the input data or be utilized in additional analyses such as machine learning and predictive analytics. Although algorithms are used in data mining and ML, outputs from data mining help optimize decisions, for example, detecting valuable clinical information will help the practitioner make better decisions and increase the quality of care. However, training a machine with the extracted knowledge will enable predicting the diagnosis or prognosis of a new patient.9

Dimensionality reduction is performed before machine training to eliminate irrelevant and redundant data, improve learning accuracy, and enhance output comprehensibility. The main types of dimensionality reduction are feature selection and feature extraction. Feature selection involves selecting data that contains the most relevant information for solving a particular problem; a subset of the original data is maintained and used for machine training.44 For instance, Bianchi et al,45 conducted a study aiming to detect temporomandibular joint osteoarthritis (TMJ OA). Only the most robust features were selected for machine training out of the collected 52 clinical, biological and radiomic markers and 1326 interactions. Similarly, in AI systems for two-dimensional cephalometric analyses, the rates of success of landmark detection and classification of skeletal and dental problems depend on the proper selection of the diagnostic variables.46 On the other hand, feature extraction transforms the original features into a new, smaller set of more significant features.44 In the study of Bianchi et al,45 twenty three-dimensional imaging features of bone texture and bone morphometry, that quantify the initial morphological changes in the condylar trabecular bone, were extracted and used for machine training instead of analysing the whole Cone-Beam Computed Tomography (CBCT) scan grey level voxels.

After implementing and tailoring the previous steps according to the aim of the ML model, the data are considered ready for advanced analysis and machine learning.

2.2.3 |. In-depth data analysis

Following data preparation, each clinical application of artificial intelligence in orthodontics requires the selection of the proper machine-learning methodology, training the ML model and evaluation of the developed model’s performance.

The success of the ML algorithms depends on the thorough comprehension of what algorithms can provide, the limitations of algorithms, and how that will support and fit into clinical care. Hence, communication between data analysts, data scientists and clinicians is important during all phases of CDSSs development.47 Within dentistry, different ML algorithms have been utilized based on the size of the data, variables/features to analyse, and the objective of the model. ML can involve unsupervised or supervised learning.48

In unsupervised learning, algorithms detect hidden patterns within an unlabelled data set. That means all variables within the training data set are utilized as inputs, and the machine will automatically discover structures/patterns within that data set without receiving instructions about the desired outcomes.48,49 Based on the problem at hand, unsupervised learning algorithms will split the data set into groups (clustering) or find rules representing the relationship between variables within a data set (association).49 For example, in a study conducted by Auconi et al,50 combinations of variables (inputs) were provided to the fuzzy cluster, which detected the best phenotypic factors to group a sample of Class-III patients into subjects with increased mandibular dimensions, subjects with increased maxillomandibular divergence, and subjects with intermediate characteristics between the two groups.

Supervised learning algorithms, on the other hand, analyse training data sets with predetermined inputs and outputs. Consequently, the inferred ML model can predict the outputs of new data. Common tasks of supervised learning algorithms include classification and regression.49 The classification task aims to detect a function (discrete value) that aids in splitting the data set into classes based on various parameters.51 For example, using a classification algorithm and a training data set consisting of patients’ intra-oral and cephalometric findings, the supervised ML model can determine (i.e. classify) the cases that need or do not need a tooth extraction for orthodontic reasons.52 In the regression task, the correlation between dependent and independent variables is detected during the machine training, and the developed model can predict continuous variables.51 For instance, an ML model trained with a regression algorithm can predict dental age (continuous variable) from the pulp-to-tooth ratio of the canines.53 When training with supervised ML algorithms, it is possible to generate an overfit model that performs well only on the training data set. Thus, it is essential to evaluate the developed model’s generalizability and tune the model parameters through validation methods, for example, cross-validation.54 Last, the performance of the model should be evaluated using the test data set. For that, various methods exist and should be selected based on the task of the ML algorithms (classification or regression) and the type of outcomes.55

Interestingly, a review conducted by Asiri et al9 revealed that most of the ML applications in orthodontics have utilized supervised machine learning algorithms to automate clinical procedures that execute or assist in diagnosis and treatment planning. Radiographs were commonly targeted in developing those ML models as they are considered essential tools for orthodontic diagnosis, treatment planning and evaluation of treatment outcomes. Discrepancies in landmarks identification have been recognized as a critical source of error in cephalometric analyses. Since the analysis’ diagnostic value relies on the reproducibility and precision of landmarks identification, interests to develop an automated approach have increased to reduce the laboriousness of the task and subjectivity of the analysts. Park et al56 reported that training the machine with the YOLOv3 algorithm for automatic labelling of 80 landmarks resulted in small error plots and 5% improvements in the accuracy compared to top benchmarks reported in the literature. Additionally, the mean computational time consumed per image was only 0.05 seconds. Similarly, Kunz et al57 reported promising results obtained with an AI algorithm that can analyse new cephalometric X-rays with precision comparable to the gold standard, that is, experienced human examiners. Furthermore, Lee et al58 presented an automated framework for cephalometric landmarks detection with the implementation of confidence regions (95%) around the estimated positions of the landmarks. This will allow clinicians to gauge the accuracy of the size and location of the calculated landmarks.

Besides using the two-dimensional radiographs for the cephalometric analysis, accurate measurements can be attained with CBCT imaging modality; CBCT provides an accurate three-dimensional spatial representation of the oral and craniofacial structures. However, the accuracy of manual landmarks plotting on the CBCT requires substantial effort, experience, and time. Gupta et al reported an automatic knowledge-based landmark detection algorithm able to produce accurate cephalometric measurements comparable to those computed from manual identification.59 Automation of the radio-graphic analysis has also involved attempts to estimate the skeletal maturation that provides the best estimate of the individual’s biological age and aid treatment planning.60 Kashif et al61 used a classifier algorithm and developed a tool that can help predict the bone age from hand radiographs with a mean error of 0.6 years compared to the average reading of two experienced radiologists. Kok et al,62 on the other hand, reported different algorithms that predict the skeletal maturation from the cervical vertebrae on lateral cephalograms. ML algorithms have also been utilized to develop an automated imaging system that provides objective morphological facial assessment during the orthodontic diagnosis process.63 Furthermore, various methods for automatic volumetric segmentation of CBCT images were developed using ML algorithms. This will allow objective generation of three-dimensional models for advanced diagnosis and treatment planning whilst saving the time and efforts required for manual segmentation.64

Growth and development of orofacial complex are influenced by the interaction of the genetic and environmental factors. Orthodontists are mainly using phenotype-driven diagnostic tools like cephalometric analyses to predict the growth in individuals with Class-II or class-III skeletal malocclusions. However, future studies of genetics, epigenetics and metabolic pathways that utilize advanced machine learning tools will transform the process of orthodontic diagnosis and treatment planning.47,65

2.2.4 |. Data communication

The next step after developing the ML-based model is to communicate the ML findings with the clinicians. Production of predictive models in a real-world healthcare setting is more challenging than developing models in an experimental environment. Therefore, it is important for clinical experts who were not involved in tools development to test and validate the system’s performance. Following the deployment of the ML model, it should be monitored for reliability and correction of errors since clinical protocols and populations are changing over time.66

The CDSSs, reviewed in this manuscript, will improve Orthodontic care only if clinicians utilize ML and AI tools to analyse the inter-relationships among the dentition, craniofacial skeleton and soft tissues. Then, translate the acquired knowledge towards the advancement of orthodontic diagnosis, treatment planning, evaluation of growth and development, assessment of treatment progression, outcomes and stability.

3 |. DATA STORAGE FOR COMPUTATION AND INTEGRATION

Our clinical research team, the DBCIA,67 has developed a web-based system called Data Storage Computation and Integration (DSCI) for the management of data science approaches in TMJ and dental CDSSs. The DSCI allows clinicians and researchers to store and share de-identified data between multiple clinical centres. In addition, it allows data processing, in-depth data analysis with several machine learning algorithms and outcomes communication with the users (Figure 2B).68 The security and privacy of the access to the DSCI are handled using Jason Web Tokens, with encryption of each user who requests to log in. The DSCI uses Amazon Web Services which enable distributed computing across multi-site clinical centres. Furthermore, the web data management server architecture facilitates scalability and inclusion of plugins or processing pipelines to exploit data sets stored in the web system resources.69

3.1 |. TMJ clinical decision support system

3.1.1 |. Rationale

Osteoarthritis of the TMJ (TMJ OA) is a chronic debilitating disease that affects millions of people and poses a burden on public health globally.44 It is a multifactorial disease that results from biological and mechanical events that destabilize normal coupling of synthesis and degradation of the subchondral bone and the articular cartilage.70 Diagnosis of the TMJ OA is currently based on pre-existent clinical signs and symptoms/imaging markers following the recommendations of the Diagnostic Criteria for Temporomandibular Disorders (DC/TMD).71 However, several studies showed that clinical diagnosis is poorly correlated with the bony changes in CBCT images. Therefore, new assessment tools are needed for the precision of the diagnoses.70

3.1.2 |. TMJ decision optimization

The CDSS for the TMJ, deployed in the DSCI, enables reliable detection of the TMJ OA and visualization of surface changes of the affected condyles (Figure 3). The TMJ CDSS has components for data storage of biological, clinical and imaging data (e.g. magnetic resonance images, panoramic images, CT and CBCT scans). Furthermore, it possesses a tool for data processing (TMJseg) that allows automatic segmentation of the condyles from the CBCT scans.72 Regarding the in-depth analytics component of the CDSS, the DSCI’s statistical analysis, cross-validation and machine learning (Light GBM and XGBoost) tools permit users to integrate patient-specific multi-source data and to obtain a diagnosis of the TMJ condition, that is, healthy or diseased. A recent study by Bianchi et al44 showed that the use of the DSCI tools facilitated the diagnosis of TMJ OA in its initial stages with an accuracy of 0.823. Moreover, they demonstrated that the interaction of the biomolecular features has a large contribution to the prediction of the TMJ OA status. Furthermore, the DSCI has a machine-learning model that detects changes of the condyles’ surfaces (Shape Variation Analyzer) which aids in classifying the TMJ OA disease into different categories based on the degree of the condylar degeneration.73

FIGURE 3.

FIGURE 3

Open in a new tab

Data science approaches in the TMJ clinical decision support system

3.2 |. Dental clinical decision support system

3.2.1 |. Rationale

Currently, the commercial companies that fabricate clear aligners are utilizing data from digital dental models and applying AI algorithms to predict and plan teeth movement, as well as to undertake teeth segmentation. However, such AI algorithms have not been validated and require caution by clinicians in terms of utilizing the provided predictions as well as monitoring treatments’ results.56 Moreover, these technological advancements also require the integration of multi-source data capture, including clinical information and three-dimensional imaging exams such as CBCT, digital dental models (DDMs), photographs, lateral cephalogram and panoramic X-rays.66

3.2.2 |. Dental decision optimization

The CDSS for dental applications deployed in the DSCI integrates dental crowns’ and root canals’ relevant clinical information from the DDMs and CBCT scans, respectively (Figure 4). In addition, it provides tools for automatic segmentation of root canals (RootCanalSeg), teeth and gums (DentalModelSeg).74 Overcoming the registration challenges created by merging the information from different imaging modalities and DDM, a work in progress, will enable reliable quantitative assessments of teeth movement (Figure 4).

FIGURE 4.

FIGURE 4

Open in a new tab

Sequence of data science approaches in the dental decision support system

The DSCI AI tools incorporate a full spectrum of data science approaches that provide optimized clinical information and promote personalized care delivery across various fields of dentistry.

4 |. CONCLUSION

Clinical Decision Support Systems incorporate knowledge with patient-specific data to serve clinicians with tools that enhance their clinical decision-making process. Thorough understanding of the steps involved in developing DSS and communication between clinicians, data scientists and analysts are keys to creating successful tools that fit into the clinical workflow.

Funding information

American Association of Orthodontists Foundation; National Institute of Dental and Craniofacial Research, Grant/Award Number: R01DE024450

Footnotes

CONFLICT OF INTEREST

The authors declare no potential conflicts of interest with respect to the authorship and/or publication of this article.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available upon reasonable request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

REFERENCES

  • 1.Mascitti M, Campisi G. Dental public health landscape: challenges, technological innovation and opportunities in the 21st century and COVID-19 pandemic. Int J Environ Res Public Health. 2020;17(10):3636. 10.3390/ijerph17103636 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Finkelstein J, Zhang F, Levitin S, Cappelli D. Using big data to promote precision oral health in the context of a learning healthcare system. J Public Health Dent. 2020;80(S1):S43–S58. 10.1111/jphd.12354 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Dash S, Shakyawar S, Sharma M, Kaushik S. Big data in healthcare: management, analysis and future prospects. J Big Data. 2019;6(1):1–25. 10.1186/s40537-019-0217-0 [DOI] [Google Scholar]
  • 4.Pastorino R, De Vito C, Migliara G, et al. Benefits and challenges of big data in healthcare: an overview of the european initiatives. Eur J Public Health. 2019;29(Supplement_3):23–27. 10.1093/eurpub/ckz168 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Frank M, Drikakis D, Charissis V. Machine-learning methods for computational science and engineering. Computation. 2020;8(1):15. 10.3390/computation8010015 [DOI] [Google Scholar]
  • 6.Panahiazar M, Taslimitehrani V, Jadhav A, Pathak J. Empowering personalized medicine with big data and semantic web technology: promises, challenges, and use cases. Washington, DC, USA. (pp. 790–795). IEEE International Conference on Big Data (Big Data); 2014. 10.1109/BigData.2014.7004307 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Vikram K, Karjodkar F. Decision support systems in dental decision making: an introduction. J Evid Based Dent Pract. 2009;9(2):73–76. 10.1016/j.jebdp.2009.03.003 [DOI] [PubMed] [Google Scholar]
  • 8.Fieschi M, Gouvernet J. Reasoning foundations of medical diagnosis revisited. Yearb Med Inform. 1999;08(01):78–82. 10.1055/s-0038-1637913 [DOI] [PubMed] [Google Scholar]
  • 9.Asiri S, Tadlock L, Schneiderman E, Buschang P. Applications of artificial intelligence and machine learning in orthodontics. APOS Trends Orthod. 2020;10:17–24. 10.25259/apos_117_2019 [DOI] [Google Scholar]
  • 10.Khanagar SB, Al-Ehaideb A, Vishwanathaiah S, et al. Scope and performance of artificial intelligence technology in orthodontic diagnosis, treatment planning, and clinical decision-making - a systematic review. J Dent Sci. 2021;16(1):482–492. 10.1016/j.jds.2020.05.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Wiharto W Clinical decision support systems theory and practice. Jurnal Teknosains. 2018;7(2):148. 10.22146/teknosains.38641 [DOI] [Google Scholar]
  • 12.Helmons P, Suijkerbuijk B, Nannan Panday P, Kosterink J. Drug-drug interaction checking assisted by clinical decision support: a return on investment analysis. J Am Med Inform Assoc. 2015;22(4):764–772. 10.1093/jamia/ocu010 [DOI] [PubMed] [Google Scholar]
  • 13.Sutton R, Pincock D, Baumgart D, Sadowski D, Fedorak R, Kroeker K. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3(1):17. 10.1038/s41746-020-0221-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Zikos D, DeLellis N. CDSS-RM: a clinical decision support system reference model. BMC Med Res Methodol. 2018;18(1):137. 10.1186/s12874-018-0587-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Joda T, Waltimo T, Probst-Hensch N, Pauli-Magnus C, Zitzmann N. Health data in dentistry: an attempt to master the digital challenge. Public Health Genomics. 2019;22(1–2):1–7. 10.1159/000501643 [DOI] [PubMed] [Google Scholar]
  • 16.Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential. Health Inf Sci Syst. 2014;2(1):3. 10.1186/2047-2501-2-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Nanayakkara S, Zhou X, Spallek H. Impact of big data on oral health outcomes. Oral Dis. 2018;25(5):1245–1252. 10.1111/odi.13007 [DOI] [PubMed] [Google Scholar]
  • 18.Lu J, Liu A, Song Y, Zhang G. Data-driven decision support under concept drift in streamed big data. Complex Intell Syst. 2019;6(1):157–163. 10.1007/s40747-019-00124-4 [DOI] [Google Scholar]
  • 19.Joda T, Waltimo T, Pauli-Magnus C, Probst-Hensch N, Zitzmann N. Population-based linkage of big data in dental research. Int J Environ Res Public Health. 2018;15(11):2357. 10.3390/ijerph15112357 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Library guides: data resources in the health sciences: clinical data. Guides.lib.uw.edu. https://guides.lib.uw.edu/hsl/data/findclin. Published 2021. Accessed February 26, 2021
  • 21.Cohen M Impact of the HITECH financial incentives on EHR adoption in small. Physician-Owned Practices. Int J Med Inform. 2016;94:143–154. 10.1016/j.ijmedinf.2016.06.017 [DOI] [PubMed] [Google Scholar]
  • 22.Henry J, Pylypchuk Y, Searcy T, Patel V. Adoption of Electronic Health Record Systems Among U.S. Non-Federal Acute Care Hospitals: 2008–2015. Washington, DC, USA: Office of the National Coordinator for Health Information Technology; 2016. [Google Scholar]
  • 23.Jamoom E, Yang N. Table of electronic health record adoption and use among office-based physicians in the U.S. by state: 2015 National Electronic Health Records Survey; 2016 [Google Scholar]
  • 24.Electronic Health Record Data Governance and Data Quality in the Real World. HIMSS. https://www.himss.org/resources/electronic-health-record-data-governance-and-data-quality-real-world. Published 2021. Accessed March 3, 2021.
  • 25.Wynants L, Kent D, Timmerman D, Lundquist C, Van Calster B. Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting. Diagn Progn Res. 2019;3(1):6. 10.1186/s41512-019-0046-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Perazzo J, Rodriguez M, Currie J, Salata R, Webel A. Creation of data repositories to advance nursing science. West J Nurs Res. 2017;41(1):78–95. 10.1177/0193945917749481 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Favaretto M, Shaw D, De Clercq E, Joda T, Elger B. Big data and digitalization in dentistry: a systematic review of the ethical issues. Int J Environ Res Public Health. 2020;17(7):2495. 10.3390/ijerph17072495 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Johnny PJ. BigMouth Dental Data Repository. Bigmouth.uth.edu. https://bigmouth.uth.edu/. Published 2021. Accessed February 26, 2021 [Google Scholar]
  • 29.Tucker K, Branson J, Dilleen M, et al. Protecting patient privacy when sharing patient-level data from clinical trials. BMC Med Res Methodol. 2016;16(S1):77. 10.1186/s12874-016-0169-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.de Dumast P, Mirabel C, Cevidanes L, et al. A web-based system for neural network based classification in temporomandibular joint osteoarthritis. Comput Med Imaging Graph. 2018;67:45–54. 10.1016/j.compmedimag.2018.04.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Brosset S, Dumont M, Cevidanes L, et al. Web infrastructure for data management, storage and computation. Proc SPIE Int Soc Opt Eng. 2021;11600:116001N. 10.1117/12.2582283 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Schwendicke F, Samek W, Krois J. Artificial Intelligence in Dentistry: Chances and Challenges. J Dent Res. 2020;99(7):769–774. 10.1177/0022034520915714 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Artificial Intelligence: Implications for the Future of Work | | Blogs | CDC. Blogs.cdc.gov. https://blogs.cdc.gov/niosh-science-blog/2019/08/26/ai/.Published 2021. Accessed March 2, 2021. [Google Scholar]
  • 34.Gudivada V, Apon A, Ding J. Data quality considerations for big data and machine learning: going beyond data cleaning and transformations. Int J Advances in Software. 2017;10(1):1–20. [Google Scholar]
  • 35.Tallon-Ballesteros A, Riquelme J. Deleting or keeping outliers for classifier training? 2014 sixth world congress on Nature and Biologically Inspired Computing (NaBIC 2014). 2014. 10.1109/nabic.2014.6921892 [DOI] [Google Scholar]
  • 36.Al-Jabery K, Obafemi-Ajayi T, Olbricht G, Wunsch D. Computational Learning Approaches to Data Analytics in Biomedical Applications. Academic Press, an Imprint of Elsevier; 2020. [Google Scholar]
  • 37.Moore C Image Normalization | Radiology Reference Article | Radiopaedia.org.. Radiopaedia.org. https://radiopaedia.org/articles/image-normalization?lang=us. Published 2021. Accessed March 6, 2021. [Google Scholar]
  • 38.Ashburner J, Friston KJ. Chapter 33-Spatial Normalisation Using Basis Functions. Frackowiak RSJ, Friston KJ, Frith CD et al., Human Brain Function. (2nd ed.). Cambrige, MA: Academic Press; 2004:655–672.ISBN 9780122648410. [Google Scholar]
  • 39.Ioshida M, Muñoz BA, Rios H, et al. Accuracy and reliability of mandibular digital model registration with use of the mucogingival junction as the reference. Oral Surg Oral Med Oral Pathol Oral Radiol. 2019;127(4):351–360. 10.1016/j.oooo.2018.10.003 [DOI] [PubMed] [Google Scholar]
  • 40.Reinhold J, Dewey B, Carass A, Prince J. Evaluating the impact of intensity normalization on MR image synthesis. Proc SPIE Int Soc Opt Eng. 2019;10949:109493H. 10.1117/12.2513089 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Bianchi J, Paniagua B, De Oliveira Ruellas AC. 3D slicer craniomaxillofacial modules support patient-specific decision-making for personalized healthcare in dental research. Multimodal Learn Clin Decis Support Clin Image Based Proc (2020). 2020;12445:44–53. 10.1007/978-3-030-60946-7_5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Data mining. En.wikipedia.org. https://en.wikipedia.org/wiki/Data_mining. Published 2021. Accessed March 6, 2021. [Google Scholar]
  • 43.Liu H Feature Selection. In: Sammut C, Webb GI, (eds.). Encyclopedia of Machine Learning. Boston, MA: Springer; 2011. 10.1007/978-0-387-30164-8_306 [DOI] [Google Scholar]
  • 44.Bianchi J, de Oliveira Ruellas AC, Gonçalves JR, et al. Osteoarthritis of the Temporomandibular Joint can be Diagnosed Earlier Using Biomarkers and Machine Learning. Sci Rep. 2020;10(1):8012. 10.1038/s41598-020-64942-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Hwang HW, Moon JH, Kim MG, Donatelli RE, Lee SJ. Evaluation of automated cephalometric analysis based on the latest deep learning method. Angle Orthod. 2021;91(3):329–335. 10.2319/021220-100.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Thomas BD. Machine learning algorithms: a clinician and data analyst partnership. http://Acumenmd.com. https://acumenmd.com/blog/machine-learning-algorithms-a-clinician-and-data-analyst-partnership/. Published 2021. Accessed March 8, 2021. [Google Scholar]
  • 47.Allareddy V, Rengasamy Venugopalan S, Nalliah R, Caplin J, Lee M, Allareddy V. Orthodontics in the era of big data analytics. Orthod Craniofac Res. 2019;22(S1):8–13. 10.1111/ocr.12279 [DOI] [PubMed] [Google Scholar]
  • 48.Alloghani M, Al-Jumeily D, Mustafina J, Hussain A, Aljaaf AJ. A systematic review on supervised and unsupervised machine learning algorithms for data science. In Berry M, Mohamed A, Yap B, eds. Supervised and Unsupervised Learning for Data Science. Unsupervised and Semi-Supervised Learning New York, NY: Springer, Cham; 2020. 10.1007/978-3-030-22475-2_1 [DOI] [Google Scholar]
  • 49.Auconi P, Scazzocchio M, Cozza P, McNamara J, Franchi L. Prediction of class III treatment outcomes through orthodontic data mining. Eur J Orthod. 2014;37(3):257–267. 10.1093/ejo/cju038 [DOI] [PubMed] [Google Scholar]
  • 50.Regression vs classification in machine learning - Javatpoint. www.javatpoint.com. https://www.javatpoint.com/regression-vs-classification-in-machine-learning. Published 2021. Accessed March 6, 2021. [Google Scholar]
  • 51.Jung S, Kim T. New approach for the diagnosis of extractions with neural network machine learning. Am J Orthod Dentofac Orthop. 2016;149(1):127–133. 10.1016/j.ajodo.2015.07.030 [DOI] [PubMed] [Google Scholar]
  • 52.Farhadian M, Salemi F, Saati S, Nafisi N. dental age estimation using the pulp-to-tooth ratio in canines by neural networks. Imaging Sci Dent. 2019;49(1):19. 10.5624/isd.2019.49.1.19 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Sidey-Gibbons J, Sidey-Gibbons C. Machine learning in medicine: a practical introduction. BMC Med Res Methodol. 2019;19(1):64. 10.1186/s12874-019-0681-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Jiang T, Gradus J, Rosellini A. Supervised machine learning: a brief primer. Behav Ther. 2020;51(5):675–687. 10.1016/j.beth.2020.05.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Park J-H, Hwang H-W, Moon J-H, et al. Automated identification of cephalometric landmarks: part 1—comparisons between the latest deep-learning methods YOLOV3 and SSD. Angle Orthod. 2019;89(6):903–909. 10.2319/022019-127.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Faber J, Faber C, Faber P. Artificial intelligence in orthodontics. APOS Trends Orthod. 2019;9:201–205. 10.25259/apos_123_2019 [DOI] [Google Scholar]
  • 57.Lee J, Yu H, Kim M, Kim J, Choi J. Automated cephalometric landmark detection with confidence regions using bayesian convolutional neural networks. BMC Oral Health. 2020;20(1):270. 10.1186/s12903-020-01256-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Gupta A, Kharbanda O, Sardana V, Balachandran R, Sardana H. Accuracy of 3D cephalometric measurements based on an automatic knowledge-based landmark detection algorithm. Int J Comput Assist Radiol Surg. 2015;11(7):1297–1309. 10.1007/s11548-015-1334-7 [DOI] [PubMed] [Google Scholar]
  • 59.Buschang P, Roldan S, Tadlock L. Guidelines for assessing the growth and development of orthodontic patients. Semin Orthod. 2017;23(4):321–335. 10.1053/j.sodo.2017.07.001 [DOI] [Google Scholar]
  • 60.Kashif M, Deserno T, Haak D, Jonas S. Feature description with SIFT, SURF, BRIEF, BRISK, or FREAK? A general question answered for bone age assessment. Comput Biol Med. 2016;68:67–75. 10.1016/j.compbiomed.2015.11.006 [DOI] [PubMed] [Google Scholar]
  • 61.Kök H, Acilar A, İzgi M. Usage and comparison of artificial intelligence algorithms for determination of growth and development by cervical vertebrae stages in orthodontics. Prog Orthod. 2019;20(1):41. 10.1186/s40510-019-0295-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Murata S, Lee C, Tanikawa C, Date S. Towards a fully automated diagnostic system for orthodontic treatment in dentistry. 2017 IEEE 13th International Conference on e-Science (e-Science). 2017. 10.1109/escience.2017.12 [DOI] [Google Scholar]
  • 63.Bayirli B, Kim-Berman H, Puntillo A. Embracing novel technologies in dentistry and orthodontics. Hdl.handle.net. http://hdl.handle.net/2027.42/153991. Published 2021. Accessed March 9, 2021 [Google Scholar]
  • 64.Varma G, Harsha B, Palla S, Sravan S, Raju J, Rajavardhan K. Genetics in an Orthodontic Perspective. J Adv Clin Res Insights. 2019;6(3):86–90. 10.15713/ins.jcri.267 [DOI] [Google Scholar]
  • 65.Wiens J, Saria S, Sendak M, et al. Do no harm: a roadmap for responsible machine learning for health care. Nat Med. 2019;25(9):1337–1340. 10.1038/s41591-019-0548-6 [DOI] [PubMed] [Google Scholar]
  • 66.Li P, Kong D, Tang T, et al. Orthodontic treatment planning based on artificial neural networks. Sci Rep. 2019;9(1):2037. 10.1038/s41598-018-38439-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Dental and craniofacial bionetwork for image analysis, DCBIA. https://sites.google.com/a/umich.edu/dentistry-image-computing/. Accessed March 11, 2021
  • 68.Heinrichs J, Lim J. Integrating web-based data mining tools with business models for knowledge management. Decis Support Syst. 2003;35(1):103–112. 10.1016/s0167-9236(02)00098-2 [DOI] [Google Scholar]
  • 69.Khanagar SB, Al-ehaideb A, Maganur PC, et al. Developments, application, and performance of artificial intelligence in dentistry – a systematic review. J Dent Sci. 2021;16(1):508–522. 10.1016/j.jds.2020.06.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Su N, Liu Y, Yang X, Shen J, Wang H. Correlation between oral health-related quality of life and clinical dysfunction index in patients with temporomandibular joint osteoarthritis. J Oral Sci. 2016;58(4):483–490. 10.2334/josnusd.16-0224 [DOI] [PubMed] [Google Scholar]
  • 71.Schiffman E, Ohrbach R, Truelove E, et al. Diagnostic criteria for Temporomandibular Disorders (DC/TMD) for clinical and research applications: recommendations of the International RDC/TMD consortium network* and orofacial pain special interest group†. J Oral Facial Pain Headache. 2014;28(1):6–27. 10.11607/jop.1151 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Brosset S, Dumont M, Bianchi J, et al. 3D Auto-Segmentation of Mandibular Condyles. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). 2020. 10.1109/embc44109.2020.9175692 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Tubau N, de Dumast P, Yatabe M, et al. Shape variation analyzer: a classifier for temporomandibular joint damaged by osteoarthritis. Proc SPIE Int Soc Opt Eng. 2019;10950:1095021. 10.1117/12.2506018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Dumont M, Prieto J, Brosset S, et al. Patient specific classification of dental root canal and crown shape. Shape Med Imaging. 2020;12474:145–153. 10.1007/978-3-030-61056-2_12 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data that support the findings of this study are available upon reasonable request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

RESOURCES