Validate artificial intelligence for the diagnosis of periodontal disease

Abstract

Background

Periodontal disease is among the most prevalent diseases globally, yet it often goes undetected, with only 27% of cases reported to receive treatment in the US. In situations when clinical examination is not possible, radiographic findings may serve as an alternative method for periodontal disease diagnosis. Several Artificial Intelligence (AI) platforms were developed to detect the amount of radiographic bone loss with different accuracy levels. Therefore, this study aimed to validate the Overjet AI platform for diagnosing periodontal disease using full-mouth radiographs against the gold standard of clinical-radiographic analysis, and to compare its diagnostic accuracy with manual radiographic assessment by a general practitioner (GP).

Methods

In this study radiographic records of patients aged over 29 years were utilized to validate the use of radiographic analysis (using full-mouth radiographs) solely by GP and by Overjet AI software for detecting periodontal disease. A sample size calculation was performed with 95% power and an alpha of 0.05 to identify an effect size of 0.2. To evaluate the diagnostic accuracy of the Overjet AI software, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated against the gold standard of the combined clinical and radiographic analysis conducted by a periodontist. Cohen’s Kappa agreement was used to compare results from manual radiographic analysis by GP verses AI platform.

Results

The study included radiographic records of 103 patients. Results showed that detecting periodontal disease across different severity level using radiographic analysis manually achieved 100 − 83% sensitivity and 94 − 90% specificity. While Overjet AI software achieved 100 − 82% sensitivity and 96 − 89% specificity. The Cohen’s Kappa agreement between the GP and AI platform results was between 0.49 and 0.85 representing a moderate to almost perfect agreement.

Conclusion

AI-based radiographic analysis offers a rapid and accurate alternative to manual dentist assessments, particularly for detecting moderate to severe periodontal disease. The findings suggest a potential for integrating AI technology with conventional clinical exam to improve the efficiency and accuracy of periodontal disease detection and monitoring.

Introduction

Periodontal diseases consist of a wide range of inflammatory conditions that destroy the tooth supporting structures, which might lead to tooth loss and systemic inflammation. The initiation and progression of periodontal disease are often attributed to an imbalance in the oral microbiota, which triggers an interaction with the host’s immune defenses, leading to inflammation and disease [1]. Periodontal disease is the leading cause of tooth loss among adults that impairs the dental function and quality of life. The pathophysiology of periodontitis remains incompletely understood; however, it involves microbial challenges posed by periodontal pathogens such as Porphyromonas gingivalis, Treponema denticola, Tannerella forsythia, and Aggregatibacter actinomycetemcomitans, along with a complex inflammatory host response that contributes significantly to the breakdown of periodontal tissues [2]. Over the last two decades, epidemiological evidence has suggested a link between periodontal diseases and conditions like glucose intolerance, dyslipidemia, low-grade systemic inflammation, and systemic diseases, including cardiovascular disease (CVD) and diabetes [3].

Clinicians and researchers employ various case definitions for periodontal disease. The assessment of this condition relies on clinical examinations, including measurements of probing depth (PD) and clinical attachment loss (CAL), as well as radiographic findings and a thorough review of medical and dental histories to ensure an accurate diagnosis [4]. For large-scale epidemiological studies or surveillance, such as the National Health and Nutrition Examination Survey (NHANES), the Centers for Disease Control and Prevention (CDC) and the American Academy of Periodontology (AAP) have established a case definition for periodontitis based solely on clinical measures (CAL and PD) [5]. In situations where clinical measurements are unavailable, radiographic findings, specifically radiographic bone loss (RBL), may serve as an alternative method for periodontal disease classification [6]. Moreover, one of the primary motivations for integrating AI into healthcare, including dentistry, is to enable accurate and efficient analysis of patient records and diagnoses [7]. Given that diagnosing periodontal disease is a time-intensive process heavily reliant on the provider’s skills and experience, AI systems were developed to assist professionals by delivering detection capabilities that are as precise as, or even superior to, those of dental practitioners [8]. Even with good experience, clinician fatigue is expected, which could affect the diagnosis accuracy. Periodontal disease often goes undetected, and it is reported that only 27% is treated [9].

Several AI companies are developing advanced solutions for dental disease diagnosis, focusing on improving accuracy and efficiency [7]. Among these, Overjet stands out by creating AI-powered software that analyzes dental radiographs to detect and quantify periodontal disease and other oral conditions [7]. Overjet was established in 2018 at the Harvard Innovation Labs aiming to standardize radiographic assessments and enhance diagnostic precision for dental professionals. Several AI models have been developed for detecting the amount of RBL, with reported accuracy ranging from 73.4% to 99% [8]. The aim of this study was to assess the validity of the Overjet AI platform for detecting periodontal disease, we evaluated its diagnostic accuracy against the reference standard (combined clinical and radiographic assessment). Diagnostic accuracy measurements were calculated to provide quantitative evidence of the test’s validity. The agreement between AI-based and manual radiographic assessments by a general practitioner (GP) was assessed.

Materials and methods

Study design and reporting standards

This retrospective diagnostic accuracy study was conducted and reported in accordance with the STARD (Standards for Reporting Diagnostic Accuracy Studies) guidelines and the CLAIM (Checklist for Artificial Intelligence in Medical Imaging) recommendations to ensure transparent, standardized reporting of AI-based diagnostic performance. These frameworks guided the description of the index tests (AI-based radiographic assessment and manual radiographic assessment by a GP), the reference standard (combined clinical and radiographic evaluation by a periodontist), dataset handling, blinding procedures, model inputs, and performance metrics.

Institution Review Board (IRB) exemptions were obtained from both Tufts University (HRP-210) and Harvard University (IRB23-1410) prior to initiating the study. Also, Data Use Agreement (DUA24-0762) was obtained to ensure the lawful sharing, protection, and confidentiality of patient information.

Participants and inclusion criteria

Patients’ files were randomly selected from the dental records at Tufts University School of Dental Medicine (TUSDM). The inclusion criteria were patients 30 years or older, completed periodontal examination chart (CAL, PD, gingival recessions) conducted by residents or faculty members from the periodontal department, good quality full mouth radiographs (4 bitewings and 14 periapical radiographs), no history of systemic diseases, and no history of periodontal surgeries. The general practitioner and the AI platform performed radiographic assessments independently and were blinded to the reference standard diagnosis. The periodontist performing the reference standard assessment was blinded to both index test results.

Clinical measures, in conjunction with radiographic analysis (in line with the periodontal disease definition from the 2017 world workshop on the classification of periodontal and peri-implant diseases), were utilized for the diagnosis of periodontal disease [6]. Moreover, radiographic analysis solely was performed (manually using MiPACS software and by AI using Overjet software) to assess periodontal disease for the same participants. Three diagnostic methods for periodontal disease were assessed:

• A)Manual radiographic assessment

  MiPACS Dental Enterprise Viewer was used by a GP as the index test to view and measure RBL on full mouth radiographs (4 bitewing and 14 periapical images). If the distance between the cementoenamel junction (CEJ) and alveolar bone crest was ≤ 1.5 mm on these radiographs (it is the normal location of bone crest and it means there is no RBL), participants were classified as no periodontal disease [6, 11]. Patients were categorized as stage I if the amount of RBL on periapical radiographs was less than 15% on at least two non-neighboring teeth. Those who had RBL between 15 and 33% on at least two non-neighboring teeth were classified as stage II. Finally, patients with RBL extending to the middle third and beyond on at least two non-neighboring teeth were categorized as stage III/IV [6].

  Also, periodontal disease was categorized as “yes” or “no” where yes was given to participants who had any RBL (distance between CEJ and bone crest was >1.5 mm on radiographs) on at least two non-neighboring teeth, and no for those who did not [6, 10].

  In addition, periodontal disease was categorized as “moderate to severe” or “no moderate to severe” where moderate to severe was given to participants who had RBL more than 15% on at least two non-neighboring teeth, and no for those who did not. Finally, periodontal disease was categorized as “severe” or “no severe” where severe was given to participants who had RBL more than 33% on at least two non-neighboring teeth, and no for those who did not [6].

• B)AI-based radiographic analysis:

  The AI platform by Overjet was used as the second index test to measure RBL on full mouth radiographs. The AI system received imaging data only, without access to clinical or demographic variables. All outputs were automatically generated without human postprocessing or manual correction. Patients were classified for periodontal disease as healthy, stage I, stage II, and stage III/IV. The AI platform measured the length of root surface uncovered by bone (distance from the CEJ to the alveolar bone crest), the total root length (distance from the CEJ to the apex), and the percentage of uncovered root surface. The method used for RBL measurement and periodontal disease classification followed the same criteria outlined in point (B). Based on a correction factor from average root lengths of each tooth (it was also accounted for the differences in average root lengths of both sexes for anteriors and premolars only) [11, 12] and the amount of uncovered root surface, the RBL was calculate.

• C)Combination of clinical and radiographic methods

  The combination method conducted by an experienced periodontist was considered the gold standard or the reference for evaluating how effectively the first two methods (radiographic analysis by GP, and radiographic analysis by AI) detected periodontal disease. Moreover, diagnostic results from the radiographic analysis by the AI platform was compared to radiographic analysis by a GP with 5-year clinical experience.

As recommended by CLAIM, detailed information on dataset composition was recorded, including the number of radiographs, imaging modalities, and acquisition parameters. All radiographs were acquired as part of routine clinical care using standardized imaging protocols at TUSDM. No images were reprocessed, modified, or enhanced prior to analysis. The dataset was used exclusively for external testing; no portion of the images was used to train, develop, or fine-tune the Overjet AI model.

Sample size

The sample size was determined using G*Power software (version 3.1.9), with a 95% power and an alpha of 0.05, to detect an effect size of 0.2. Considering the 0.42 prevalence of periodontal disease in the US population [13], a sample size of 82 subjects was required. To account for any insufficient data on patients’ records, an initial sample size of 100 subjects was used.

Statistical analysis

Periodontal disease assessment using clinical measures alone (CAL and pocket depth) was conducted. In addition, radiographic analysis solely was performed (manually using MiPACS software and by AI using Overjet software) to assess periodontal disease for the same participants.

The diagnostic results from radiographic findings by AI platform as well as radiographic findings by GP were compared to the combined assessment method (using both the clinical measures and radiographic analysis) conducted by an experienced periodontist, which was considered the gold (reference) standard. Diagnostic accuracy measures including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and overall accuracy were calculated to provide quantitative evidence of the AI platform’s validity in detecting periodontal disease. These measures quantify how well the AI test identifies disease presence and severity compared with the gold standard, supporting its use as a valid diagnostic tool (Figs. 1).

Fig. 1.

Comparing radiographic findings by Overjet AI platform and radiographic findings by general practitioner to the combination method (the gold standard). GP: General Practitioner, AI: Artificial Intelligence, PPV: Positive Predictive Value, NPV: Negative Predictive Value

The overall accuracy of the diagnostic method was calculated to indicate the proportion of true positive and true negative cases correctly identified by the test out of the total cases. The accuracy test reflects the proportion of correct classifications in a diagnostic test [14]. Cohen’s Kappa was used to measure the level of agreement between the diagnostic results from the AI and the dentist’s radiographic analyses for detecting various severity levels of periodontal disease. Cohen’s Kappa values, which range from 0 to 1, signify the degree of agreement, with 1 indicating perfect agreement. The interpretations were as follows: ≤ 0 (poor), 0.01–0.20 (slight), 0.21–0.40 (fair), 0.41–0.60 (moderate), 0.61–0.80 (substantial), and 0.81–1.00.81.00 (almost perfect) [15]. Finally, the time taken for diagnosis using radiographs was evaluated to compare the time required for the manual diagnosis by a GP versus the AI platform (Fig. 1). In alignment with CLAIM, diagnostic performance metrics were reported with complete transparency, including sensitivity, specificity, predictive values, overall accuracy, and agreement metrics. No subgroup analyses or model recalibration procedures were performed.

Results

Participants description

A total of 103 patients met the inclusion criteria and were included in the study. All participants were selected from Tufts dental records of patients who visited the periodontal department between 2020 and 2024. The average age of the participants was 47 years, with 45% being male. The prevalence of periodontal disease in this sample was 87% (95% CI: 79–93), while the prevalence of severe periodontal disease was 46% (95% CI: 36–56).

Diagnostic accuracy of radiographic analysis

The diagnostic accuracy of radiographic analysis using the Overjet AI software was evaluated against the gold (reference) standard and compared with manual radiographic diagnosis by a GP for detecting any, moderate to severe, and severe periodontal disease. Across all severity levels, the AI software significantly reduced diagnostic time, completing the assessment in 1–2 min, compared to approximately 10 min required for manual diagnosis by a GP.

For any periodontal disease (Table 1), the AI software showed a sensitivity of 100% (95% CI: 16–100), a specificity of 89% (95% CI: 81–94), a PPV of 15% (95% CI: 2–45), an NPV of 100% (95% CI: 96–100), and an overall accuracy of 89% (95% CI: 84–95). In contrast, the dentist’s diagnosis yielded a sensitivity of 100% (95% CI: 54–100), a specificity of 93% (95% CI: 86–97), a PPV of 46% (95% CI: 19–75), an NPV of 100% (95% CI: 96–100), and an overall accuracy of 93% (95% CI: 86–97). The Cohen’s Kappa test indicated a moderate level of agreement between the AI and dentist diagnoses (Kappa = 0.49).

Table 1.

Diagnostic accuracy of radiographic analysis by AI and by general practitioner for detecting any periodontal disease

Test	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy (95% CI)
Radiographic Analysis by AI	1.00 (0.16, 1.00)	0.89 (0.81, 0.94)	0.15 (0.02, 0.45)	1.00 (0.96, 1.00)	0.89 (0.82, 0.95)
Radiographic Analysis by Dentist	1.00 (0.54, 1.00)	0.93 (0.86, 0.97)	0.46 (0.19, 0.75)	1.00 (0.96, 1.00)	0.93 (0.86, 0.97)

CI: Confidence Interval, PPV: Positive Predictive Value, NPV: Negative Predictive Value

When detecting moderate to severe periodontal disease (Table 2), the AI software achieved a sensitivity of 82% (95% CI: 60–95), a specificity of 89% (95% CI: 80–95), a PPV of 67% (95% CI: 46–83), an NPV of 95% (95% CI: 87–99), and an overall accuracy of 87% (95% CI: 79–93). The dentist’s diagnosis showed slightly better performance, with a sensitivity of 90% (95% CI: 70–99), a specificity of 90% (95% CI: 82–96), a PPV of 70% (95% CI: 50–86), an NPV of 97% (95% CI: 91–100), and an overall accuracy of 90% (95% CI: 83–95). A Cohen’s Kappa test indicated an almost perfect agreement between the AI and the dentist’s results (Kappa = 0.85).

Table 2.

Diagnostic accuracy of radiographic analysis by AI and by general practitioner for detecting moderate to severe periodontal disease

Test	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy (95% CI)
Radiographic Analysis by AI	0.82 (0.60, 0.95)	0.89 (0.80, 0.95)	0.67 (0.46, 0.83)	0.95 (0.87, 0.99)	0.87 (0.79, 0.93)
Radiographic Analysis by Dentist	0.90 (0.70, 0.99)	0.90 (0.82, 0.96)	0.70 (0.50, 0.86)	0.97 (0.91, 1.00)	0.90 (0.83, 0.95)

CI: Confidence Interval, PPV: Positive Predictive Value, NPV: Negative Predictive Value

For severe periodontal disease (Table 3), the AI software demonstrated a sensitivity of 83% (95% CI: 71–92), a specificity of 96% (95% CI: 86–100), a PPV of 96% (95% CI: 86–100), an NPV of 84% (95% CI: 74–92), and an overall accuracy of 89% (95% CI: 82–95). The dentist’s diagnosis had a sensitivity of 83% (95% CI: 71–92), a specificity of 90% (95% CI: 80–96), a PPV of 88% (95% CI: 77–95), an NPV of 86% (95% CI: 76–93), and an overall accuracy of 86% (95% CI: 78–92). The Cohen’s Kappa test showed a substantial agreement between the AI and the dentist’s diagnoses (Kappa = 0.79).

Table 3.

Diagnostic accuracy of radiographic analysis by AI and by general practitioner for detecting severe periodontal disease

Test	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy (95% CI)
Radiographic Analysis by AI	0.83 (0.71, 0.92)	0.96 (0.86, 1.00)	0.96 (0.86, 1.00)	0.84 (0.72, 0.92)	0.89 (0.82, 0.95)
Radiographic Analysis by Dentist	0.83 (0.70, 0.92)	0.94 (0.83, 0.99)	0.94 (0.82, 0.99)	0.84 (0.72, 0.92)	0.88 (0.81, 0.94)

CI: Confidence Interval, PPV: Positive Predictive Value, NPV: Negative Predictive Value

Discussion

The findings from the radiographic analysis reveal that artificial intelligence demonstrates high diagnostic accuracy in detecting periodontal disease across varying levels of severity. Notably, the Overjet AI software showed excellent sensitivity (82–100%) and specificity (89–96%) across all three diagnostic categories (any, moderate, and severe periodontal disease). For detecting any periodontal disease, both AI and the dentist achieved perfect sensitivity (100%), but the AI had a lower PPV (15% vs. 46%), suggesting that while the AI identifies all true cases, it produces more false positives. In moderate and severe periodontal disease, the AI software maintained robust sensitivity (82–83%) and high specificity (89–96%), aligning closely with the dentist’s diagnostic performance. Importantly, the AI’s ability to identify severe periodontal disease demonstrated particularly strong predictive values, with a PPV of 96%, indicating a high likelihood that positive diagnoses reflect true cases.

A critical advantage of AI over manual diagnosis is the substantial reduction in diagnostic time. While a dentist required around 10 min to complete the radiographic assessment, the AI system achieved the same task in 1–2 min, representing a 90 − 80% reduction in diagnostic time without significant loss in accuracy. Cohen’s Kappa test results indicate moderate agreement between AI and dentist diagnoses in detecting any periodontal disease (Kappa = 0.49), and strong agreement in detecting moderate (Kappa = 0.85) and severe periodontal disease (Kappa = 0.79). These findings indicate that AI software performance was closely aligning with human clinicians, especially in detecting more advanced cases.

The diagnostic accuracy of radiographic analysis method for periodontal disease detection in this study showed that Overjet AI software achieved an accuracy of 89% in detecting severe periodontal disease, comparable to the 88% accuracy observed with a GP. This is consistent with recent study by Tariq et al. indicating that AI models can effectively detect RBL and classify disease severity, offering a faster and less clinician-intensive alternative to traditional methods [16]. However, another study by Revilla-Leon et al. has reported variable AI performance in detecting periodontal disease, with overall diagnostic accuracy ranging from 73.6% to 99% [8]. This variability is primarily attributed to differences in the specific AI algorithms used and the quality and type of radiographs analyzed (e.g., panoramic, bitewing, periapical, and CBCT) [8, 16].

A key strength of this study is the use of a comprehensive gold standard diagnostic method, which combined clinical and radiographic assessments conducted by an experienced periodontist with over 10 years of expertise in the field. This rigorous reference standard enhances the reliability of the validation process by minimizing diagnostic errors and providing a robust benchmark for evaluating the accuracy of clinical measures, AI-driven radiographic analysis, and manual radiographic assessment. Additionally, the study utilized multiple diagnostic approaches, allowing for a comprehensive comparison of the strengths and weaknesses of each method across varying severities of periodontal disease. This study is one of a very few studies assessed the validation of using AI in periodontal disease diagnosis.

However, the study has several limitations. First, the manual radiographic analysis was performed by a single GP, meaning the diagnostic outcomes reflect individual variability and may not represent broader clinical practice patterns. Involving multiple practitioners could have provided a more generalizable assessment and reduced the risk of bias. Second, the Overjet AI platform did not directly provide the percentage of RBL. Instead, RBL had to be estimated using average root lengths for each tooth, introducing potential measurement errors, and affecting the accuracy of AI-based diagnosis. Lastly, the study sample exhibited a high prevalence of periodontal disease (87%), leaving only 13% of participants without the condition. This imbalance can amplify the impact of minor diagnostic errors, where a single misclassification could substantially influence the calculated sensitivity, specificity, PPV, and NPV. Future studies with a more balanced sample composition and diverse clinical assessors may provide additional insights into the generalizability and accuracy of these diagnostic methods.

This study holds considerable implications for the application of artificial intelligence in diagnosing periodontal disease. The demonstrated accuracy and efficiency of AI, highlight its potential to restructure clinical workflows, enabling faster and more consistent detection of periodontal disease. By reducing the time required for diagnosis by 90 − 80%, the AI software can increase patients’ access to care, alleviate clinician workload, and enhance early detection and intervention. This time-saving advantage is particularly valuable in large-scale screening programs and in settings with limited resources where access to specialized care might be restricted.

Additionally, the study emphasizes the potential of AI to complement human expertise rather than replace it. While the AI exhibited a higher false-positive rate in less severe cases, its ability to detect severe periodontal disease with high accuracy suggests it could serve as a reliable triage tool for identifying patients who require urgent intervention. The findings also underscore the importance of integrating AI systems into existing diagnostic protocols to improve standardization and reduce diagnostic variability. As AI technology progresses, its adoption in dental practice is expected to elevate the accuracy, efficiency, and accessibility of periodontal disease detection, ultimately improving patient outcomes on a broader scale.

Conclusion

AI-based radiographic analysis offers a rapid and accurate alternative to manual dentist assessments, achieving a 90% reduction in diagnostic time while maintaining strong agreement with human diagnoses, particularly for detecting moderate to severe periodontal disease. These findings highlight the potential for integrating AI technology with traditional clinical methods to enhance efficiency and accuracy in periodontal disease detection and monitoring. While AI shows promising diagnostic performance, there is still room for improvement, particularly in enhancing its accuracy across all severity levels through continued advancements in machine learning algorithms.

Abbreviations

AAP

American Academy of Periodontology

AI

Artificial Intelligence

CAL

Clinical Attachment Loss

CDC

Centers for Disease Control and Prevention

CEJ

Cementoenamel Junction

CI

Confidence Interval

CVD

Cardiovascular Disease

GP

General Practitioner

NPV

Negative Predictive Value

PD

Probing Depth

PPV

Positive Predictive Value

RBL

Radiographic Bone Loss

Authors’ contributions

**AH** contributed to the conception, design, analysis, and interpretation of the data, as well as the drafting of the manuscript.**ZN** contributed to the design, analysis, and interpretation of the data, and substantively revised the manuscript.**AA** contributed to the data acquisition, analysis, and interpretation of the data, and revised the manuscript.**CH** contributed to the conception and design of the study and substantively revised the manuscript.All authors approved the final submitted version and agreed to be personally accountable for their own contributions and any part of the work.

Funding

The project was funded by KAU Endowment (WAQF) at king Abdulaziz University, Jeddah, Saudi Arabia. The authors, therefore, acknowledge with thanks WAQF and the Deanship of Scientific Research (DSR) for technical and financial support.This research received no funding.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

Institutional Review Board (IRB) exemptions were obtained from both Tufts University Health Sciences Institutional Review Board (HRP-210) and Harvard University Institutional Review Board (IRB23-1410) prior to initiating the study. The Tufts University IRB waived the requirement for informed consent, as the study involved secondary analysis of de-identified patient data. A Data Use Agreement (DUA24-0762) was obtained to ensure the lawful sharing, protection, and confidentiality of patient information. The study was conducted in accordance with the ethical principles of the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.