Table 1.
Characteristics of the studies included in the systematic review evaluating artificial intelligence-based conversational agents in orthodontics.
| Study | Orthodontic context and sample | AI chatbot(s)/version | Comparator/evaluators | Evaluation outcomes/main focus |
|---|---|---|---|---|
| Abu Arqub et al., 202412 | Clear aligner treatment; 111 questions | ChatGPT; version NR | 5 orthodontists/CAT experts; evidence-based rubric | Accuracy of AI-generated CAT claims; 4-point accuracy scale |
| Alkhamees, 20249 | Impacted canines, interceptive orthodontics, orthognathic surgery; 60 Q/A pairs | ChatGPT-4 | 5 experienced orthodontists | Accuracy and comprehensiveness; 5-point Likert |
| Arslan et al., 20245 | General orthodontic FAQs; 24 questions | ChatGPT; Google Bard | 1 orthodontist + 1 resident; model comparison | Accuracy/quality; modified Likert, ICC, word/reference counts |
| Daraqel et al., 20246 | General orthodontic questions; 100 questions in 10 domains | ChatGPT-3.5; Google Bard; accessed July 16, 2023 | 5 blinded orthodontic assessors | Accuracy, completeness, response time/length; AOI, VAS, ICC |
| Dursun and Bilici Gecer, 202414 | Clear aligner patient FAQs; 20 questions | ChatGPT-3.5; ChatGPT-4; Gemini; Copilot | 2 orthodontist evaluators | Accuracy, reliability, quality, readability; Likert, modified DISCERN, GQS, FRES |
| Hatia et al., 202416 | Interceptive orthodontics; 21 open questions + 7 clinical scenarios | ChatGPT-4; April 21, 2023 | 10 specialized orthodontists from postgraduate schools | Accuracy/completeness of answers and clinical scenarios; Likert scales |
| Kılınç & Mansız, 202413 | General and treatment-related FAQs; 34 questions | ChatGPT March 23 and May 24, 2023 versions | Version comparison; 2 orthodontists | Reliability and readability over time; DISCERN, Flesch-Kincaid, ICC |
| Kurt Demirsoy et al., 202410 | Clear aligners, lingual orthodontics, esthetic braces, TMD; 4 prompts | ChatGPT-4 | 35 orthodontists, 35 dental students, 35 patients | Reliability and quality; DISCERN and GQS |
| Makrygiannakis et al., 202415 | Clinically relevant orthodontic questions; 10 questions | ChatGPT-3.5; ChatGPT-4; Google Bard; Microsoft Bing | 2 specialist orthodontists; evidence-based answers | Comprehensiveness, scientific accuracy, clarity, relevance; 0-10 rubric |
| Metin & Goymen, 20257 | General orthodontic knowledge; 40 true/false questions | ChatGPT-4; ChatGPT-4o; Copilot; Gemini 1.5 Pro; Claude 3.5 Sonnet | 30 students, 30 dentists, 30 orthodontists; answer key | Accuracy and response consistency; percentage accuracy, Cronbach's alpha |
| Naureen & Kiani, 202423 | MARPE, clear aligners, CBCT; 90 questions | ChatGPT-4; Google Bard | 2 independent raters; standard answer key | Knowledge accuracy; 5-point scoring, ICC, paired tests |
| Salmanpour et al., 20258 | AAO patient-facing FAQs; 15 questions | ChatGPT-4.0; Microsoft Copilot; Oct 5-25, 2024 | Expert answers; 52 orthodontists + 102 patients | Adequacy of chatbot vs expert answers; 10-point Likert, Cronbach's alpha, Flesch-Kincaid |
| Tanaka et al., 202524 | Orthodontic treatment planning; 1 clinical case | ChatGPT-3.5; Bing Copilot; Google Bard/Gemini; Feb 2, 2024 | Expert orthodontist plan; 34 orthodontists | Agreement with AI vs human treatment plans; Likert, VAS, ANOVA |
| Vassis et al., 202425 | Pretreatment side-effect education; 28 prompts + 46 patient perceptions | ChatGPT-3.5; ChatGPT-4; Oct 2023 version | 3 experts + orthodontic patients; standard university text | Reliability, content validity, patient preference; Likert scales |
| Yurdakurban et al., 202426 | Orthognathic surgery patient information; 60 questions | ChatGPT-4; OpenEvidence; MediSearch | Researcher-developed guideline-based questions | Quality, reliability, readability, similarity; EQIP, DISCERN-adapted, GQS, SMOG |
| Zhou et al., 202511 | Orthodontic patient FAQs; 30 Chinese-language questions | ChatGPT-4; accessed Sept 16, 2023 | 2 orthodontists; 4 senior orthodontist raters | Ranking of AI vs orthodontists; Friedman, Nemenyi, Kendall W, Spearman |
Abbreviations: AAO, American Association of Orthodontists; AI, artificial intelligence; ANOVA, analysis of variance; AOI, Accuracy of Information; CA, clear aligner; CAT, clear aligner treatment; CBCT, cone-beam computed tomography; DISCERN, consumer health-information assessment tool; EB, esthetic braces; EQIP, Ensuring Quality Information for Patients; FAQ, frequently asked question; FRES, Flesch Reading Ease Score; GPT, Generative Pre-trained Transformer; GQS, Global Quality Scale; ICC, intraclass correlation coefficient; IQR, interquartile range; LLM, large language model; LO, lingual orthodontics; MARPE, mini-implant-assisted rapid palatal expansion; NR, not reported; Q/A, questions and answers; SD, standard deviation; SMOG, Simple Measure of Gobbledygook; TMD, temporomandibular disorders; VAS, visual analog scale.