Evaluation of Large Language Model‐Based Chatbots for Dental Trauma Management: A Comparative Study Based on Accuracy, Consistency and Information Quality

ABSTRACT

This study evaluated and compared the accuracy, consistency, readability, and information quality of three LLM‐based chatbots, namely ChatGPT‐5, Claude AI (Sonnet 4.0), and Perplexity (Mistral Large 2), in addressing traumatic dental injury questions. Forty true/false statements were submitted to each chatbot three times at weekly intervals to assess accuracy and consistency. Additionally, chatbot responses for 25 open‐ended case‐based questions were evaluated for readability, understandability and actionability, information reliability and quality. For true/false questions, Perplexity showed the highest accuracy, followed by Claude and ChatGPT. For open‐ended responses, ChatGPT excelled in readability (FRE: 62.4 ± 7.6), Perplexity in understandability (91.0 ± 4.3) and actionability (93.0 ± 6.4) and Claude in information reliability (mDISCERN total: 61.2; no variability observed). All chatbots achieved acceptable global quality scores (> 4.4). These findings emphasise the complementary role of chatbots in dental trauma management. Tool selection should be based on intended use, while continued human oversight remains essential in clinical decision‐making.

1. Introduction

Traumatic dental injuries (TDIs) are a common health concern with a worldwide prevalence of 18%–33%, predominantly during the first two decades of life [1, 2]. TDIs can cause functional and aesthetic problems if they are not appropriately managed. The treatment outcome is significantly affected by emergency intervention at the injury site as well as the knowledge and skills of the dentist. For TDIs, evidence‐based care depends on expert consensus and established treatment guidelines.

Studies demonstrate that TDIs predominantly take place at home or school. Consequently, in addition to dentists, parents, caregivers, and teachers should have basic knowledge of emergency dental trauma management, as timely intervention is as critical as the appropriate treatment [2, 3, 4]. Many patients or caregivers may seek health information online, primarily for emergency interventions or treatment options, due to their inability to attend or access dental appointments [5, 6].

Large language model (LLM)‐based chatbots are a type of artificial intelligence (AI) technology that trains computers to perform human‐like tasks like learning, reasoning and making decisions. LLM‐based chatbots utilise artificial intelligence techniques such as machine learning and natural language processing to mimic human conversation by generating immediate, human‐like responses [7, 8]. With recent developments in technology, the use of LLM‐based chatbots has increased in popularity to provide answers to complex topics, as well as health‐related issues. Chatbots can respond to public inquiries to provide information about prognosis, treatment, side effects and outcomes [9]. For these reasons, LLM‐based chatbots are considered potentially useful in dental trauma cases, where timely intervention is essential. Additionally, both educators and students are increasingly using LLM‐based chatbots to obtain medical information. Its application is particularly relevant in continuous education sectors, such as healthcare, for disseminating professional knowledge [10].

OpenAI Inc. (San Francisco, CA, USA) introduced ChatGPT (Chat Generative Pre‐Trained Transformer), the first LLM‐based chatbot, on November 30, 2022. It set a record by gaining 100 million new users in the first 3 months after launch [9]. ChatGPT generates human‐like responses to natural language inputs using deep learning techniques, and it employs a vast knowledge base to offer answers that are meaningful and contextually relevant. Claude, created by Anthropic (San Francisco, CA, USA), is a large language model focused on ethical considerations and safety. Lastly, Perplexity (Perplexity AI Inc., San Francisco, CA, USA), created in 2022 by Aravind Srinivas and his group, is a search engine driven by AI that provides natural language results.

Despite growing interest in using chatbots in medical research, there are also concerns about the reliability and accuracy of chatbot‐provided health‐related information. Although AI systems can provide rapid responses without navigating multiple websites, several limitations exist, including the possibility of inaccurate or incomplete information and inconsistent responses over time [6, 11].

Although some studies have been conducted to evaluate the performance of LLM‐based chatbots answering dental‐trauma‐related questions, the information remains limited about the current topic. More investigation is required to fully comprehend the range of responses that different chatbots produce, especially when it comes to medical applications where safety depends on consistent response content.

This study aimed to evaluate and compare the accuracy, validity, readability and consistency of answers provided by three LLM‐based chatbots (ChatGPT, Claude AI and Perplexity) for both professional and patient questions about traumatic dental injuries, specifically assessing their adherence to IADT guidelines and ability to address frequently asked questions. The study tested two main hypotheses: first, that there would be no significant differences in accuracy rates among the three LLM‐based chatbots for true/false questions related to TDI, with all chatbots demonstrating similar consistency; and second, that there would be no significant differences among the three LLM‐based chatbots in terms of readability, understandability, actionability, information reliability and global quality scores for open‐ended TDI questions.

2. Materials and Methods

Ethical approval was not required as no human participants were involved in the present study. The present study evaluated the accuracy, validity, readability and consistency of answers provided by ChatGPT (ChatGPT‐5), Claude AI (Claude Sonnet 4.0) and Perplexity (Mistral Large 2), both professional and patient questions about TDIs.

2.1. Design of Questions

This study was designed to evaluate chatbot performance using two distinct types of questions: structured true/false statements for dental professionals and open‐ended case scenarios for patients or caregivers. For the professional perspective, 40 true/false statements related to traumatic dental injuries (TDIs) were developed in accordance with the International Association of Dental Traumatology (IADT) guidelines, which define widely accepted evidence‐based management protocols [12, 13, 14, 15]. The statements were prepared and validated by two dental specialists, one in endodontics and one in restorative dentistry, each with more than 10 years of academic and clinical experience. For the patient perspective, 25 open‐ended case‐based questions were compiled from commonly asked queries on online platforms and social media. These were rephrased using an AI tool that was not included in the current comparison to achieve a standardised style while preserving the original intent. The same two specialists reviewed the final set of patient questions for clarity and representativeness.

2.2. Chatbot Processing

All questions were submitted to three LLM‐based chatbots—ChatGPT (ChatGPT‐5), Claude AI (Claude Sonnet 4.0), and Perplexity (Mistral Large 2)—by a single experienced user. The 40 true/false statements were asked three times at one‐week intervals (August–September 2025), allowing for repeated‐measures assessment of both accuracy and consistency. In contrast, the 25 open‐ended case‐based questions were asked only once, during the final week, since consistency testing was not applicable for open‐ended formats. Temporal consistency was not assessed for open‐ended responses because such answers inherently allow for multiple equally valid formulations, where variations in wording, structure, or explanatory detail do not necessarily indicate inconsistency in clinical meaning. Repeating open‐ended queries over time could therefore artificially inflate apparent variability without providing meaningful insight into response reliability. To avoid carry‐over effects, a new chat window was opened for every question, and browsing history and cookies were cleared before each session. For the true/false section, a standardised instruction was given before the statements, whereas the open‐ended case scenarios were posed directly to simulate a realistic patient‐chatbot interaction. The user submitting the questions did not have prior access to chatbot answers and only stored the responses for later evaluation, ensuring independence from chatbot outputs during data collection.

2.3. Evaluation Process

The collected responses were evaluated by two independent specialists who were blinded to chatbot identity, as all names and identifiers were removed prior to assessment. For the true/false questions, accuracy was determined by comparison with the reference answers derived from the IADT guidelines, and responses were coded as accurate or inaccurate. Consistency was defined as identical answers across the three sessions for each statement, while any variation was classified as inconsistent.

For the open‐ended questions, several complementary parameters were analysed. The Global Quality Score (GQS), a five‐point Likert‐type scale (1 = lowest quality, 5 = highest quality), was used to rate the overall validity and usefulness of the responses, where higher scores indicated superior quality of health information. Reliability of information was evaluated using the modified DISCERN (mDISCERN) instrument, which consists of 16 items rated on a five‐point Likert scale [16]. Total scores range from 16 to 80, with values ≤ 32 classified as poor, 33–63 as fair and ≥ 64 as good quality. In line with previous applications, the analysis was conducted both for the three subscales—DS1 (accuracy & credibility), DS2 (quality of information) and DS3 (professionalism and tone)—as well as for the total mDISCERN score (DT). Additionally, misinformation (MISINFO) was assessed using a five‐point Likert scale, where a score of 1 indicated completely inaccurate or misleading content, and a score of 5 represented fully accurate and reliable information consistent with accepted clinical guidelines.

Readability was assessed using the Flesch Reading Ease (FRE) and the Flesch–Kincaid Grade Level (FKGL). The FRE provides scores ranging from 0 to 100, with higher scores representing easier readability. The FKGL estimates the U.S. school grade level required to comprehend the text. Both indices were automatically calculated using Microsoft Word's built‐in readability tool. The FRE score was calculated as 206.835 minus 1.015 times the average sentence length (total words divided by total sentences), minus 84.6 times the average syllables per word (total syllables divided by total words). The FKGL was calculated as 0.39 times the average sentence length (total words divided by total sentences), plus 11.8 times the average syllables per word (total syllables divided by total words), minus 15.59. These coefficients are derived from the original formulas described by Flesch and Kincaid et al., which are widely adopted in readability research [17, 18]. Finally, understandability and actionability were examined with the Patient Education Materials Assessment Tool (PEMAT‐P). The PEMAT‐P includes two subscales: PEMAT‐U (understandability) and PEMAT‐A (actionability). Each item is rated dichotomously (agree/disagree), and the final scores are expressed as percentages, with higher scores reflecting better understandability and actionability.

2.4. Statistical Analysis

For true/false questions, differences in accuracy between the three LLM‐based chatbots (ChatGPT, Claude, Perplexity) were analysed using a generalized estimating equations (GEE) model to account for the correlation between repeated measurements on the same set of questions over 3 weeks. Correct/incorrect responses (0/1) served as the dependent variable, with chatbot type and week as independent variables. Odds ratios (OR) with 95% confidence intervals (CI) were calculated for pairwise comparisons.

To assess how consistently each chatbot responded over time, Fleiss' kappa (κ) was calculated for each chatbot based on its answers to the same 40 questions across the 3 weeks. This statistic reflects the level of agreement beyond chance. As a complementary measure, we also computed the percentage of questions with identical answers in all weeks (‘% identical Answers’) to provide a more intuitive indicator of stability.

All statistical analyses for open‐ended questions were performed using repeated‐measures approaches, considering that the same set of 25 questions was answered by each of the three LLM‐based chatbots. Descriptive statistics were calculated as mean ± standard deviation (SD). Overall group differences among the three chatbots were evaluated using the Friedman test. When significant differences were observed, pairwise comparisons were conducted with the Wilcoxon signed‐rank test and p‐values were adjusted using the Holm correction for multiple testing.

All statistical analyses were performed using R software (version 4.5.1; R Foundation for Statistical Computing, Vienna, Austria), and a p‐value < 0.05 was considered statistically significant.

3. Results

3.1. True/False Questions

The highest accuracy rates were observed in responses obtained from Perplexity across 3 weeks, followed by Claude and ChatGPT, respectively (Figure 1). Generalized estimating equations (GEE) analysis demonstrated statistically significant differences in accuracy among the three evaluated chatbots (Table 1). Perplexity achieved the highest accuracy, markedly outperforming ChatGPT. Specifically, the odds of ChatGPT providing a correct answer were approximately one‐fifth those of Perplexity (OR = 0.222; 95% CI: 0.072–0.685; p = 0.009). Claude also showed higher accuracy than ChatGPT, although this difference did not reach statistical significance (OR = 0.342; 95% CI: 0.107–1.091; p = 0.070), suggesting a potential but inconclusive advantage. Direct comparison between Claude and Perplexity revealed no significant difference in accuracy (OR = 0.649; 95% CI: 0.129–3.272; p = 0.601). In ranking, Perplexity demonstrated the highest accuracy, followed by Claude and ChatGPT.

FIGURE 1.

Distribution of accuracy rates of responses obtained from three LLM‐based chatbots across weeks.

TABLE 1.

Pairwise comparisons of chatbot accuracy using generalized estimating equations (GEE).

Comparison	Odds ratio (OR)	95% confidence interval (CI)	p‐value
ChatGPT vs. Claude	0.342	0.107–1.091	0.070
ChatGPT vs. Perplexity	0.222	0.072–0.685	0.009
Claude vs. Perplexity	0.649	0.129–3.272	0.601

Note: An OR (odds ratio) < 1 indicates that the first chatbot listed has a lower likelihood of producing a correct response compared to the second chatbot. A p‐value < 0.05 indicates a statistically significant difference.

Fleiss' kappa analysis indicated near‐perfect consistency for ChatGPT (κ = 0.946) and perfect consistency for Claude (κ = 1.000), corresponding to identical responses across the 3 weeks for 97.5% and 100% of the questions, respectively (Table 2). Perplexity demonstrated a κ of 0.298, equating to identical responses for 90.0% of the questions. Despite the high proportion of repeated answers, the lower kappa value for Perplexity reflects comparatively greater week‐to‐week variability in its responses.

TABLE 2.

Consistency of chatbot responses over 3 weeks, measured using Fleiss' kappa and the percentage of questions with identical answers across all weeks.

Chatbot	Fleiss' kappa (κ)	% Identical answers
ChatGPT	0.946	97.5%
Claude	1.000	100.0%
Perplexity	0.298	90.0%

Note: Fleiss' kappa values closer to 1 indicate stronger agreement across weeks. ‘% Same Answers’ refers to the percentage of questions for which the chatbot provided the same answer in all 3 weeks.

3.2. Open‐Ended Questions

A total of 25 open‐ended questions were evaluated across the three chatbots. Descriptive statistics and comparisons of readability, understandability, actionability are summarised in Table 3 while mDISCERN subscales and total scores are in Table 4. The FRE, FKGL, and total mDISCERN scores for each LLM‐based chatbot's responses are presented in Figure 2a–c using scatter plots.

TABLE 3.

Readability and patient education assessment scores across three LLM‐based chatbots.

Metric	ChatGPT (C1)	Claude (C2)	Perplexity (C3)	χ 2	p‐value	Pairwise comparison
FRE	62.4 ± 7.6	58.6 ± 9.0	59.2 ± 8.2	11.84	0.003**	C1 > C2 (p = 0.004), C1 > C3 (p = 0.005)
FKGL	8.5 ± 1.3	8.7 ± 1.9	8.5 ± 2.1	0.63	0.731 ns	ns
PEMAT‐U	88.7 ± 3.5	88.7 ± 5.4	91.0 ± 4.3	15.61	0.00041***	C3 > C1 (p = 0.0004), C3 > C2 (p = 0.0051)
PEMAT‐A	85.0 ± 4.8	88.2 ± 8.9	93.0 ± 6.4	26.90	0.0001***	C2 > C1 (p = 0.0127), C3 > C1 (p < 0.0001), C3 > C2 (p = 0.0052)

Note: Data are presented as mean ± standard deviation (SD). Statistical analysis was performed using the Friedman test for repeated measures, followed by pairwise Wilcoxon signed‐rank tests with Holm correction. A p‐value < 0.05 was considered statistically significant.

Abbreviations: FKGL, Flesch–Kincaid Grade Level; FRE, Flesch Reading Ease; ns, not significant; PEMAT‐A, actionability; PEMAT‐U, Patient Education Materials Assessment Tool—Understandability.

^*** p < 0.001.

^** p < 0.01.

TABLE 4.

mDISCERN subscale and total scores across three LLM‐based chatbots.

Metric	ChatGPT (C1)	Claude (C2)	Perplexity (C3)	χ 2	p‐value	Pairwise comparison
DS‐1	16.3 ± 4.8	17.3 ± 8.9	17.6 ± 6.4	17.28	0.0002***	C3 > C1 (p = 0.0049), C2 > C1 (p = 0.047)
DS‐2	37.1 ± 1.9	38.9 ± 0.8	36.6 ± 1.0	14.08	0.00087***	C2 > C1 (p = 0.034), C2 > C3 (p = 0.0031)
DS‐3	5.00 ± 3.4	5.00 ± 3.9	4.96 ± 5.7	2.00	0.368 ns	ns
mDISCERN‐T	58.4 ± 0.0	61.2 ± 0.0	59.2 ± 0.2	13.14	0.0014**	C2 > C1 (p = 0.033), C2 > C3 (p = 0.034)

Note: Data are presented as mean ± standard deviation (SD). Statistical analysis was performed using the Friedman test for repeated measures, followed by pairwise Wilcoxon signed‐rank tests with Holm correction. A p‐value < 0.05 was considered statistically significant.

Abbreviations: DS, mDISCERN subscales; DT, mDISCERN total score; ns, not significant.

^*** p < 0.001.

^** p < 0.01.

FIGURE 2.

(a) Scatter plots with median lines displaying FRE scores across the three LLM‐ based chatbots, (b) Scatter plots with median lines displaying FKGL scores across the three LLM‐ based chatbots, (c) Scatter plots with median lines displaying mDISCERN total scores across the three LLM‐ based chatbots.

Regarding readability, ChatGPT achieved significantly higher FRE scores compared with Claude and Perplexity (p < 0.01), whereas no differences were observed in FKGL. For understandability and actionability (PEMAT‐P), Perplexity demonstrated the highest performance. The PEMAT‐U and PEMAT‐A scores for a total of 25 responses provided by each LLM‐based chatbot are presented using a scatter plot in Figure 3a,b. Specifically, Perplexity scored significantly higher than ChatGPT and Claude in understandability (p < 0.01) and outperformed both in actionability as well (p < 0.001).

FIGURE 3.

(a) Scatter plots with median lines displaying PEMAT‐U scores across the three LLM‐ based chatbots (b) Scatter plots with median lines displaying PEMAT‐A scores across the three LLM‐ based chatbots.

Within the mDISCERN assessment, Claude obtained the highest mean scores for DS2 and the total score (DT), showing significant advantages over the other chatbots (p < 0.01). For DS1, both Claude and Perplexity scored marginally higher than ChatGPT, while no differences were found in DS3.

The distribution of GQS and MISINFO scores regarding the different LLM‐based chatbots are presented in Figure 4. No significant differences were observed among the chatbots in terms of GQS. Although Friedman testing indicated an overall significant difference for misinformation (p = 0.018), pairwise comparisons did not reveal statistically significant differences between individual chatbot pairs after correction.

FIGURE 4.

(a) Distribution of GQS of responses obtained from three LLM‐ based chatbots, (b) Distribution of MISINFO scores of responses obtained from three LLM‐ based chatbots.

4. Discussion

This study aimed to evaluate and compare three LLM‐based chatbots (ChatGPT‐5, Claude‐Sonnet 4.0, and Perplexity‐Mistral Large 2) in terms of accuracy, validity, readability, and consistency when addressing both professional and patient questions about traumatic dental injuries. Specifically, the chatbots' compliance with IADT guidelines and their ability to address frequently asked questions were evaluated. The first hypothesis, which predicted no significant difference in accuracy rates among the three LLM‐based chatbots for true/false questions related to TDI, was rejected. Our findings revealed significant differences, with Perplexity showing superior accuracy compared to ChatGPT (OR = 0.222; 95% CI: 0.072–0.685; p = 0.009). In terms of consistency, the hypothesis was partially supported, with ChatGPT and Claude showing near‐perfect to perfect consistency (κ = 0.946 and κ = 1.000, respectively), while Perplexity exhibited lower consistency (κ = 0.298). The second hypothesis, which predicted that there would be no significant difference between the three LLM‐based chatbots in terms of readability, comprehensibility, actionability, information reliability, and overall quality scores in open‐ended TDI questions, was also rejected. Significant differences emerged across multiple parameters, with each chatbot demonstrating different strengths: ChatGPT excelled in readability (FRE: 62.4 ± 7.6), Perplexity showed superior performance in comprehensibility and actionability (PEMAT‐U: 91.0 ± 4.3; PEMAT‐A: 93.0 ± 6.4), while Claude achieved the highest mDISCERN scores (DT: 61.2 ± 0.0).

The highest overall accuracy rates were observed with Perplexity (95% mean value) compared with Claude (92.5% mean value) and ChatGPT (80.3% mean value). These findings are contrary to previous findings that presented higher accuracy rates in ChatGPT than Perplexity [11]. Although the question types (true/false) were similar in both studies, in the present study the questions were prepared to include all IADT guidelines related to the management of traumatic dental injuries, while in the other study the main inquiry was limited to the management of avulsed teeth only. It has been pointed out that AI accuracy rates above 90% represent acceptable performance for diagnostic tasks in endodontics [19]. Based on this, it might be stated that Perplexity and Claude are more reliable in the clinical decision‐making process. However, an interesting finding has emerged regarding the relationship between accuracy and consistency among the three chatbots. While Perplexity exhibits high accuracy but low consistency, ChatGPT and Claude demonstrate high consistency at varying levels of accuracy. This pattern suggests different underlying mechanisms for information access and processing among chatbots [20]. The balance between accuracy and consistency has important implications for clinical applications such as emergency dental trauma management, where both reliable and factually correct responses are critically important. IADT guidelines emphasise the essentiality of rapid and accurate emergency intervention for optimal outcomes in traumatic dental injuries [12, 13, 14]. This finding underscores the need to consider both dimensions when evaluating AI tools for clinical applications, as variability in responses may impact decision‐making in time‐sensitive situations. This discrepancy can be explained by the different aspects of agreement captured by the applied metrics. Fleiss' kappa quantifies agreement beyond chance and is sensitive to even minor response variations, such as small wording changes, additional explanatory details, or reference citations, which may not affect the overall correctness of the response but can substantially reduce kappa values. In contrast, percentage agreement provides a more intuitive measure of repeated identical outputs without accounting for chance agreement.

All three chatbots demonstrated acceptable overall information quality with global quality scores above 4.4. However, the variation in the mDISCERN subscales highlights the importance of a multidimensional assessment of information reliability. Claude's performance on DS2 (treatment information quality) and total mDISCERN scores demonstrates strong alignment with clinical guidelines, which is essential for maintaining treatment standards in dental traumatology [16]. Developed as a standardised method for evaluating written consumer health information, the mDISCERN tool provides a systematic assessment of treatment‐related information quality. Claude's high mDISCERN scores suggest strong alignment with evidence‐based treatment protocols and indicate its potential value for professional consultation and educational applications [16]. Perplexity's high comprehensibility and actionability scores demonstrate its potential to provide clear and actionable guidance in emergency situations. These findings are particularly important given the increasing integration of AI into healthcare education, as they indicate that comprehensive training is required for healthcare professionals to effectively use AI systems by understanding their capabilities and limitations [21].

Considering that dental trauma affects an estimated 50%–60% of the world's population and that there are approximately 60 million new patients annually, the provision of accessible information becomes a critical issue [22]. The significantly higher FRE scores obtained with ChatGPT, compared with Claude and Perplexity, demonstrated its potential accessibility for patients and caregivers with different educational backgrounds during dental trauma emergencies, where clear communication is crucial. The present findings were also aligned with a previous study indicating a required educational level above 8th grade based on FKGL scores [6]. The American Medical Association recommends that patient‐focused content should be below a sixth grade reading level, but the current results exceeded this recommendation [23]. However, it should be noted that when users ask inquiries that contain technical phrases, the AI chatbot adjusts its linguistic structure and replies with equivalent technical terms. Additionally, the readability of responses can be increased by additional prompts asking for simplifying the language to the appropriate educational level [24]. The comprehensibility and actionability performance of Perplexity, as measured by PEMAT, is consistent with its design as a search engine that provides contextual responses. This feature may be valuable in dental trauma scenarios where clear and actionable guidance is essential, considering that the PEMAT assessment tool specifically measures patients' likelihood of understanding the information provided and knowing how to act [25].

The results of the present study indicated that different LLM‐based chatbots can play complementary roles in dental trauma management based on their unique performance characteristics. Each chatbot demonstrated specific strengths that could be utilised in different contexts: readability for patient communication, evidence‐based compliance for professional guidance, and actionability for emergency situations. This complementary approach can improve the overall quality of emergency dental trauma care by appropriately utilising each tool's specific strengths. The critical nature of timing in dental trauma management cannot be overstated. The 2020 IADT guidelines emphasise that the primary goal is to establish an approach to immediate or emergency care for traumatic dental injuries, as appropriate emergency intervention can significantly affect long‐term outcomes [12, 13, 14, 15]. In this context, the consistency of information presentation becomes as important as accuracy, as conflicting advice in emergency situations can lead to delayed or inappropriate treatment decisions. In real‐world settings, inaccurate or inconsistent information provided by chatbots during emergency dental trauma situations may directly influence patient or caregiver behaviour, potentially leading to delayed presentation, inappropriate self‐management, or deviation from evidence‐based guidelines. Such risks are particularly critical in traumatic dental injuries, where early and correct intervention has a decisive impact on prognosis. These findings highlight the importance of cautious integration of chatbot‐based information into clinical decision‐making processes and reinforce the necessity of professional oversight, especially in time‐sensitive emergency contexts.

The findings of this study have important implications for health professional education and training programs. As AI applications continue to expand in healthcare, there is a growing need for comprehensive training programs that help healthcare professionals understand both the capabilities and limitations of these tools [21]. The different performance profiles of the chatbots examined highlight the importance of understanding which AI tools are most suitable for specific clinical contexts and educational objectives. Furthermore, as these technologies become more prevalent in clinical practice, the integration of AI literacy into medical and dental curricula is becoming increasingly important. Educational institutions should develop structured approaches to teach future healthcare professionals how to effectively use AI tools while maintaining critical thinking skills and clinical judgment.

This study has several limitations that must be acknowledged. The evaluation was limited to English responses and did not consider cultural variations in dental trauma management. Furthermore, the study focused on text‐based responses without evaluating multimedia capabilities that could enhance patient understanding. The evaluation was conducted at specific points in time and may not reflect the continuous evolution of these AI systems. Future research should investigate the performance of chatbots across different languages and cultural contexts, evaluate their ability to handle complex, multi‐variable clinical scenarios, and assess their integration with existing dental emergency protocols. Additionally, longitudinal studies examining the evolution of chatbot accuracy and consistency over time will provide valuable insights into their reliability for clinical applications. Research into developing hybrid systems that combine the strengths of different chatbots may also be beneficial. Furthermore, studies examining the real‐world impact of chatbot recommendations on treatment outcomes and patient satisfaction will provide important evidence for their clinical utility.

This comparative evaluation of three LLM‐based chatbots has revealed different performance profiles in handling dental trauma‐related queries. While no chatbot demonstrated superiority across all evaluated parameters, each exhibited specific strengths that could be utilised in different clinical contexts. Meaningful differences in accuracy, consistency, readability, and information reliability between ChatGPT, Claude AI, and Perplexity underscore the importance of careful selection based on intended use. These findings provide evidence‐based guidance for healthcare providers, educators, and patients in selecting appropriate AI tools for dental trauma knowledge and emergency management, while underscoring the necessity of continuous human oversight in clinical decision‐making. As AI technologies continue to evolve, continuous evaluation and validation will be essential to ensure the safe and effective integration of these tools into clinical practice and healthcare education.

Author Contributions

Vasfiye Isik and Soner Sismanoglu: conception. Soner Sismanoglu: design. Rana Ikbal Sengul: data collection. Soner Sismanoglu: analysis/interpretation. Rana Ikbal Sengul and Vasfiye Isik: literature research. Vasfiye Isik and Soner Sismanoglu: writing manuscript. Soner Sismanoglu: critical review.

Ethics Statement

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author.