Caudal Anesthesia Patient Education Materials: Comprehensive Analysis of Generative Artificial Intelligence and Traditional Patient Information Leaflets

1.

Effective communication is central to high‐quality pediatric anesthetic care, particularly for regional techniques such as caudal anesthesia, where parental understanding directly influences perioperative outcomes. Preoperative education helps explain the rationale, benefits, and risks of the procedure while fostering trust [1]. Evidence suggests that structured counseling reduces parental anxiety, which is closely linked to adverse pediatric outcomes including emergence agitation and delayed recovery [2]. Despite its importance, effective communication is often limited by time constraints and variable health literacy, especially in high‐volume centers in lower‐middle‐income settings. As a result, parents frequently rely on online information, which is often inconsistent in quality. Traditional institutional leaflets remain reliable but are inherently static and lack personalization [3]. The emergence of publicly available generative artificial intelligence (AI) tools, such as ChatGPT, Gemini, and DeepSeek, offers a potential shift toward more adaptive and accessible patient education, although their accuracy and clinical reliability require validation.

We conducted a comparative observational study evaluating patient information leaflets (PILs) generated by ChatGPT 4.0 (OpenAI), Gemini 2.5 Flash (Google), and DeepSeek‐V3 against a standard National Health Service (NHS) leaflet. AI‐generated PILs were created using a uniform prompt: “Generate a detailed patient information leaflet about caudal anesthesia in children for parents or guardians.” All materials were anonymized, standardized in format, and assigned random codes (details in File S1).

Evaluation was based on readability, emotional tone, factual accuracy, understandability, and actionability. Readability was assessed using Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKGL). Sentiment analysis evaluated tone, while factual accuracy was scored independently by 10 pediatric anesthetists (≥ 10 years' experience) using a 10‐point Likert scale. The Patient Education Materials Assessment Tool (PEMAT) was used for understandability and actionability. Statistical analysis was performed using ANOVA with post hoc Tukey's HSD (α = 0.05) (details in File S2).

AI‐generated leaflets demonstrated improved readability compared to the NHS leaflet, with DeepSeek showing the lowest FKGL and highest FRE scores. Sentiment analysis revealed variation in tone: ChatGPT produced a neutral‐to‐positive tone (+10.2), whereas the NHS leaflet had a more negative or serious tone (−34). PEMAT scores differed significantly for understandability (p < 0.001) and actionability (p = 0.013). ChatGPT performed comparably to NHS in understandability and was superior to Gemini, while DeepSeek showed consistent performance across domains. Expert‐rated accuracy did not differ significantly between groups, although the NHS leaflet achieved the highest mean score, followed by DeepSeek, ChatGPT, and Gemini (Table 1).

TABLE 1.

Comparative analysis of NHS (human made) vs. Gen‐AI PEMs (ChatGPT, Gemini, DeepSeek).

	NHS	ChatGPT	Gemini	DeepSeek
Accuracy (mean ± SD) (p > 0.05)	9.80 ± 0.42	9.2 ± 0.79	9.2 ± 0.63	9.6 ± 0.51
Readability
FRE	57	57.2	44.3	61.4
FKGL	9.10	7.40	10.20	7.00
Sentiment analysis	−34	+10.2	−3.4	−25.6
PEMAT
Understandabilty* *Post hoc with Tukey HSD analysis (p < 0.05) [NHS vs. Gemini, ChatGPT vs. Gemini, Gemini vs. DeepSeek]	93.2	93	88	91.67
Actionability# #Post hoc with Tukey HSD analysis (p < 0.05) [ChatGPT vs. Gemini, Gemini vs. DeepSeek]	86.9	84.67	89.67	90.33

Note: Accuracy scores given by 10 experts expressed as mean ± SD range of Likert 1–10; readability scores measured by FRE range [1–100, > 60 easy readability] FKGL reflects US School Grades < 6–8 is easy readability, recommended < 5; Sentiment analysis range [<−25 negative/serious tone, −25 to+25 neutral and >+25 positive/enthusiastic tone]; PEMAT—patient education material assessment tool by AHRQ—agency for healthcare research and quality [> 80% good understandability and actionability].

The differences observed across models likely reflect their underlying design and purpose. Generative AI systems are trained on diverse language datasets optimized for clarity and engagement, which may explain their superior readability and more accessible tone [4]. In contrast, traditional leaflets are developed within a framework that prioritizes clinical precision, standardization, and medico‐legal safety, often resulting in more formal language and reduced readability. The comparable accuracy across groups suggests that AI tools can generate clinically acceptable information when guided by appropriate prompts; however, their slightly lower expert scores may reflect the absence of domain‐specific validation and a tendency toward generalization [5, 6]. Higher actionability in AI‐generated materials may be attributed to their structured, stepwise style of information delivery, which aligns with caregiver needs. Generative AI systems, while powerful for drafting and scaling information, remain susceptible to probabilistic errors and hallucinations, necessitating mandatory clinician oversight [7, 8]. These findings support the concept that while AI enhances accessibility and engagement, traditional resources retain an advantage in reliability and consistency.

Our findings also align with social referencing theory, where parental emotional state influences the child's perioperative experience. The more neutral or reassuring tone observed in AI‐generated outputs may therefore have practical implications in reducing caregiver anxiety. However, an overly optimistic tone without appropriate risk framing may compromise informed consent. This highlights the importance of balancing empathy with accuracy in patient education.

This study has limitations. Only a limited number of widely available AI models were assessed using a single standardized prompt, which may not reflect real‐world iterative interactions. Inter‐rater reliability was not formally measured, and patient or caregivers' perspectives were not included. Despite these limitations, the study provides a structured comparison using validated assessment tools.

In conclusion, AI‐generated patient education materials demonstrate promising improvements in readability, clarity, and usability while maintaining acceptable levels of accuracy. A hybrid approach, combining the adaptability of generative AI with clinician oversight, may offer the most effective strategy for delivering accurate, accessible, and contextually appropriate patient education, particularly in resource‐constrained settings. Future studies should focus on real‐world patient outcomes, including comprehension, anxiety reduction, and decision‐making.

Supporting information

File S1: Source compilation of standard and AI‐generated patient information leaflets for pediatric caudal anesthesia. This annexure compiles the complete text of the four evaluated patient leaflets, comprising the institutional standard (NHS) and three large language model generations (ChatGPT, Google Gemini, and DeepSeek).

File S2: Methodological framework and evaluation tools for patient information leaflet assessment. This document provides the comprehensive criteria, mathematical formulas, and scoring interpretations for the Flesch Reading Ease (FRE), Flesch–Kincaid Grade Level (FKGL), patient education materials assessment tool (PEMAT‐P), sentiment analysis, and expert Likert scale metrics.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.