Large Language Model–Generated Patient Education Materials in Pediatric Ophthalmology: A Scoping Review

Abstract

Effective management of pediatric ophthalmic conditions such as myopia, amblyopia, and strabismus depend on caregivers understanding patient education materials (PEMs). However, many existing resources exceed recommended grade 8 reading levels. This review maps AI-generated PEMs in pediatric ophthalmology, summarizes reported outcomes across key educational domains, and identifies barriers and opportunities for clinical integration. Following PRISMA-ScR guidelines, a search of Ovid MEDLINE, Embase, and Scopus (January 2015–July 2025) identified studies evaluating LLM-generated PEMs in pediatric ophthalmology. Eligible studies evaluated artificial intelligence for pediatric ophthalmology patient education and reported at least one educational outcome. Data were extracted independently by two reviewers and synthesized descriptively. Twenty studies (2023–2025) evaluated 3286 AI-generated materials across 14 LLMs. Readability was assessed in 17 studies; 59% (37/63) of study-level LLM evaluations exceeded recommended 8th-grade reading levels. ChatGPT-4o evaluations were more often acceptable (52%) than ChatGPT-3.5 evaluations (18%), although these were not head-to-head comparisons. Median understandability (PEMAT-U) exceeded adequacy thresholds (≥70%). Professional society materials were more accurate and higher quality but significantly less readable. No included study explicitly classified outputs as hallucinations under its applied assessment framework; however, incomplete treatment-related content was common. Only two studies assessed patient or caregiver perspectives. LLM-generated PEMs in pediatric ophthalmology are generally understandable and accessible but often exceed readability standards and lack actionable guidance. While professional society materials remain more accurate, LLMs offer scalable tools for generating patient-centered resources. Integrating LLMs into clinician-supervised workflows and emphasizing co-designed, multimodal education strategies may enhance accessibility and clinical safety.

Introduction

Pediatric ophthalmic diseases such as myopia, strabismus, and ptosis are highly prevalent and can profoundly affect visual development and long-term health outcomes in children.¹ Effective management during critical developmental periods depends not only on timely diagnosis and treatment but also on caregiver understanding and adherence to recommended therapies.² Inadequate comprehension of patient education materials (PEMs) has been directly linked to poor adherence and suboptimal visual outcomes in conditions such as amblyopia and myopia control, positioning patient education as a cornerstone of pediatric ophthalmic care.³ Because education in pediatric ophthalmology is frequently directed at caregivers rather than at children themselves, caregiver-facing materials may carry different health literacy requirements than patient-facing materials, as caregivers must interpret information on the child’s behalf and translate it into actions such as administering treatment. In this review, PEMs refer to both patient-facing and caregiver-facing materials.

However, existing PEMs often fail to meet caregivers’ needs. Many are written at reading levels exceeding the recommended sixth- to eighth-grade thresholds, available in limited languages, and insufficiently tailored to diverse literacy backgrounds.^4–6 These shortcomings contribute to health literacy gaps, caregiver anxiety, and inconsistent treatment adherence, ultimately undermining clinical effectiveness.⁵

Artificial intelligence (AI), particularly large language models (LLMs), has emerged as a potential solution. LLMs can generate natural language outputs that may improve accessibility and readability of PEMs while offering scalable personalization.⁴ Early applications suggest LLM-generated PEMs can enhance clarity and engagement compared with traditional resources, though concerns remain regarding accuracy, clinical nuance, and validation.^4,7

In pediatric ophthalmology, LLMs have been applied to conditions including cataract, amblyopia, strabismus, and myopia, with preliminary studies suggesting improvements in readability and accessibility.⁴ However, evidence remains fragmented, with inconsistent reporting of outcomes across readability, accuracy, quality, and actionability domains. Ethical and practical considerations, including data privacy, algorithmic bias, and integration into clinical workflows, further underscore the need for systematic evaluation.^4,7

To address this gap, we conducted a scoping review to systematically map the existing evidence on LLM-generated PEMs in pediatric ophthalmology. A scoping review, rather than a systematic review, was selected because the existing literature is heterogeneous in study design, LLMs evaluated, prompts, clinical conditions, and outcome instruments, making it better suited to mapping the breadth and nature of the evidence than to pooling results or answering a narrowly defined question.⁸ Furthermore, LLM technology evolves rapidly, with new model versions released frequently, which necessitates periodic mapping of the evidence to keep pace with the tools available to patients and caregivers. Our objectives were to (1) describe the scope of existing studies and LLM applications, and (2) synthesize reported outcomes across readability, accuracy, quality, and actionability domains, including comparisons between LLM-generated and professionally authored PEMs. By charting the current landscape, this review aims to inform the development of reliable, patient-centered resources that leverage AI innovation alongside expert-authored content.

Methods

Reporting Guidelines

This scoping review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) to ensure methodological rigor and transparency (Supplementary Table 1).⁸ The review relied on publicly available data and thus, exempt from research ethics board approval.

Search Formulation

A comprehensive search strategy was developed in consultation with a medical librarian (LD) and executed on July 23, 2025, in Ovid MEDLINE, Ovid Embase, and Scopus. Embase and Scopus were included to ensure coverage of international and non-English literature. The search strategy incorporated controlled vocabulary and keywords spanning four domains: artificial intelligence, pediatrics, ophthalmology, and patient education. Searches were limited to publications from January 1, 2015 to July 23, 2025. Although LLM-based tools such as ChatGPT became widely available only in late 2022, this earlier start date was chosen to ensure that AI-based patient education tools preceding LLMs, such as earlier chatbots, were not missed. No language restrictions were applied. Full search strategies for each database are provided in Supplementary Table 2. In addition to the bibliographic databases, a grey literature search was conducted in which the first 200 results from Google Scholar were screened for eligibility.

Screening

Results were deduplicated and imported into Covidence.⁹ Two reviewers (AM, TA) independently screened titles, abstracts, and full texts per PRISMA-ScR guidelines, resolving discrepancies through discussion. No formal calibration exercise was conducted before title and abstract screening. Inter-rater agreement was high for both title and abstract screening (Cohen’s κ = 0.98) and full-text review (κ = 1.00).

Studies were eligible if they evaluated AI in pediatric ophthalmology patient education and reported at least one patient education outcome. Exclusion criteria included commentaries, editorials, and conference abstracts without full text, studies not specific to pediatric ophthalmology, and animal or preclinical studies. Brief communications and letters were included if they contained original data. Reference lists of included studies were reviewed for additional eligible articles.

Data Extraction

Data extraction was performed independently by two reviewers (AM, TA) using a standardized template, with consensus reached through discussion. Extracted variables included study characteristics (authors, year, country, design), AI tools evaluated, comparators, outcomes assessed, and key findings. For each outcome, the corresponding measurement instrument was also extracted, such as readability formulas or validated tools for quality and understandability.

Readability Analysis

Readability outcomes were heterogeneously reported across studies using validated English and non-English indices, with thresholds summarized in Supplementary Table 3. To enable cross-study comparison, each LLM’s overall performance within a study was classified into standardized interpretive bands commonly recommended in the health literacy literature: Easy (≤6^th grade), Acceptable (7th–8^th grade), and Difficult (≥9^th grade) (Table 1).^4,7,10,11 Decimal values were rounded to the nearest whole grade prior to classification.

Table 1.

Grade-Level Interpretive Bands for Readability Indices in Included Studies

Index/Formula (Language)	Easy (≤6th Grade)	Acceptable (7th–8th Grade)	Difficult (≥9th Grade)
Flesch–Kincaid Grade Level (FKGL) (English)	≤6	7–8	≥9
Simple Measure of Gobbledygook (SMOG) (English)	≤6	7–8	≥9
Coleman–Liau Index (CLI) (English)	≤6	7–8	≥9
Gunning Fog Index (GFI) (English)	≤6	7–8	≥9
Flesch Reading Ease (FRE) (English)	≥80%	60%–79%	≤59%
Ateşman Readability Formula (Turkish)	≥71	41–70	≤40
Bezirci-Yılmaz Readability Formula (Turkish)	1–5	6–9	≥10
Chinese Readability Index Explorer (CRIE) (Chinese)	≤6	7–8	≥9
Dale–Chall Readability Formula (English)	≤4.9	5.0–5.9	≥6.0

Each LLM was assessed separately within each study. For studies reporting multiple readability indices per LLM, a majority rule approach was applied to assign an overall band. In cases of discordance, the disagreement was noted, and all differing bands were recorded. This approach ensured that variability across indices was transparently captured while maintaining consistency in band assignment.¹² Each study-level LLM evaluation was counted once, regardless of the number of materials generated in the underlying study.

Data Synthesis

Extracted outcomes were summarized descriptively. When available, raw values were reported; otherwise, medians were prioritized over means to better reflect score distributions. No inferential statistical analyses or meta-analyses were performed. Results were organized in tables and supported by narrative synthesis, with graphical representations included where appropriate to illustrate trends across AI tools, comparators, and outcome domains. Consistent with PRISMA-ScR guidance, formal critical appraisal of individual studies was not performed, as the objective of this scoping review was to map the nature and extent of the available evidence rather than to assess methodological quality or risk of bias.⁸

Results

Study Selection

The database search yielded 195 records (Embase, n=97; Scopus, n=56; MEDLINE, n=42), of which 121 remained for title and abstract screening after removal of 74 duplicates. Ninety-two were excluded at this stage, leaving 29 articles for full-text review. Of these, 10 were excluded (wrong patient population, n=5; wrong study design, n=3; wrong outcomes, n=2), and 19 met the inclusion criteria. Screening of the first 200 Google Scholar results identified no additional eligible studies, and reference tracking identified one additional study, yielding 20 included studies (Figure 1).

Figure 1.

PRISMA flow diagram of study selection. Flow of records through identification, screening, full-text eligibility assessment, and inclusion, including records identified through the database search, the grey literature search (Google Scholar), and citation searching.

Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Study Characteristics

A total of 20 studies published between 2023 and 2025 were included, predominantly cross-sectional in design (Table 2). One included study was published online ahead of print on May 27, 2025, before the search date, and is cited by its final issue date (November 2025).³² Across all studies, 3286 AI-generated materials were evaluated, assessing 14 different LLMs, with ChatGPT-3.5 and ChatGPT-4o most frequently evaluated.

Table 2.

Characteristics of Included Studies Evaluating LLM-Generated Patient Education Materials

Study	Country	Study Design	Number of LLM Generated PEMs	LLM Model	Evaluators	Language	Pediatric Ophthalmology Topic(s)	Outcomes Measured (Tools)
Rasmussen et al13	Denmark	Brief Communication	125	ChatGPT-3.5	Two VKC-specialist ophthalmologists	English	VKC	Accuracy (Likert scale
Lim et al14	Singapore & China	Cross-sectional	93	ChatGPT-3.5, ChatGPT-4o, Bard	Three pediatric ophthalmologists	English	Myopia	Accuracy, Comprehensiveness (Likert scale)
Chang et al15	Taiwan	Cross-sectional	60	ChatGPT-3.5	10 Ophthalmologists	Chinese	Myopia	Readability (CRIE), Appropriateness, Usability, Clarity (Likert scale)
Dihan et al16	US & Egypt	Cross-sectional	120	ChatGPT-3.5, ChatGPT-4o, Bard	Three pediatric ophthalmologists	English	Pediatric cataract	Quality (DISCERN), Understandability/Actionability (PEMAT-U/A), Accuracy (Likert), Readability (SMOG, FKGL)
Dihan et al17	US & Egypt	Cross-sectional	180	ChatGPT-3.5, ChatGPT-4o, Bard	Three pediatric ophthalmologists	English	Childhood glaucoma	Quality (DISCERN), Understandability/Actionability (PEMAT-U/A), Accuracy (Likert), Readability (SMOG, FKGL)
Dizdar Yigit et al18	Turkey & Austria	Cross-sectional	30	ChatGPT-3.5	One strabismus specialist	English	Strabismus	Appropriateness (Expert evaluation), Readability (FRE, FKGL, Gunning Fog, CLI, SMOG)
Doğan et al19	Turkey	Cross-sectional	75	ChatGPT-3.5, Bard, Bing Chat	Two pediatric ophthalmologists	English	Amblyopia	Appropriateness (Expert evaluation), Understandability/Actionability (PEMAT-U/A), Readability (FRE, FKGL, CLI)
Yılmaz et al20	Turkey	Cross-sectional	66	ChatGPT-3.5, Bard, Copilot	Two ophthalmologists	English	Strabismus	Accuracy (SOLO taxonomy), Understandability/Actionability (PEMAT-U/A), Readability (FKGL)
Nikdel et al21	US & Iran	Cross-sectional	110	ChatGPT-4o	Two pediatric ophthalmologists	English	Amblyopia, Childhood myopia	Accuracy (Expert categorical rating), Reproducibility
Postaci et al22	Turkey	Cross-sectional	270	ChatGPT-4o, ChatGPT-4o mini, Gemini	Two ophthalmologists	Turkish	ROP	Readability (Ateşman, Bezirci-Yılmaz), Accuracy, Comprehensiveness (Likert, ordinal scale)
Wu et al23	US	Cross-sectional	8	ChatGPT-3.5, Google Assistant	Two ophthalmologists	English	Amblyopia	Keyword adherence (AAPOS keywords %), Readability (FKGL), Word count
Ahmed et al24	UK	Cross-sectional	817	ChatGPT-3.5	Three pediatric ophthalmologists	English	Pediatric ophthalmology, Strabismus	Accuracy (4-point ordinal scale), Readability (FKGL)
Delsoz et al25	US	Cross-sectional	180	ChatGPT-3.5, ChatGPT-4o, Gemini	Two ophthalmologists	English	Childhood myopia	Quality (DISCERN), Understandability/Actionability (PEMAT-U/A), Accuracy (Likert), Readability (FKGL, SMOG)
Ermis et al26	Turkey	Cross-sectional	60	ChatGPT-4o, Claude 3, Gemini, Copilot	Three retina specialists	English	ROP	Appropriateness, Comprehensiveness (Likert, categorical rating), Readability (FKGL, Gunning Fog, CLI, SMOG, FRE)
Gary et al27	US	Cross-sectional	408	ChatGPT-3.5, ChatGPT-4o	Two pediatric ophthalmologists	English	Strabismus	Accuracy (Categorical rating), Readability (FRE, FKGL)
Guven et al28	Turkey	Cross-sectional	60	ChatGPT-4o	Two ophthalmologists	English	Strabismus, Amblyopia	Acceptability (Categorical rating), Readability (FRE, FKGL, SMOG, CLI, Gunning Fog)
Huang et al29	China	Cross-sectional	216	ChatGPT-4o, Qwen-72B, Yi-34B	80 parents, Three ophthalmologists	Chinese	Orthokeratology	Readability (Chinese tool), Correctness, Relevance, Safety, Clarity, Helpfulness, Satisfaction (Likert scales - expert & parent)
Lin et al30	China	Cross-sectional	30	ChatGPT-4o mini	Three ophthalmologists	English	Congenital cataract	Correctness, Completeness, Helpfulness, Safety (Likert scale), Readability (FRE, FKGL, DCS)
Wang et al31	China	Cross-sectional (2-phase)	354	ChatGPT-4o, Qwen, Baichuan 2, PaLM 2	Six ophthalmologists, 30 patients	Chinese	Conjunctivitis	Readability (Chinese platform), Correctness, Completeness, Helpfulness, Satisfaction, Professionalism, Humanization, Safety (Likert scales - expert & patient)
Yang et al32	US	Cross-sectional	24	ChatGPT-4o	Three English-speaking, Two Spanish-speaking pediatric ophthalmologists	English, Spanish	Strabismus surgery, Cataract surgery, Nasolacrimal duct probing	Quality (QGLOP), Readability (SMOG, FKGL, Gunning Fog)

Abbreviations: VKC, Vernal keratoconjunctivitis; ROP, Retinopathy of prematurity; PEMs, Patient education materials; CRIE, Chinese Readability Index Explorer; DISCERN, validated instrument for assessing quality of written health information; PEMAT-U/A, Patient Education Materials Assessment Tool - Understandability/Actionability; FKGL, Flesch-Kincaid Grade Level; SMOG, Simple Measure of Gobbledygook; FRE, Flesch Reading Ease; CLI, Coleman-Liau Index; SOLO, Structure of Observed Learning Outcomes; AAPOS, American Association for Pediatric Ophthalmology and Strabismus; QGLOP, Quality of Generated Language Outputs for Patients; DCS, Dale-Chall Score.

Readability was the most frequently assessed outcome (17/20 studies), followed by accuracy (10/20), understandability and actionability (5/20), appropriateness (4/20), and quality (4/20). Five studies compared AI-generated PEMs with materials from professional societies, and two incorporated patient or caregiver perspectives alongside expert evaluation (Table 2).

Readability

Readability was assessed across 17 studies using established English and non-English indices (Supplementary Table 3). A total of 2958 LLM outputs yielded 63 distinct study-level LLM evaluations; the proportions reported below refer to these evaluations rather than to individual outputs.

Most evaluations (58.7%, n=37) failed to meet recommended 6th–8^th grade readability standards, falling into the Difficult band (≥9^th grade); the full distribution across bands is shown in Figure 2.¹¹

Figure 2.

Distribution of readability classifications across study-level LLM evaluations (n = 63). Tree map showing readability classifications for 63 study-level LLM evaluations across 17 studies. Counts represent study-level LLM evaluations, not individual AI-generated patient education materials. Evaluations were classified as Easy (≤6^th grade), Acceptable (7th–8^th grade), Difficult (≥9^th grade), or Discordant when multiple readability indices assigned different bands; box size is proportional to the number of evaluations.

Abbreviation: LLM, large language model.

Readability classifications differed between ChatGPT-3.5 and ChatGPT-4o (Figure 3): 82% of ChatGPT-3.5 evaluations were rated Difficult, compared with 33% of ChatGPT-4o evaluations. Because these comparisons arise from different studies, prompts, conditions, evaluators, and sample sizes rather than controlled head-to-head testing, they should be interpreted descriptively.

Figure 3.

Distribution of readability classifications by LLM. Stacked bar chart showing readability classifications for patient education materials generated by different large language models across all included studies. Each bar represents the number of study-level evaluations for a given model, not the number of individual AI-generated materials, stratified by readability band: Easy (≤6^th grade), Acceptable (7th–8^th grade), Difficult (≥9^th grade), or Discordant (no consensus between multiple readability indices). Abbreviation: LLM, large language model.

Several infrequently evaluated models, including ChatGPT-4o mini, Microsoft Copilot, Claude 3, Bing Chat, PaLM 2, and two Chinese-language models, were rated Difficult in all assessments (Figure 3; Supplementary Table 3); given the small number of evaluations (one to three per model), no model-specific conclusions can be drawn. Overall, Easy readability was rare across all models.

Accuracy, Appropriateness, and Clinical Safety

Ten studies assessed accuracy using heterogeneous expert scales. Newer models generally demonstrated higher accuracy, with diagnostic and prognostic content outperforming treatment and prevention domains.^13,14

Hallucinations and Misinformation

For this review, hallucinations were defined as plausible-sounding but factually incorrect or fabricated information generated by LLMs.³³ None of the included studies provided a definition of hallucination or explicitly reported hallucinations in their outputs. Because most studies evaluated factual accuracy rather than systematically assessing hallucinations using standardized frameworks, the absence of reported hallucinations should be interpreted cautiously and does not indicate that outputs were free of incorrect content. Using standardized misinformation scales, studies reported no misinformation for ChatGPT-3.5, ChatGPT-4o, and Bard in childhood cataract and glaucoma, and for ChatGPT-3.5, ChatGPT-4o, Gemini, and Claude in myopia.^16,17,25 In the largest evaluation (n=817 LLM outputs), 78% of ChatGPT-3.5 outputs were rated as completely accurate, 19% as correct but insufficient, and 2.8% as partially inaccurate.²⁴

Factual Errors and Incompleteness

Although no hallucinations were reported, factual errors and omissions were identified across conditions, particularly in treatment and prevention domains. Rasmussen et al¹³ reported the lowest accuracy in vernal keratoconjunctivitis treatment content, with deficits in surgical options and medication safety. Similarly, Lim et al¹⁴ identified misinformation risks in myopia treatment and prevention. Although all LLMs performed poorly in this domain, ChatGPT-4o significantly outperformed ChatGPT-3.5 and Bard (p < 0.05).¹⁴ Nikdel et al²¹ found approximately 5.4–5.6% of ChatGPT-4o outputs were rated unacceptable due to factual errors. Beyond domain-specific issues, Gary et al²⁷ found that 32% of ChatGPT-3.5 and ChatGPT-4o responses to patient-centered strabismus questions were accurate but incomplete.

Potentially Harmful Content and Appropriateness

Within the same strabismus evaluation, Gary et al²⁷ also found that 4% of outputs were rated as potentially harmful, primarily within prognosis, treatment, and research domains. Appropriateness varied by model and condition. In retinopathy of prematurity, ChatGPT-4o and Claude 3 achieved 100% appropriate ratings, while Microsoft Copilot received inappropriate ratings in 10% of evaluations.²⁶ For amblyopia, ChatGPT-3.5, Bard, and Bing Chat achieved 80–84% appropriateness overall but all failed to generate appropriate definitions for basic concepts such as “What is amblyopia?” and “What is refractive amblyopia?”.¹⁹

Quality

Four studies assessed the quality of LLM-generated PEMs using standardized instruments. Across the three studies using the DISCERN instrument, models achieved scores in the good-to-high quality range, with ChatGPT-3.5, ChatGPT-4o, and Bard performing comparably, while Gemini received lower ratings. In the single study utilizing the Quality of Generated Language Outputs for Patients (QGLOP) instrument, ChatGPT-4o demonstrated moderate-to-high quality across accuracy, bias, currency, and tone, with minimal differences between English and Spanish outputs (Figure 4).³²

Figure 4.

Standardized quality scores of LLM generated patient education materials. Quality scores (discern and QGLOP) were normalized to a 0–100 scale across four studies. ChatGPT-3.5, ChatGPT-4o, and Bard demonstrated higher quality than Gemini.

Abbreviations: DISCERN, validated instrument for assessing the quality of written consumer health information; QGLOP, Quality of Generated Language Outputs for Patients; LLM, large language model.

Actionability and Understandability

Five studies assessed understandability and actionability using the Patient Education Materials Assessment Tool (PEMAT), which evaluates patients’ ability to comprehend information (PEMAT-U) and identify concrete next steps (PEMAT-A); scores ≥70% are considered adequate.³⁴

Across all LLMs, median and mean PEMAT-U scores exceeded the 70% adequacy threshold, indicating materials were understandable (Figure 5). In contrast, median and mean PEMAT-A scores consistently fell below 70% across all studies and models (range 40–69%). This gap between high understandability and low actionability was observed consistently across all LLMs evaluated.

Figure 5.

PEMAT understandability and actionability scores by LLM. PEMAT understandability (PEMAT-U; circles) and actionability (PEMAT-A; squares) scores for patient education materials generated by large language models. Scores ≥70% are considered adequate. Vertical lines connect paired scores for each model configuration. Citation numbers indicate source studies.

Abbreviations: PEMAT, Patient Education Materials Assessment Tool; PEMAT-U, PEMAT understandability; PEMAT-A, PEMAT actionability; LLM, large language model.

AI Compared to Pre-Existing Educational Resources

Five studies directly compared LLM-generated and professional society PEMs (Table 3). Professional society materials consistently fell into the Difficult band, with the exception of one study falling into the Discordant band. Readability comparisons with LLMs yielded mixed results: ChatGPT-4o, Gemini, and Bard produced significantly more readable content than professional society materials in three studies, while older models (ChatGPT-3.5) were generally comparable or less readable.^19,20,23,32 Professional materials significantly outperformed LLMs on all quality and accuracy assessments.^20,32 Findings for understandability and actionability (PEMAT) were mixed across studies.^19,20

Table 3.

Comparison of LLM-Generated and Professional Society Patient Education Materials

Study (Ref)	Professional Society (Readability Band †)	LLM Comparator	Readability Outcome	Understandability/Actionability (PEMAT)	Accuracy/Quality (Tool)
Doğan et al19	AAPOS (Difficult)	ChatGPT-3.5, Bard, Bing Chat	Bard > AAPOS*; ChatGPT-3.5 < AAPOS*; Bing Chat ≈ AAPOS	AAPOS (PEMAT-U: 81.5%, PEMAT-A: 74.6%) > all LLMs*	Not assessed
Yılmaz et al20	AAPOS (Difficult)	ChatGPT-3.5, Bard, Microsoft Copilot	AAPOS & Microsoft Copilot > ChatGPT-3.5 & Bard*; all Difficult	PEMAT -U: Microsoft Copilot > Bard > AAPOS (71.0%) ≈ ChatGPT-3.5; PEMAT-A: LLMs > AAPOS (52.2%); NS overall	AAPOS (SOLO: 4.14) > ChatGPT-3.5 & Microsoft Copilot*
Wu et al23	AAPOS (Difficult)	ChatGPT-3.5, Google Assistant	ChatGPT-3.5 & Google Assistant < AAPOS (NS)	Not assessed	Not assessed
Yang et al32	AAPOS (Difficult)	ChatGPT-4o (EN/ES)	ChatGPT-4o (Acceptable) > AAPOS (Difficult) (NS)	Not assessed	AAPOS (QGLOP: accuracy, currency, tone) > ChatGPT-4o*
Postacı et al22	TOA (Discordant‡)	ChatGPT-4o, 4o mini, Gemini	ChatGPT-4o & Gemini > TOA*; ChatGPT-4o mini ≈ TOA	Not assessed	Not assessed

Notes: †Readability bands: Easy ≤6^th grade; Acceptable 7th-8^th grade; Difficult ≥9^th grade. ‡Discordant: Different readability indices classified same material into different bands (TOA: Bezirci-Yılmaz = Difficult, Ateşman = Acceptable). Symbols: > higher than/more readable than; < lower than/less readable than; ≈ comparable; NS not significant; *p<0.05.

Abbreviations: AAPOS, American Association for Pediatric Ophthalmology and Strabismus; TOA, Turkish Ophthalmological Association; PEMAT, Patient Education Materials Assessment Tool; U, understandability; A, actionability; SOLO, Structure of Observed Learning Outcomes; QGLOP, Quality of Generated Language Outputs for Patients; EN, English; ES, Spanish.

Parent and Caregiver Reported Outcomes

Two studies assessed patient and caregiver perspectives. Huang et al²⁹ surveyed 80 parents evaluating orthokeratology materials, finding all LLMs received high scores (clarity >4.8/5.0, satisfaction >4.7/5.0) with no significant differences between models (Table 2). Wang et al³¹ evaluated 30 conjunctivitis patients comparing ChatGPT-4o and Qwen across seven domains, finding no significant differences overall. Despite this, Qwen scored higher on professionalism, completeness, and safety while ChatGPT-4o performed better in readability and satisfaction; physician assessments diverged from these patient ratings, ranking ChatGPT-4o higher for safety and Qwen higher for readability. Notably, patient-rated professionalism correlated with satisfaction but not with physician-assessed correctness.

Discussion

This scoping review represents the first comprehensive synthesis of evidence on the use of LLMs for patient education in pediatric ophthalmology. Across 20 studies published between 2023 and 2025, LLMs demonstrated high understandability and moderate accuracy but often exceeded recommended readability thresholds, demonstrated limited actionability, and were outperformed in quality and accuracy by professional society resources. Although all LLMs share similar underlying architectures for natural language generation, performance variations, such as the higher accuracy and readability reported for ChatGPT-4o across studies, may reflect differences in training data scale, fine-tuning, and multimodal integration rather than fundamental differences in their underlying technology.³⁵

Comparison with Professional Society Resources

Head-to-head comparisons revealed complementary strengths and weaknesses. Professional society resources consistently outperformed LLMs in accuracy and quality but were far less accessible due to poor readability (Table 3). These findings highlight an ongoing trade-off between precision and accessibility, as expert-authored materials maintain high factual accuracy yet rely on complex language that limits usability for patients and caregivers. In contrast, LLM outputs were generally more understandable, scalable across languages, and modestly improved readability, reflecting their ability to simplify medical text and generate layperson-friendly explanations. Notably, emerging evidence published following the completion of our literature search demonstrates promising results for this integrated approach. A recent investigation found that employing prompt-engineering strategies to rewrite expert-authored patient education materials significantly improved both readability and understandability while maintaining informational accuracy.⁴ These findings suggest that LLMs, when used as supplemental tools under appropriate clinical oversight, can enhance the accessibility of high-quality professional society materials without compromising their factual integrity. This represents an emerging area of research that warrants further investigation as a potential model for optimizing patient education in pediatric ophthalmology.

The Readability–Understandability Paradox

The finding that LLM-generated materials often score highly on understandability despite failing to meet readability standards highlights a critical nuance in pediatric patient education. Readability formulas measure mechanical features such as sentence and word length, whereas understandability tools like the PEMAT-U assess organization, logical sequencing, and term definitions.^11,34 Consequently, while caregivers may follow a message once decoded, high reading levels remain a key barrier to access, especially for those with limited literacy, non-native speakers, or caregivers under stress.^36,37 In pediatric ophthalmology, where caregiver comprehension directly influences adherence to treatments such as amblyopia patching, myopia control, and topical medication adherence this paradox carries direct clinical implications.³ Achieving excellence in both metrics is therefore essential to ensure educational resources translate into improved visual outcomes.^38,39

Accuracy and Quality

Consistent with the broader ophthalmology literature on LLM generated PEMs, LLMs in pediatric ophthalmology demonstrate a clear performance gap between diagnostic and treatment-focused content.⁴⁰ While these models provide accurate diagnostic and prognostic information, their reliability declines in treatment and prevention domains where clinical nuance is critical. This limitation is particularly concerning in pediatrics, where incomplete or inaccurate guidance regarding surgical options or medication safety can carry significant risk.⁴¹

These accuracy findings should also be interpreted in light of the prompting strategies used, which varied considerably across the included studies. The current evidence cannot reliably distinguish deficiencies intrinsic to the models from those attributable to how they were prompted, and some observed shortcomings in accuracy, readability, or actionability may reflect prompt design rather than inherent model capability.

The higher quality and reliability of professional society materials observed across studies reinforce that AI, though valuable for language simplification, cannot yet function independently of expert review. These findings underscore the need for clinician oversight; rather than replacing traditional resources, LLMs should be integrated as supervised adjuncts to ensure that educational materials remain both accessible and medically sound.

Actionability

A critical finding is the persistent gap between high understandability and low actionability across LLM-generated materials. While models effectively explained concepts, they consistently scored below the 70% PEMAT-A threshold, indicating limited actionable guidance.^34,38,39

All included studies evaluated LLM outputs generated through July 2025, before the policy changes described here; these changes therefore cannot explain the low actionability observed in this review and are presented as context for future evaluations. In October 2025, major LLM providers implemented policy changes that substantially altered permissible medical content.^42–44 For instance, OpenAI’s updated usage policies now prohibit the provision of tailored medical advice without appropriate involvement by a licensed professional.⁴³ Under these frameworks, LLMs may explain concepts but are constrained from producing the step-by-step instructions that PEMAT-A evaluates.

Going forward, these restrictions may limit the extent to which LLM-generated materials can achieve high PEMAT-A scores, because actionability instruments prioritize explicit “next steps”, precisely the type of output now constrained by safety standards. Future studies should therefore consider these policy changes when evaluating and interpreting actionability metrics such as PEMAT-A.

Actionable guidance is particularly important in pediatric ophthalmology, where outcomes depend on precise caregiver actions including patching adherence, medication administration, and timely escalation.^34,38,39 Providing such guidance requires integrating LLMs into clinician-controlled workflows, where physicians use prompt-engineering to generate accessible content while retaining responsibility for personalized, actionable instructions.⁴ This positions LLMs as tools enhancing physician-directed education rather than standalone sources, aligning with current policies and pediatric care needs.

Patient and Caregiver Perspectives

A notable finding is the gap between patient satisfaction and clinical accuracy. Caregivers reported high satisfaction with orthokeratology and conjunctivitis materials, yet patient perceptions of quality did not correlate with physician assessments of correctness.^29,31 Moreover, only two of the 20 included studies incorporated patient or caregiver perspectives, and none measured whether LLM-generated education improves caregiver comprehension, adherence, shared decision-making, or visual outcomes. The current evidence therefore primarily reflects surrogate properties of generated text, such as readability, accuracy, and understandability, rather than demonstrated benefit to patients and families. Future research should jointly assess user experience and clinical validity to ensure materials are both accessible and medically sound.

Limitations

The evidence base is primarily constrained by substantial methodological heterogeneity, as varied evaluation frameworks and inconsistent reporting practices currently limit direct meta-analysis. These challenges reflect broader gaps in readability research, where the lack of standardized summary measures necessitates the use of interpretive bands, a pragmatic but less precise approach. The majority-rule approach used to assign these bands also involves subjective judgment, as readability indices weight text features differently and do not necessarily measure equivalent constructs, particularly across languages; band assignments may therefore have differed under alternative classification rules. In addition, readability proportions were calculated across study-level LLM evaluations, each contributing equally regardless of whether the underlying study generated eight or several hundred materials; these proportions are therefore descriptive and should not be interpreted as pooled output-level prevalence estimates. Similarly, comparisons between models, such as ChatGPT-3.5 and ChatGPT-4o, arise from different studies, prompts, conditions, evaluators, and sample sizes rather than controlled head-to-head testing. End-user perspectives were largely absent, with only 10% of studies incorporating patient or caregiver input. All analyses adopted a binary “AI vs expert” which overlooks the clinically relevant potential of collaborative, hybrid workflows.

Consistent with scoping review methodology, no formal critical appraisal or risk-of-bias assessment of the included studies was performed, and the review protocol was not prospectively registered. This review therefore maps reported findings without judging the methodological quality of the source studies, and comparative statements regarding accuracy, quality, readability, or clinical safety, including observations that one model or resource performed better than another, should be interpreted descriptively.

The rapid evolution of LLMs also complicates interpretation, as studies published between 2023 and 2025 may already reflect outdated models; for example, newer versions such as ChatGPT-5 were not captured in this review. Additionally, language bias remains a limitation when comparing readability across languages, since most tools are language-specific and may not measure equivalent constructs.⁴⁵ For instance, Yang et al³¹ applied English-based readability indices to Spanish outputs, potentially introducing measurement error. Lastly, most professional society PEMs are freely available and often distributed in print, LLM-based tools often require internet access and subscriptions. Although these costs are generally minimal, they may inadvertently widen health disparities for families with limited digital literacy or resources.

Future Directions

These limitations highlight key priorities for future research. Rather than choosing between AI and expert authorship, the most promising direction is integrating LLMs into professional workflows to combine their respective strengths. Studies should validate whether LLMs can enhance the readability of expert-authored materials while maintaining clinical accuracy, with particular attention to prompt-engineering strategies that optimize understandability.^4,10,46,47 Given that current industry policies position LLMs as tools for general medical information rather than individualized guidance, future research should focus on hybrid models where physicians leverage LLM-enhanced content for accessibility while retaining responsibility for providing actionable, personalized instructions. Randomized trials examining these integrated workflows, including multimodal approaches that pair LLM-simplified text with clinician-reviewed visual aids, animations, or interactive demonstrations, are needed to determine whether such strategies improve caregiver comprehension and treatment adherence compared to traditional education methods.^34,47 In addition, standardized evaluation frameworks, including consistent readability indices, accuracy scales, and reporting of prompts and model versions, are needed to improve comparability across future studies.

Finally, caregiver co-design should be central to development. Evidence across health education shows that involving end-users improves clarity, satisfaction, and trust; integrating caregiver feedback with clinician oversight will be critical to ensure AI-augmented education translates into measurable improvements in understanding and visual outcomes.^2,47

Conclusion

This scoping review highlights both the potential and current limitations of large language models in pediatric ophthalmology. LLMs generate understandable and scalable materials but often exceed readability standards and demonstrate limited actionability. Accuracy varied across studies, models, and conditions, with important deficiencies in treatment-related content, and remained below that of professional resources. Evidence remains fragmented, with limited patient input and few hybrid approaches, and derives predominantly from cross-sectional evaluations of generated text rather than studies demonstrating improved caregiver knowledge, adherence, safety, or patient outcomes. Prospective clinical studies evaluating real-world caregiver outcomes are needed. Future work should focus on clinician-supervised integration that leverages LLM strengths in accessibility while ensuring accuracy and actionable guidance through professional oversight. Incorporating equity considerations and caregiver co-design will be critical to ensuring AI-augmented materials meet the diverse needs of pediatric ophthalmology patients and families.

Disclosure

Dr Edsel Ing reports Support for the manuscript from Royal Alexandra Hospital, during the conduct of the study; Consulting fees from Canada Health Agency; Leadership or fiduciary roles from Alberta Health Services, Chair, World Association of Medical Editors, Eye Physicians and Surgeons Association of Alberta, outside the submitted work. Dr Carlos Solarte reports Grants or contracts from Amman Builders Inc., outside the submitted work. The authors declare no other conflicts of interest related to this work.