Table 2.
Comparison of reading grade levels between institutional, initial LLM and simplified LLM responses to bariatric surgery frequently asked questions
| Grade range | Gunning Fog Score | Flesch–Kincaid Grade Level | Coleman–Liau Index | SMOG Index | Automated Readability Index | Linsear–Write Formula | |
|---|---|---|---|---|---|---|---|
| Institutions | 10th grade to college sophomore | 14.2 (4.4) | 11.0 (3.8) | 11.1 (3.3) | 10.4 (3.0) | 10.6 (4.7) | 12.0 (5.5) |
| GPT-3.5 Initial | College freshman to college graduate | 18.1 (2.7)* | 13.6 (2.3)* | 14.2 (1.8)* | 13.0 (1.8)* | 13.8 (2.7)* | 14.7 (3.4)* |
| GPT-3.5 Simplified | 10th grade to college freshman | 13.4 (2.6) | 9.6 (2.0)* | 11.6 (1.6) | 9.9 (1.7) | 9.7 (2.3) | 10.1 (2.8)* |
| GPT-4 Initial | 12th grade to college senior | 15.6 (2.6)* | 11.8 (2.0) | 12.4 (1.6)* | 11.5 (1.7)* | 11.7 (2.4) | 12.8 (3.3) |
| GPT-4 Simplified | 6th grade to 9th grade | 9.4 (1.9)* | 6.2 (1.5)* | 8.0 (1.4)* | 7.0 (1.2)* | 5.8 (1.9)* | 7.1 (2.1)* |
| Bard Initial | 9th grade to college freshman | 13.3 (2.7) | 9.8 (2.6)* | 9.5 (1.6)* | 9.9 (2.0) | 9.2 (2.9)* | 11.4 (4.1) |
| Bard Simplified | 8th grade to 12th grade | 12.1 (2.6)* | 8.5 (2.4)* | 8.8 (1.4)* | 9.0 (2.0)* | 7.8 (2.6)* | 9.9 (3.5)* |
All values are presented as mean (standard deviation)
LLM large language model
*p < 0.05 when compared to institutional scores