Table 1. Student's t-test was used to compare the mean DISCERN ratings between three human reviewers and ChatGPT. Six out of 16 of the DISCERN questions were rated differently when comparing three human reviewers to ChatGPT.
* indicates p-values less than 0.05
| TABLE 1 DISCERN Questions | Reviewer | N | Mean | Std. Error Mean | p-value |
| 1. Are the aims clear? | Human | 27 | 4.222 | 0.180 | 0.283 |
| ChatGPT | 27 | 4.444 | 0.097 | ||
| 2. Does it achieve its aims? | Human | 27 | 4.074 | 0.168 | 0.077 |
| ChatGPT | 27 | 3.704 | 0.117 | ||
| 3. Is it relevant? | Human | 27 | 4.259 | 0.147 | 0.183 |
| ChatGPT | 27 | 4.519 | 0.124 | ||
| 4. Is it clear what sources of information were used to compile the publication (other than the author or producer)? | Human | 27 | 3.037 | 0.322 | 0.597 |
| ChatGPT | 27 | 2.852 | 0.127 | ||
| 5. Is it clear when the information used or reported in the publication was produced? | Human | 27 | 2.741 | 0.327 | 0.032* |
| ChatGPT | 27 | 3.519 | 0.112 | ||
| 6. Is it balanced and unbiased? | Human | 27 | 2.963 | 0.253 | 0.27 |
| ChatGPT | 27 | 3.296 | 0.158 | ||
| 7. Does it provide details of additional sources of support and information? | Human | 27 | 2.852 | 0.281 | 0.491 |
| ChatGPT | 27 | 2.630 | 0.152 | ||
| 8. Does it refer to areas of uncertainty? | Human | 27 | 2.963 | 0.299 | 0.353 |
| ChatGPT | 27 | 3.296 | 0.191 | ||
| 9. Does it describe how each treatment works? | Human | 27 | 3.667 | 0.207 | 0.002* |
| ChatGPT | 27 | 4.407 | 0.096 | ||
| 10. Does it describe the benefits of each treatment? | Human | 27 | 3.593 | 0.202 | 0.016* |
| ChatGPT | 27 | 4.148 | 0.088 | ||
| 11. Does it describe the risks of each treatment? | Human | 27 | 2.333 | 0.316 | 0.001* |
| ChatGPT | 27 | 3.667 | 0.151 | ||
| 12. Does it describe what would happen if no treatment is used? | Human | 27 | 2.407 | 0.187 | 0.005* |
| ChatGPT | 27 | 3.185 | 0.185 | ||
| 13. Does it describe how the treatment choices affect the overall quality of life? | Human | 27 | 3.333 | 0.151 | 0.766 |
| ChatGPT | 27 | 3.259 | 0.197 | ||
| 14. Is it clear that there may be more than one possible treatment choice? | Human | 27 | 3.296 | 0.244 | 0.309 |
| ChatGPT | 27 | 3.630 | 0.214 | ||
| 15. Does it provide support for shared decision-making? | Human | 27 | 3.593 | 0.222 | 0.01* |
| ChatGPT | 27 | 2.852 | 0.166 | ||
| 16. Based on the answers to all of the above questions, rate the overall quality of the publication as a source of information about treatment choices | Human | 27 | 3.148 | 0.271 | 0.094 |
| ChatGPT | 27 | 3.655 | 0.115 |