To the Editor:
Tsui et al. [1]. reported their experience with using a popular online artificial intelligence (AI) chatbot, ChatGPT for ophthalmic symptoms triage. The authors scripted ten prompts for common ophthalmic scenarios that patients may experience including retinal detachment, exudative age-related macular degeneration, ocular surface disease, etc. The authors found that ChatGPT only provided suitable and precise responses for eight out of the ten responses with the two remaining answers being both unsuitable and imprecise [1]. Since the release of this study, OpenAI released GPT-4 which is reported to have a broader knowledge base, image analysis capabilities, as well as greater problem solving abilities than the previous version [2].
We examined the performance of GPT-4 by re-testing its ability to respond to the two scenarios that it struggled with previously. We then graded GPT-4’s response to these prompts, and the responses produced were now both precise and suitable (Table 1). For example, in response to the prompt “I have macular degeneration and things look wavy today”, GPT-4 now recommends seeking immediate medical attention if experiencing severe or sudden visual changes, which is a more precise and suitable answer clinically. GPT-4 also re-states a second time that it strongly recommends the patient to seek medical attention. Previously, ChatGPT would state the importance of talking to a doctor, but then mention the importance of wearing sunglasses and quitting smoking which is an unsuitable and imprecise response to a potentially irreversible loss of vision.
Table 1.
Appropriateness of responses by GPT-4.
| Prompt given | Previous Response Grade | GPT-4 Response Grade |
|---|---|---|
| I have macular degeneration and things look wavy today. | Imprecise, Unsuitable | Precise, Suitablea |
| My eye is red after an eye injection yesterday. | Imprecise, Unsuitable | Precise, Suitable |
aExample response: I’m sorry to hear that you’re experiencing wavy vision due to macular degeneration. Macular degeneration is a condition that affects the central part of the retina, which is responsible for sharp, detailed vision. It can cause a variety of symptoms, including blurry or distorted vision, difficulty seeing fine details, and changes in color perception. If you are experiencing sudden or severe changes in your vision, I would recommend seeking immediate medical attention from an eye doctor or ophthalmologist. They can perform a thorough exam and provide a proper diagnosis and treatment plan. In the meantime, there are a few things you can do to manage your symptoms and protect your vision. It’s important to avoid smoking and to eat a healthy, balanced diet with plenty of fruits and vegetables, as well as foods rich in omega-3 fatty acids, such as salmon and walnuts. You may also want to consider using visual aids, such as magnifying glasses or special computer software, to help you see more clearly. Again, I strongly recommend that you seek medical attention as soon as possible to ensure that you receive the appropriate care for your condition.
The utility of large language models and chatbots for medical uses will rapidly expand in the future. Our group has also recently examined how ChatGPT could potentially be a useful tool in medical emergencies during long-duration spaceflight, where signal delays to ground-based experts would result in delays in receiving medical care [3].
A unique advantage to GPT-4 is its image analysis capabilities which may revolutionize ophthalmic image triage. However, it is important to remember that current GPT technology has performance limitations which must be accounted for prior to it ever being used clinically [4]. We applaud the authors for this groundbreaking study and invite them to share their perspectives on the anticipated differences between ChatGPT and GPT-4 in ophthalmic symptoms triage.
Author contributions
EW—conceptualization, writing. JO—conceptualization, writing. NZ—review, intellectual support. SK—review, intellectual support. PS—review, intellectual support. AT—review, intellectual support. AGL—review, intellectual support.
Funding
NASA Grant [80NSSC20K183]: A Non-intrusive Ocular Monitoring Framework to Model Ocular Structure and Functional Changes due to Long-term Spaceflight.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Tsui JC, Wong MB, Kim BJ, Maguire AM, Scoles D, VanderBeek BL, et al. Appropriateness of ophthalmic symptoms triage by a popular online artificial intelligence chatbot. Eye. 2023. 10.1038/s41433-023-02556-2. [DOI] [PMC free article] [PubMed]
- 2.Waisberg E, Ong J, Masalkhi M, Kamran SA, Zaman N, Sarker P, et al. GPT-4: a new era of artificial intelligence in medicine. Ir J Med Sci. 2023. 10.1007/s11845-023-03377-8 [DOI] [PubMed]
- 3.Waisberg E, Ong J, Masalkhi M, Zaman N, Kamran SA, Sarker P, et al. Generative Pre-trained Transformers (GPT) and Space Health: A Potential Frontier in Astronaut Health During Exploration Missions. Prehospital Disaster Med. Published online (accepted). [DOI] [PMC free article] [PubMed]
- 4.Alser M, Waisberg E Concerns with the usage of ChatGPT in Academia and Medicine: A viewpoint. Am J Med Open. 2023:100036. 10.1016/j.ajmo.2023.100036. [DOI] [PMC free article] [PubMed]