Table 2. Sample AAO question inputs and GPT model’s output.

Abbreviations: AAO: American Academy of Ophthalmology, GPT: Generative Pre-trained Transformer

Input Type	AAO Difficulty	Input	GPT-3.5 Output	GPT-3.5 Outcome	AAO Agreement	GPT-4.0 Output	GPT-4.0 Outcome	AAO Agreement
Text	86% (least)	A 3-month-old exhibits bilateral eye irritation, with complaints of chronic, mostly clear tearing and occasional lash mattering. The exam is otherwise unremarkable, including the presence of normal intraocular pressure. What is the most appropriate initial management? Topical antibiotic-steroid eye drops, dacryocystorhinostomy, observation and digital massage, nasolacrimal duct probing	The most appropriate initial management for a 3-month-old with bilateral eye irritation and complaints of chronic, mostly clear tearing and occasional lash mattering would be observation and digital massage. In infants, excessive tearing and eye irritation can often be caused by nasolacrimal duct obstruction, which is a common condition that occurs when the nasolacrimal duct, a tube that carries tears from the eye to the nose, becomes blocked. Nasolacrimal duct obstruction can be caused by a variety of factors, including inflammation, infection, or structural abnormalities. Observation and digital massage can be effective in relieving the symptoms of nasolacrimal duct obstruction in infants. Digital massage involves gently massaging the area around the inner corner of the eye, where the nasolacrimal duct is located, in order to help clear the blockage. If the symptoms resolve with digital massage, further treatment may not be necessary. If the symptoms do not improve with digital massage, or if the infant has other signs or symptoms that suggest a more serious underlying condition, further testing or treatment may be necessary. Topical antibiotic-steroid eye drops would not be the most appropriate initial management in this case, as there is no evidence of infection or inflammation on exam. Dacryocystorhinostomy and nasolacrimal duct probing are surgical procedures that are typically reserved for cases of nasolacrimal duct obstruction that do not respond to non-surgical treatment, such as digital massage.	Correct	86%	In a 3-month-old with bilateral eye irritation, chronic clear tearing, and occasional lash mattering, the most appropriate initial management would be observation and digital massage. These symptoms are suggestive of nasolacrimal duct obstruction, which is relatively common in infants and often resolves spontaneously within the first year of life. Parents can be instructed to perform digital massage, also known as the “​​​Crigler massage,”​​​​​​​ which involves applying gentle pressure over the nasolacrimal sac area to facilitate the opening of the obstruction. If the symptoms persist or worsen, further intervention such as nasolacrimal duct probing may be considered.	Correct	86%
Text	29% (most)	Several years ago, a patient had a small-aperture corneal inlay implanted to treat presbyopia, as shown. The patient has since developed diabetic retinopathy requiring intervention. What procedure is compromised by the presence of the inlay? Applanation tonometry, focal laser to the macula, injection of vascular endothelial growth factor (VEGF) medications, fundus photography	The presence of a small-aperture corneal inlay may compromise the accuracy of applanation tonometry, a procedure used to measure intraocular pressure (IOP) in the eye. Applanation tonometry involves the application of a small, flat pressure-sensing device to the surface of the cornea. The inlay, which is a thin, opaque disk that is implanted in the cornea to change its refractive power and treat presbyopia, can affect the way the cornea flattens under pressure and, therefore, may interfere with the accuracy of IOP measurements. The presence of the inlay should not compromise focal laser to the macula, injection of vascular endothelial growth factor (VEGF) medications, or fundus photography. These procedures do not involve contact with the inlay or the front surface of the cornea. It is important to carefully consider the potential impact of any ocular implants or devices on diagnostic or therapeutic procedures, and to take appropriate precautions as needed. In the case of a small-aperture corneal inlay, it may be necessary to use an alternative method of IOP measurement or to adjust the measurement technique to account for the presence of the inlay.	Incorrect	38%	The presence of a small-aperture corneal inlay can compromise the accuracy of applanation tonometry. Applanation tonometry is used to measure intraocular pressure (IOP), and the presence of the corneal inlay can alter the corneal biomechanics, leading to inaccurate IOP measurements. Focal laser to the macula, injection of vascular endothelial growth factor (VEGF) medications, and fundus photography are typically not compromised by the presence of a corneal inlay, as these procedures target the posterior segment of the eye and are not directly affected by the presence of the inlay in the cornea.	Incorrect	38%
Image	46% (most)	A 60-year-old woman complains of difficulty reading. She is under treatment for fibromyalgia, arthritis, interstitial cystitis, and hypertension. She has no family with retinal disease. Her visual acuity is 20/20 in each eye with macular appearance and fundus autofluorescence as shown. What is the most likely diagnosis? Retinal toxicity associated with pentosan polysulfate sodium, age-related macular degeneration, pattern dystrophy, retinal toxicity associated with hydroxychloroquine	Based on the information provided, the most likely diagnosis is age-related macular degeneration (AMD). AMD is a common condition that affects the central portion of the retina, called the macula, which is responsible for detailed vision such as reading and driving. It typically occurs in people over the age of 50 and is more common in those with a family history of the condition. The patient’s age and the presence of macular degeneration are consistent with the diagnosis of AMD. The other conditions listed (retinal toxicity associated with pentosan polysulfate sodium, pattern dystrophy, and retinal toxicity associated with hydroxychloroquine) are not as likely based on the information provided. It is important to perform additional testing (such as imaging studies and laboratory tests) to confirm the diagnosis and determine the appropriate course of treatment.	Incorrect	12%	The image is not provided, but based on the description provided, the patient is experiencing difficulty reading and is under treatment for several conditions, including fibromyalgia, arthritis, interstitial cystitis, and hypertension. Considering her medical history and the absence of any family history of retinal disease, the most likely diagnosis is: 1. Retinal toxicity associated with pentosan polysulfate sodium: This medication is used to treat interstitial cystitis and has been associated with a risk of retinal toxicity, which can present with difficulty reading and changes in the macular appearance. Other differential diagnoses mentioned include: 2. Age-related macular degeneration (AMD): Although the patient is 60 years old, the absence of family history and her relatively good visual acuity of 20/20 in each eye make AMD less likely in this case. 3. Pattern dystrophy: This is a group of inherited retinal diseases, but the patient has no family history of retinal disease, which makes this diagnosis less likely. 4. Retinal toxicity associated with hydroxychloroquine: This medication is used for treating various autoimmune conditions, such as rheumatoid arthritis and lupus. However, the patient’s medical history doesn’t mention the use of hydroxychloroquine. In summary, given the patient’s medical history, the most likely diagnosis is retinal toxicity associated with pentosan polysulfate sodium.	Correct	46%
Image-aided	46% (most)	A 60-year-old woman complains of difficulty reading. She is under treatment for fibromyalgia, arthritis, interstitial cystitis, and hypertension. She has no family with retinal disease. Her visual acuity is 20/20 in each eye with macular appearance and fundus autofluorescence as shown. Macular findings include pigmented parafoveal deposits. What is the most likely diagnosis? Retinal toxicity associated with pentosan polysulfate sodium, age-related macular degeneration, pattern dystrophy, retinal toxicity associated with hydroxychloroquine	Based on the information provided, it seems that the patient is experiencing difficulty reading and has macular findings that include pigmented parafoveal deposits. These findings are consistent with age-related macular degeneration (AMD), which is a common cause of visual impairment in older adults. AMD is characterized by the accumulation of deposits in the macula, which can lead to vision loss. Retinal toxicity associated with pentosan polysulfate sodium or hydroxychloroquine would not be the most likely diagnosis in this case, as there is no mention of the patient taking these medications. Pattern dystrophy is a rare genetic disorder that can cause vision loss, but it typically presents with characteristic patterns of retinal pigment epithelial (RPE) abnormalities, which are not described in this case. In conclusion, the most likely diagnosis in this case is age-related macular degeneration. Further testing, such as imaging studies and visual field testing, may be helpful in confirming the diagnosis and determining the appropriate management plan.	Incorrect	12%	The patient’s macular findings and her history of interstitial cystitis treatment suggest the diagnosis of retinal toxicity associated with pentosan polysulfate sodium. Pentosan polysulfate sodium is a medication commonly used for the treatment of interstitial cystitis, a chronic condition causing bladder pain and pressure. Studies have suggested a link between the long-term use of pentosan polysulfate sodium and a unique form of retinal toxicity, which can lead to vision changes. Age-related macular degeneration typically presents with drusen and changes in the retinal pigment epithelium, pattern dystrophy usually has a family history and specific patterns on autofluorescence, and retinal toxicity associated with hydroxychloroquine typically presents with parafoveal loss of the ellipsoid zone and "flying saucer" sign on OCT, which does not seem to be the case here.	Correct	46%
Image	77% (middle)	A 16-year-old girl presents with painless decreased vision in her right eye. She denies any history of head or eye trauma. Her slit-lamp exam is shown. What consultation should you order next? Rheumatology, neurology, cardiology, nephrology	Based on the information provided, the most appropriate consultation to order next would be with a rheumatology specialist. The slit-lamp examination shows the presence of a white, fluffy, and diffuse infiltrate in the anterior chamber of the eye, which is consistent with the diagnosis of uveitis. Uveitis is an inflammation of the uveal tract, which includes the iris, ciliary body, and choroid. It can be caused by a variety of factors, including autoimmune disorders, infections, and other underlying systemic conditions. Rheumatology is a subspecialty of internal medicine that focuses on the diagnosis and treatment of autoimmune and inflammatory conditions. A rheumatology consultation would be appropriate to help determine the cause of the patient’s uveitis and to guide management. While neurology, cardiology, and nephrology are all important specialties, they are not specifically indicated in this case based on the information provided. It is important to accurately diagnose and manage uveitis to prevent potential complications, such as vision loss or glaucoma.	Incorrect	13%	Given the very limited information, it’s challenging to make an accurate choice. However, considering the patient’s age and the common causes of painless vision loss in this age group, I would lean towards a neurology consultation. This is because neurological conditions like optic neuritis or a mass lesion (such as a brain tumor) pressing on the optic nerve can cause painless vision loss and may be more common in this age group. But remember, without more specific information, this is a guess and in a real clinical scenario, more information would be needed to make an appropriate referral.	Incorrect	2%
Image-aided	77% (middle)	A 16-year-old girl presents to the ophthalmologist with painless decreased vision in her right eye. She denies any history of head or eye trauma. Her slit-lamp exam is shown, with lens subluxation. Which one of the following consultations should you order next? Rheumatology, neurology, cardiology, nephrology	Based on the information provided, the most appropriate consultation to order would be with a rheumatologist. Lens subluxation, or displacement of the lens within the eye, can sometimes be a symptom of an underlying systemic condition such as rheumatoid arthritis or Marfan syndrome. A rheumatologist is a specialist who can evaluate the patient for these and other rheumatologic conditions that may be causing the lens subluxation. It is important to identify and address any underlying systemic conditions in order to properly manage the lens subluxation and ensure the best possible outcome for the patient’s vision.	Incorrect	13%	The patient has lens subluxation, which is a hallmark feature of Marfan syndrome - a connective tissue disorder that affects many organ systems, including the skeletal, cardiovascular, and ocular systems. Marfan syndrome is an autosomal dominant disorder caused by mutations in the fibrillin-1 gene. Cardiovascular involvement, particularly aortic root dilatation leading to aortic dissection, is the most serious manifestation of Marfan syndrome and the main cause of premature death. Thus, the most urgent next step in management is a cardiology referral for echocardiographic evaluation of the aorta. It’s important to initiate appropriate treatment and monitoring to prevent cardiovascular complications.	Correct	77%