Google’s AI chatbot “Bard”: a side-by-side comparison with ChatGPT and its utilization in ophthalmology

Bard AI, developed by Google, represents a significant advancement in the realm of artificial intelligence (AI) powered chatbots [1]. This innovative AI is designed to simulate human-like interactions, answering a wide array of user inquiries and prompts [1]. Having been trained on a vast dataset of text data, Bard utilizes sophisticated language models to generate comprehensive and informative responses to user inputs [1]. The mechanism behind Bard involves its large language model, specifically the Pathways Language Model 2 (PaLM 2), which has been engineered to excel in comprehending facts, logical reasoning, and mathematics [1]. The capacity of the model to effectively analyze and comprehend contextual information enables it to deliver comprehensive and precise responses to a wide array of inquiries [1, 2]. While Bard shares some similarities with ChatGPT, Bard AI distinguishes itself through its emphasis on factual accuracy. Bard demonstrates exceptional proficiency in understanding and answering questions that require precise information [1]. In contrast, ChatGPT, a generative language model, is proficient in generating creative and diverse types of text, such as articles, code, scripts, and more which may or may not be completely accurate [3].

One of the fundamental differences between Bard AI and ChatGPT lies in their data sources and connections to the internet. Bard AI is linked to Google’s expansive search network, granting it access to real-time information from the internet [1]. This allows Bard AI to provide up-to-date information about current events and topics [1]. On the other hand, ChatGPT is not directly connected to the internet, and leverages data up to 2021 to generate its responses [3, 4]. ChatGPT has already shown considerable potential in the field of ophthalmology, from helping triage ophthalmic symptoms [5], medical education [6], answering ophthalmic board certification practice questions [7], writing cataract surgery operative notes [8], developing ophthalmology AI algorithms to classify OCT images [9].

The utilization of Bard AI has considerable potential upside and downside for ophthalmology. The ability of Bard AI to comprehend and provide factual information makes it a valuable tool for staying up to date with the latest developments, research findings, and treatment options in ophthalmology [2]. Furthermore, Bard AI can assist in providing accurate and easy-to-understand explanations of ophthalmological conditions, and treatment plans. We compared the performance of Bard with GPT-3.5 with various ophthalmology-related prompts. Responses generated by both chatbots were compared by four designated content experts (senior ophthalmologist, ophthalmology resident, and 2 vision scientists).

We first decided to ask Bard whether an individual with a family history of glaucoma needs to see an eye doctor. The response provided by Bard was both specific and accurate, stating that early stages of glaucoma are often symptomless, and that an earlier diagnosis and treatment will yield the best chances of vision preservation (Fig. 1A). Bard then quoted a national guideline from the National Health Service (NHS), recommending individuals with a family history of glaucoma have a yearly eye test, and if over the age of 40 have an eye test every 2 years. Although this recommendation is reasonable, when directly compared with the original source material from the NHS website for patients, no such exact recommendation was provided. Instead, the NHS website states that “If you’re at a higher risk of glaucoma—for example, if you have a close relative with it—you may be advised to have more frequent tests” [10]. This phrasing is more accurate as each individual with a family history of glaucoma is different, and may have other glaucoma risk factors, thus an exact frequency of eye tests is difficult to provide. The response provided by ChatGPT was suitable as the chatbot explained glaucoma briefly and stated the importance of regular eye exams and reaching out to an eye care professional for more personalized advice (Fig. 1B).

Fig. 1. LLM-generated outputs.

A Output generated by Bard from the prompt “I have a family history of glaucoma, do I need to see an eye doctor?”. B Output generated by ChatGPT.

We then prompted both of AI chatbots about a patient reporting “flashes of lights” in one eye, and if they should attend the emergency department (Fig. 2). Both Bard and ChatGPT correctly recommended to attend the emergency department, particularly if this vision change occurred suddenly (Fig. 2). Both chatbots also appropriately stated that this symptom could be a sign of a retinal tear or detachment, requiring urgent evaluation. These AI-generated outputs were both specific, and appropriate.

Fig. 2. LLM-generated outputs.

A Output generated by Bard from the prompt “I see flashes of light in my left eye, should I go to the emergency department?”. B Output generated by ChatGPT.

Finally, we prompted both AI chatbots about what to do if a patient is seeing that lines appear blurry in one eye (Fig. 3). Generating a suitable suggestion from this input is challenging as the reason for this blurry vision can be a result from relatively benign conditions (e.g., refractive errors) or more serious conditions (e.g., eye injuries, eye infections). Bard correctly identified many possible causes and recommended a formal consult with an eye care professional if sudden visual loss occurred, or if accompanied by symptoms such as pain, floaters or flashes of light. The response generated by ChatGPT however was less specific, but still appropriate. ChatGPT initially suggested this may be due to a variety of causes including eye strain, improper eyewear, poor lighting, eye rubbing or dehydration. ChatGPT also suggested a consultation with an eye care professional if the blurriness was severe, sudden or accompanied by symptoms such as pain, floaters or flashes of light.

Fig. 3. LLM-generated outputs.

A Output generated by Bard from the prompt “Lines are blurry in one eye? What should I do?”. B Output generated by ChatGPT for the same prompt.

Finally we examined Bard’s image analysis capabilities by providing a fundus image of arteritic anterior ischemic optic neuropathy (Fig. 4) [11]. The latest update of Bard includes Google Lens integrations, allowing for image analysis alongside prompts. Bard stated that it is not a medical expert prior to providing any information. Bard correctly recognized that this image was an image of the optic nerve, however then provided information regarding glaucoma instead of arteritic anterior ischemic optic neuropathy.

Fig. 4. Output generated from Bard.

Fundus image of arteritic anterior ischemic optic neuropathy reprinted without changes from Waisberg and Micieli [11] under Creative Commons Attribution - Non Commercial (unported, v3.0) License.

Limitations

While Bard has the potential to be an exciting tool in ophthalmology clinics of the future, it is currently labeled by Google as an “experiment” and Bard describes itself as “a creative and helpful collaborator” and states that it currently has “limitations and won’t always get it right”. False responses or “hallucinations” are currently one of the greatest concerns with clinical applications AI chatbots [12]. The generation of false responses can occur as large language models do not have a full human-like like understanding of text and subtle misunderstandings can lead to errors that pose a risk to patient safety. Training data for Bard and other AI chatbots include non-peer reviewed sources which may contain factual inaccuracies. The black box nature of AI chatbots does not show clearly how an output is generated. As Bard does not clearly show citations for where the information was obtained, it is difficult to verify the sources used to generate its outputs.

The usage of more ophthalmology-specific training data will hopefully increase the accuracy and precision of future iterations of the AI chatbot [13]. Diverse ophthalmology data should also be used to train Bard as biased training data could potentially amplify inequalities in ophthalmology. Protection of confidential patient medical information is also essential for both medical and medicolegal reasons. It should be noted that all entries in Bard can be read and annotated by human reviewers for the purposes of quality improvement. Further regulatory and compliance concerns including ownership of data will need to be adjudicated in order to ensure protections for personal health care information.

Conclusion

In summary, AI chatbots including Bard have the potential to revolutionize ophthalmology. The possible future applications of Bard in ophthalmology go far beyond the applications described here. Google is committed to improving the performance of its AI chatbot, and its Gemini project will be releasing in the Fall of 2023. It is expected that newer iterations will be even more proficient with inputs that are text-based, video or images. As new, and more capable AI chatbots are released they should be approached with optimistic caution, to ensure that the fundamental priority is the safety of patients.

Author contributions

EW and JO: writing, figure development. MM: writing. NZ, PS, AGL and AT: review, intellectual support.

Competing interests

The authors declare no competing interests.