Abstract
Background
Since the widespread availability of Chat Generative Pre-Trained Transformer (ChatGPT), the public is confronted with accessible artificial intelligence tools. There is limited knowledge on the use, concerns and opportunities of ChatGPT in pharmacy practice in the Netherlands.
Objectives
The aims of this study were to explore the extent and type of use of ChatGPT in community pharmacy and to identify concerns and opportunities for pharmacy practice.
Methods
A questionnaire was developed, tested and distributed to professionals that work in community pharmacy. The answers were analysed descriptively using frequency tables.
Results
Of all participants (n = 106), 50.9 % had used ChatGPT, and 38.7 % (n = 24) of these users has used it in pharmacy. Participants saw opportunities for using ChatGPT as writing assistant or in quickly answering clinical questions. Concerns included not knowing what ChatGPT could be used for in pharmacy and not knowing what ChatGPT's answer is based on.
Conclusions
This research shows that using ChatGPT as a writing assistant is valuable and can free up time. Although clinical questions seem promising, ChatGPT's answers are currently too unreliable and do not meet the required quality standards for good pharmaceutical care. If ChatGPT is used to answer clinical questions, crossreferencing with reliable sources is recommended.
Keywords: ChatGPT, Artificial intelligence, Community pharmacy, Questionnaire, Implementation, Large language model
1. Background
Since the widespread availability of Artificial Intelligence (AI) in the form of large language models (LLMs), the general public is confronted with accessible AI-tools. AI can be used to automate repetitive tasks and help to save time in busy working environments.1 With the release of ChatGPT (Chat Generative Pre-Trained Transformer) in November 2022, openly available AI has created a paradigm shift in many working fields. ChatGPT uses Natural Language Processing to answer to prompts in a human-like way.2,3 It can be used to automate certain processes such as helping with administrational or conversational tasks. Like in many primary care settings, the workload in community pharmacies is high with increasing administration load and a shortage of pharmacy staff. Tools like ChatGPT could be used to enhance services and efficiency, such as for patient education.1,4 Therefore, the availability of AI can help speed up processes in community pharmacy to improve efficiency, effectiveness and quality of the output.3,5
Thus far, the implementation in pharmacy faces several barriers. ChatGPT 3.5's responses are based on a fixed dataset obtained during its last training update, meaning it does not include the most recent studies. However, newer models can access the internet and can therefore give more updated answers.2,3,5, 6, 7. Additionally, ChatGPT's understanding of complex scientific contexts and nuances might not always match the depth and specificity required in pharmaceutical practices where answers are influenced by context and surrounded with a level of uncertainty.3,6,8,9 Moreover, ChatGPT is not able to check the accuracy of the answers; it predicts the most likely word to be next in a sentence. The way ChatGPT formulates answers makes it seem trustworthy even though it is just generating text. This phenomenon is called hallucination and can have large implications in pharmacy.2,3,5,8 Another limitation is reproducibility: ChatGPT answers differently to the same prompt every time it is asked. This sometimes leads to a correct answer, while generating an inaccurate answer in other occasions.3
The overview given above shows opportunities, but also limitations for the use of AI-supported LLMs. Presently, many people are experimenting how these tools might be useful in their professional environment. There is limited knowledge on the current use, concerns and opportunities of large language models such as ChatGPT in pharmacy practice in the Netherlands.
The aims of this study were to explore the extent and type of use of ChatGPT in community pharmacy and to identify the concerns and opportunities for pharmacy practice.
2. Methods
2.1. Study design
This was a cross-sectional survey aimed at community pharmacy staff, including community pharmacists, pharmacy assistants and pharmacy students. A questionnaire was designed in the online survey software tool Survey Monkey (Survey Monkey, USA) and distributed by sending an open invitation to participate to subscribers to newsletters of pharmacists in the network of Health Base Foundation, an independent organisation that supports health care professionals in safe and effective use of drugs by IT-solutions, and an invitation by LinkedIn and WhatsApp. The questionnaire was open for 3 weeks in June/July 2023.2
Based on Dutch law, this study does not fall under medical research involving human subjects act and therefore did not have to undergo an institutional review board.10 The data was collected anonymously and stored in accordance with Dutch privacy regulations.
2.2. Questionnaire development
The questionnaire was developed by two pharmacist-researchers. Participants were firstly asked for background characteristics. Secondly, they rated statements about the use of ChatGPT on a 5-point Likert scale for agreement. Participants had the option to add comments. Lastly, open questions were used to retrieve what participants had used ChatGPT for.
The statements and topics were derived from earlier research and customised to the pharmacy context. 11 The extent and type of use, opportunities and concerns of ChatGPT were explored for tasks that might help pharmacy practice with respect to completeness, reliability, accuracy, privacy and environmental aspects of ChatGPT. Completeness refers to the question being fully answered by ChatGPT without the need for additional actions. Reliability refers to the references to original sources given by ChatGPT when asked by the user to check if the information was based on valid references. Accuracy refers to the answer being validated by other trusted sources.11, 12, 13, 14, 15
Finally, the study explored if pharmacy staff used ChatGPT in practice or if they saw a potential role in the future for the following six subjects: quick orientation on a rare disease, information about adverse effects, clinical decision support regarding a drug-drug interaction, writing a chapter of an annual report, checking correct dosage for a child, and preparation of a pharmacotherapeutic consultation. Participants were encouraged to use ChatGPT 3.5 for one or more pharmacy-specific questions, and to answer the questionnaire based on these experiences. In this way, pharmacy staff that had not used ChatGPT before could also participate in this part of the questionnaire.2 The questionnaire was tested by 10 community and administrative pharmacists and fine-tuned with their feedback. The final version of the questionnaire consisted of basic demographic characteristics, extent and type of use of ChatGPT and concerns and opportunities for ChatGPT in pharmacy. Demographic information was limited to profession, gender, age and familiarity with ChatGPT.11
2.3. Analysis
All answers were transferred to Microsoft Excel 2016 and analysed descriptively using frequency tables. The responses were reduced to a 3-point Likert scale of agreement. Differences between respondents who used and did not use ChatGPT were calculated using single chi-square tests for each item in excel. P-values of <0.05 were considered statistically significant. Open comments were categorised in: writing support (eg. summarize or simplify text), clinical questions (eg. drug-drug interactions) and operational questions (e.g. data analysis).4,5,9,12
3. Results
A total of 106 respondents filled in the survey, of which 76 % (n = 81) worked in pharmacy with an equal distribution according to gender and age. Twelve participants did not complete the survey. Most participants were familiar with ChatGPT and nearly half (50.9 %, n = 54) of the respondents had used it themselves. Of the participants that had used ChatGPT, 38.7 % (n = 24) used ChatGPT in pharmacy. For an overview of the participant's characteristics, extent and type of use, see Table 1.
Table 1.
Characteristics of the respondents (n = 106).
| Characteristic | Number of respondents (%) |
|---|---|
| gender | |
| female | 54 (50.9) |
| male | 47 (44.3) |
| other | 5 (4.7) |
| age (years) | |
| <30 | 23 (21.7) |
| 30–49 | 51 (48.1) |
| 50+ | 32 (30.2) |
| Profession | |
| community pharmacist⁎ | 73 (68.9) |
| community pharmacy staff | 8 (7.5) |
| student (internship community pharmacy) | 9 (8.5) |
| Other⁎⁎ | 16 (15) |
| Familiarity with ChatGPT | |
| Yes, I (roughly) know how it works | 95 (89.6) |
| Yes, I know it by name | 8 (7.5) |
| No | 3 (2.8) |
| Experience with ChatGPT | |
| Yes, I have used ChatGPT myself | 54 (50.9) |
| Yes, I have seen ChatGPT being used by others | 13 (12.3) |
| No | 39 (36.8) |
| Used ChatGPT in the pharmacy | |
| Yes I have used ChatGPT in the pharmacy | 24 (38.7) |
| No I have not used ChatGPT in the pharmacy | 38 (61.3) |
including 6 pharmacists in training for community pharmacy.
Professionals cooperating with community pharmacy, including 4 hospital pharmacists.
3.1. Opportunities
Table 2 shows the current and future opportunities for ChatGPT according to the participants. Participants were already using ChatGPT for writing support, such as for simplifying text. It was also used for clinical questions such as missed doses or questions about pharmacokinetics. Other tasks ChatGPT had been used for, included help with website design and marketing purposes.
Table 2.
Activities in pharmacy practice that currently can be - or could be in future - supported by ChatGPT. The qualitative comments have been documented verbatim. Aspects that were not found (potentially) useful are also included in this table for completeness.
| For which aspects in your work do you see a potential role of ChatGPT? | (potentially) Useful activity in n (%), (n = 72) | Positive experience with this activity in pharmacy practice in n (%), (n = 17) |
|---|---|---|
| Questions about adverse effects | 38 (53) | 7 (41) |
| Medication safety questions | 29 (40) | 5 (29) |
| Quick orientation on a subject | 38 (53) | 4 (24) |
| Control on child dosage | 24 (33) | 3 (18) |
| Prepare multidisciplinary meetings | 42 (58) | 3 (18) |
| Writing passages for annual report | 0 (0) | 0 (0) |
| Activities suggested by respondents in comments and in response to the question: are there other subjects you (could) use ChatGPT for? | ||
| Writing assistant | For (re)writing a job posting.⁎ Informational text: Once I needed an informational text about hay fever and the drugs used in it and their effects and adverse effects. ⁎ Create documents, such as letters, quality documents, etc., as support to speed up the process of writing. ⁎ For making texts clear and readable (accessible) for the website or for patients. ⁎ Translate a patient leaflet into a foreign language. ⁎ Writing scientific research and articles. ⁎ |
|
| Clinical questions | Forgot to take a tablet. What should I do? ⁎ Answer questions about kinetics / dynamics. ⁎ Checking prescriptions. Responding to all kinds of patient questions. Producing information, in which the pharmacist always has a monitoring role. To conduct better and faster medication reviews. Looking up interactions. Setting up courses. Answer general questions about adverse effects. |
|
| Operational activities | Help with website design. ⁎ PR & communication. ⁎ Logistical processes (order status, mismatches in follow-up shipments). Automation of prescription renewals. |
|
Subjects where ChatGPT currently is used for.
Respondents saw future opportunities for ChatGPT for answering patients' questions, support with medication reviews, or answering questions regarding adverse effects. In the operational area, possibilities laid in automation of prescription renewals and improving logistics. Participants did not find ChatGPT useful for writing passages for annual reports.
3.2. Concerns
Table 3 shows the current concerns for the use of ChatGPT in pharmacy. The two main concerns were not knowing what role ChatGPT might have in the pharmacy (60 %, n = 56 agree) and not knowing how ChatGPT generates information (63 %, n = 59 agree). Third, 32 % (n = 30) of the participants experienced the information of ChatGPT unreliable, whereas 31 % (n = 29) did not. Lastly, privacy and the environment were the least concern. The open answers of participants are shown in Table 3. Concerns regarding reliability, accuracy and completeness were mainly stated regarding clinical questions.
Table 3.
Concerns about the use of ChatGPT in pharmacy (n = 94) including verbatim comments from participants.
| Participants in n (%) (n = 94) |
|||
|---|---|---|---|
| Disagree | Neutral | Agree | |
| I do not know what role ChatGPT can have in my work | 21 (22) | 16 (17) | 56 (60)1 |
| The information of ChatGPT is unreliable. | 29 (31) | 34 (36) | 30 (32)2 |
| It is unclear to me how ChatGPT generates information | 18 (19) | 14 (15) | 59 (63)3 |
| I am concerned about privacy | 45 (48) | 22 (23) | 27 (29) |
| ChatGPT consumes a lot of energy and is a burden for the environment | 50 (53) | 25 (27) | 17 (19) |
| Reliability | The links to references work, but when opened you do not get the actual reference. ChatGPT quickly helps you in a certain train of thought with beautiful sentence formulations. However, the content should always be verified. I really miss the reliability of the information and the references it is based on. The answer still refers to doctor or pharmacist of course. However, it is clear that the answers are a composite from conventional sources given the answer. I see the risk that people quickly tend to take it for truth and no longer think independently whether it is correct and whether you can follow the reasoning. |
||
| Accuracy | ChatGPT does not take into account history and changes in medication use that may affect the onset of symptoms In pharmacy, you regularly write letters to job applicants, the municipality and health insurance companies, for example. ChatGPT composed a very decent letter that is usable in practice with a few minor changes. |
||
| Completeness | Does give the dosage recommendation (number of mg/kg), but you still have to calculate the exact dosage with the weight of the patient. Does not provide relevant advice. Does provide general information about the interaction and both resources. Standards are not included it seems. Advice now states that it is best for the physician to determine. So, at this time, for this question insufficient. The answer was that ChatGPT cannot give medical advice. An alternative is also not mentioned. |
||
| Impressiveness | I had not tried ChatGPT at all…. and this survey encouraged me to make an account. Surprising how much that thing can do! Heavily impressed. |
||
Not applicable: n = 1.
Not applicable: n = 3.
Not applicable: n = 2
The study showed that users of ChatGPT differed from non-users as they more frequently knew what role ChatGPT could have in their work (p = 0.002) and were less concerned about privacy issues (p = 0.014). No other statistical differences were seen between users and non-users (see supplementary table 3.1).
4. Discussion
This study showed that half of the participants have ever used ChatGPT, of whom nearly 40 % (n = 24) professionally in pharmacy. They experienced that ChatGPT was useful, mainly as a writing assistant, and that it could support pharmacy workers for a diversity of writing tasks. The participants had concerns regarding clinical questions with respect to reliability, accuracy and completeness of the answers. Privacy and the environmental burden were the least concern.
This is the first study presenting findings regarding the extent, type of use, concerns and possibilities of ChatGPT in pharmacy in the Netherlands. The potential uses of ChatGPT in pharmacy practice has been investigated in a limited number of studies. For instance, Jairoun et al. mentions that AI-tools could support pharmacists with tasks such as answering clinical questions, writing support for summarising patient records and administrative tasks.3 In another recent study, Shin et al., have shown that ChatGPT can be a useful assistant for a variety of different tasks of the community pharmacist.13
The concerns found in this study are in line with studies among pharmacists in Jordan and Thailand posing that pharmacists support the use of ChatGPT but have concerns about the accuracy of the information, regulations, ethics and bias.5,14 Studies comparing the answers of clinical pharmacists to the answers of ChatGPT show contradictory outcomes.15 Munir et al. showed limited success of ChatGPT 3.5 in answering pharmacy-based questions.16 In contrast, ChatGPT 4 showed promising and satisfactory performance in supporting medication management and wellness information in the community pharmacy setting, although further rigorous validation of these responses is required before widespread application of large language models in practice.13 A qualitative study explored the limitations of LLMs: bias, hallucination and (lack of) contextual awareness.17 This study adds to previous research because it gives more insight in the background of the concerns for pharmacy practice. This leads to better understanding of what clinical details are needed to address concerns about ChatGPT, such as accurate references, providing the specific dosing recommendation and suggesting an alternative. This study showed a difference in concerns regarding privacy between users and non-users of ChatGPT. In the medical field, privacy of patient information is paramount. There are concerns regarding the privacy of personal sensitive information in current large language models.17 While using ChatGPT or other LLMs for clinical questions, privacy should be taken into consideration. The concerns in pharmacy are comparable to the concerns posed more broadly in health care.17
Strengths of this study include that it is the first to investigate the implication of ChatGPT in community pharmacy in the Netherlands. The potential role of ChatGPT was explored by combining closed and open-ended questions presenting a broad impression of the use of ChatGPT in community pharmacy. However, this study also has several limitations. First, the study population is limited to 106 participants and there is limited data about the representativeness of this population for the working field in pharmacy. The sample size was too low to statistically compare participant characteristics and some participants did not complete the final questions of the survey. This may have led to a bias in the results the survey showed. Moreover, it is unknown how participants used ChatGPT. Formulating good questions using accurate prompts is crucial for obtaining specific, contextually appropriate responses and to reduce the risk of misinformation.17, 18, 19, 20, 21. It is likely that the quality of ChatGPT's answers lied partially in prompt engineering. Finally, this study is an impression of summer 2023. With fast evolving development and implementation of AI, the results of this study will not fully cover the current situation. However, this study explores current opinions of people who work in community pharmacy and provides insights for further implementation. Conducting in-depth interviews with community pharmacists will provide valuable insights about the current use and comfort of using AI and can assist in the development of guidelines for use of AI in pharmacy practice.
The solution to increase accuracy, reliability and completeness may be found in the development of an LLM that has been specifically trained on pharmacy practice data. Unlike general LLMs that are trained on a wide range of topics, domain-specific models are focused on a particular area of expertise. This allows them to have a deeper understanding of the context, terminology, and nuances specific to that domain.22 Currently, pharmacy-specific GPTs are available in OpenAI's GPT store. These GPTs have more specific pharmaceutical knowledge. Yet, caution with using these GPT's is still recommended.23 Moreover, OpenAI has released the most recent reasoning model o1 in September 2024. Reasoning models are better in forming a chain of thought. The o1 model performs a lot better in scientific reasoning benchmarks compared to previous models such as 4o. The exact implication of reasoning models in pharmacy is not clear yet.24., 25, 26, 27. Implementation of AI models might further be supported by training pharmacy staff in how to formulate good prompts and have knowledge of both the possibilities and limitations of the most common LLMs.9,22
5. Conclusion
This study shows that using ChatGPT for writing assistance is valuable and can free up time in pharmacies. Although use for clinical questions seem promising, ChatGPT's answers are currently too unreliable and do not meet health care quality standards. If ChatGPT is used for clinical questions, cross-referencing with reliable sources is recommended.
To further elucidate the possibilities and limitations of ChatGPT and other large language models in pharmacy, more research and practice experience is needed.
Declaration of use of generative AI
During the preparation of this work the authors used DeepL and ChatGPT in order to optimise sentences grammatically and translations. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.
Financial support
No funding was received.
Author contribution
ER drafted the manuscript and analysed and interpreted the data. VE, CR, KT and SB participated in data analysis and interpretation, and critically revised the manuscript. Supervision was done by SB. All authors approved the final version of the manuscript.
CRediT authorship contribution statement
Emma Janske de Ruiter: Writing – original draft, Methodology, Investigation, Data curation. Vesna Maria Eimermann: Writing – review & editing. Claudia Rijcken: Writing – review & editing. Katja Taxis: Writing – review & editing. Sander Diederik Borgsteede: Writing – review & editing, Supervision.
Declaration of competing interest
None.
References
- 1.Schutz N., Olsen C.A., Mclaughlin A.J., et al. ASHP statement on the use of artificial intelligence in pharmacy. Am J Health-Syst Pharm. 2020;77(23):2015–2018. doi: 10.1093/AJHP/ZXAA249. [DOI] [PubMed] [Google Scholar]
- 2.OpenAI Introducing ChatGPT. 2024. https://openai.com/blog/chatgpt November 30, 2022. Accessed January 4.
- 3.Jairoun A.A., Al-Hemyari S.S., Shahwan M., Humaid Alnuaimi G.R., Zyoud S.H., Jairoun M. ChatGPT: threat or boon to the future of pharmacy practice? Res Social Adm Pharm. 2023;19(7):975–976. doi: 10.1016/J.SAPHARM.2023.03.012. [DOI] [PubMed] [Google Scholar]
- 4.Dergaa I., Chamari K., Zmijewski P., Ben Saad H. From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing. Biol Sport. 2023;40(2):615–622. doi: 10.5114/BIOLSPORT.2023.125623. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Abu Hammour K., Alhamad H., Al-Ashwal F.Y., Halboup A., Abu Farha R., Abu Hammour A. ChatGPT in pharmacy practice: a cross-sectional exploration of Jordanian pharmacists’ perception, practice, and concerns. J Pharm Policy Pract. 2023;16(1) doi: 10.1186/S40545-023-00624-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kleebayoon A., Wiwanitkit V. Performance and risks of ChatGPT used in drug information: comment. Eur J Hosp Pharm. 2023;31(1) doi: 10.1136/EJHPHARM-2023-003864. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.OpenAI ChatGPT. 2024. https://openai.com/chatgpt/ Accessed July 22.
- 8.Morath B., Chiriac U., Jaszkowski E., et al. Performance and risks of ChatGPT used in drug information: an exploratory real-world analysis. Eur J Hosp Pharm. 2023 doi: 10.1136/EJHPHARM-2023-003750. Published online. [DOI] [PubMed] [Google Scholar]
- 9.Krumborg J.R., Mikkelsen N., Damkier P., et al. ChatGPT: first glance from a perspective of clinical pharmacology. Basic Clin Pharmacol Toxicol. 2023;133(1):3–5. doi: 10.1111/BCPT.13879. [DOI] [PubMed] [Google Scholar]
- 10.CCMO . The Central Committee on Research Involving Human Subjects; 2024. Your research: Is it subject to the WMO or not? | Investigators.https://english.ccmo.nl/investigators/legal-framework-for-medical-scientific-research/your-research-is-it-subject-to-the-wmo-or-not Accessed January 4. [Google Scholar]
- 11.Luykx J.J., Gerritse F., Habets P.C., Vinkers C.H. The performance of ChatGPT in generating answers to clinical questions in psychiatry: a two-layer assessment. World Psychiatry. 2023;22(3):479–480. doi: 10.1002/WPS.21145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Matulis J., McCoy R. Relief in sight? Chatbots, in-baskets, and the overwhelmed primary care clinician. J Gen Intern Med. 2023;38(12):2808–2815. doi: 10.1007/S11606-023-08271-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Shin E., Hartman M., Ramanathan M. Performance of the ChatGPT large language model for decision support in community pharmacy. Br J Clin Pharmacol. 2024;90(12) doi: 10.1111/BCP.16215. [DOI] [PubMed] [Google Scholar]
- 14.Boonrit N., Chaisawat K., Phueakong C., Nootong N., Ruanglertboon W. Exploring community pharmacists’ attitudes in Thailand towards ChatGPT usage: a pilot qualitative investigation. Digit Health. 2024;10 doi: 10.1177/20552076241283256. 20552076241283256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Huang X., Estau D., Liu X., Yu Y., Qin J., Li Z. Evaluating the performance of ChatGPT in clinical pharmacy: a comparative study of ChatGPT and clinical pharmacists. Br J Clin Pharmacol. 2024;90(1) doi: 10.1111/BCP.15896. [DOI] [PubMed] [Google Scholar]
- 16.Munir F., Gehres A., Wai D., Song L. Evaluation of ChatGPT as a tool for answering clinical questions in pharmacy practice. J Pharm Pract. 2024;37(6) doi: 10.1177/08971900241256731. [DOI] [PubMed] [Google Scholar]
- 17.Tian S., Jin Q., Yeganova L., et al. Opportunities and challenges for ChatGPT and large language models in biomedicine and health. Brief Bioinform. 2024;25(1):1–13. doi: 10.1093/bib/bbad493. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Giray L. Prompt engineering with ChatGPT: a guide for academic writers. Ann Biomed Eng. 2023;51:2629–2633. doi: 10.1007/s10439-023-03272-4. [DOI] [PubMed] [Google Scholar]
- 19.Cheng S.W., Chang C.W., Chang W.J., et al. The now and future of ChatGPT and GPT in psychiatry. Psychiatry Clin Neurosci. 2023;77(11):592–596. doi: 10.1111/PCN.13588/FULL. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Meskó B. Prompt engineering as an important emerging skill for medical professionals: tutorial. J Med Internet Res. 2023;25(1) doi: 10.2196/50638. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bsharat S.M., Myrzakhan A., Shen Z. Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4. Published online December 26. 2023. http://arxiv.org/abs/2312.16171 Accessed February 2, 2024.
- 22.Au Yeung J., Kraljevic Z., Luintel A., et al. AI chatbots not yet ready for clinical use. Front Digit Health. 2023;5:1161098. doi: 10.3389/FDGTH.2023.1161098/BIBTEX. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.OpenAI Introducing GPTs. November 6. 2023. https://openai.com/blog/introducing-gpts Accessed February 2, 2024.
- 24.Introducing OpenAI o1 | OpenAI. 2024. https://openai.com/index/introducing-openai-o1-preview/ Accessed December 15.
- 25.Xie S., Zhao W., Deng G., et al. Utilizing ChatGPT as a scientific reasoning engine to differentiate conflicting evidence and summarize challenges in controversial clinical questions. J Am Med Inform Assoc. 2024;31(7):1551–1560. doi: 10.1093/JAMIA/OCAE100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Temsah M.H., Jamal A., Alhasan K., Temsah A.A., Malki K.H. OpenAI o1-preview vs. ChatGPT in healthcare: a new frontier in medical AI reasoning. Cureus. 2024;16(10) doi: 10.7759/CUREUS.70640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Jones N. “In awe”: scientists impressed by latest ChatGPT model o1. Nature. 2024;634(8033):275–276. doi: 10.1038/D41586-024-03169-9. [DOI] [PubMed] [Google Scholar]