Table 1.
Details of the model attributes of the included studies.
| Authors | Model | Timing | Count | Specificity of the prompts and language |
| Al-Ashwal et al [42] | ChatGPT-3.5, ChatGPT-4, Bing, and Bard with unclear settings | One month, May 2023 | 255 drug-drug pairs | The exact prompts were provided in English. |
| Alfertshofer et al [43] | ChatGPT with unclear settings | Not provided | 1800 questions | The exact prompt was used for each question. The questions were taken from United States, United Kingdom, Italian, French, Spanish, and Indian exams. A new session was used for each question. |
| Ali et al [44] | ChatGPT Feb 9 free version with unclear settings | Not provided | 50 items | Information was provided fully in the supplementary file of the article in English. |
| Aljindan et al [45] | ChatGPT-4 with unclear settings | Not provided | 220 questions | Initial prompting that involved role playing as a medical professional. The language was English. |
| Altamimi et al [46] | ChatGPT-3.5 | Single day not otherwise specified | 9 questions | The exact prompts were provided in English. |
| Baglivo et al [47] | Bing, ChatGPT, Chatsonic, Bard, and YouChat with full details of the mode and large language model, including plugins | Exact dates were provided for each model (April 12, 13, and 14, 2023, and July 13, 2023) | 15 questions | Italian was used. |
| Biswas et al [48] | ChatGPT-3.5 with unclear settings | Exact date provided (March 16, 2023) | 11 questions | The exact prompts were provided in English, with a new session for each question. |
| Chen et al [49] | ChatGPT-4 with unclear settings | Not provided | 560 questions | The exact prompts were provided in English. |
| Deiana et al [50] | ChatGPT-3.5 and ChatGPT-4 with unclear settings | Not provided | 11 questions | The exact prompts were not explicitly provided. English was used. A new session was used for each question. Up to three iterations were allowed for incorrect responses. |
| Fuchs et al [51] | ChatGPT-3 and ChatGPT-4 with unclear settings | Exact dates provided (February 19, 2023, and March 25, 2023) | 60 questions | Dental medicine questions were translated from German to English, while the other questions were already present in English. Exact prompts were provided in English with prompting in 2 groups for the same questions: one group was primed with instructions, while the other was not primed. A total of 20 trials were conducted per group, and chat history was cleared after each trial. |
| Ghosh & Bir [52] | ChatGPT (version March 14, 2023) with unclear settings | March 14 and 16, 2023 | 200 questions | The exact prompts and language were not explicitly provided. The first response was taken as final, and the option of “regenerate response” was not used. |
| Giannos [53] | ChatGPT-3 and ChatGPT-4 with unclear settings | Not provided | 69 questions | Not explicitly provided. |
| Gobira et al [54] | ChatGPT-4 with unclear settings | Not provided | 125 questions | Portuguese was used. |
| Grewal et al [55] | ChatGPT-4 with unclear settings | The first week of May 2023 | Not clear | The exact prompts were provided in English. One follow-up prompt was used for enhancement of some prompts. |
| Guerra et al [56] | ChatGPT-4 with unclear settings | Not provided | 591 questions | The exact prompts were provided, while the language was not explicitly provided. |
| Hamed et al [57] | ChatGPT-4 with unclear settings | Not provided | Not clear | The exact prompts and language were not explicitly provided. Different prompts were tried to identify the most suitable. |
| Hoch et al [58] | ChatGPT (May 3rd version) with unclear settings | May 5 and 7, 2023 | 2576 questions | The exact prompts were provided, while the language was not explicitly provided. |
| Juhi et al [59] | ChatGPT with unclear settings | February 20, 2023, to March 5, 2023 | 40 drug-drug pairs | The exact prompts were provided in English. |
| Kuang et al [60] | ChatGPT with unclear settings | Not provided | Not clear | The exact prompts were not explicitly provided. English was used. |
| Kumari et al [61] | ChatGPT-3.5, Bard, and Bing with unclear settings | July 30, 2023 | 50 questions | The exact prompts were not explicitly provided. English was used. |
| Kung et al [62] | ChatGPT-3.5 and ChatGPT-4 with unclear settings | July 2023 | 215 questions | Not clear |
| Lai et al [63] | ChatGPT-4 (May 24; Version 3.5) with unclear settings | Not provided | 200 questions | The exact prompts and language were not explicitly provided. Three attempts to answer the complete set of questions over 3 weeks (once per week), with a new session for each question. |
| Lyu et al [64] | ChatGPT with unclear settings | Mid-February 2023 | Not clear | The exact prompts were provided in English. |
| Moise et al [65] | ChatGPT-3.5 with unclear settings | Not provided | 23 questions | The exact prompts were provided in English, with a new session for each question. |
| Oca et al [66] | ChatGPT, Bing, and Bard with unclear settings | April 11, 2023 | 20 queries for each model | The exact prompts were provided in English. |
| Oztermeli & Oztermeli [67] | ChatGPT-3.5 with unclear settings | Not provided | 1177 questions | The exact prompts were not explicitly provided. Turkish was used, with a new session for each question. |
| Pugliese et al [68] | ChatGPT with unclear settings | March 25, 2023 | 15 questions | The exact prompts were provided in English, with a new session for each question. |
| Sallam et al [69] | ChatGPT (default model) with unclear settings | February 25, 2023 | Not provided | The exact prompts were provided in English. |
| Seth et al [70] | ChatGPT-3.5, Bard, and Bing AI | Not provided | 6 questions | The exact prompts were provided in English. |
| Suthar et al [71] | ChatGPT-4 with unclear settings | Not provided | 140 cases | The exact prompts were not explicitly provided. English was used. |
| Walker et al [72] | ChatGPT-4 with unclear settings | Not provided | 5 cases | The exact prompts were not explicitly provided. English was used, with a new session for each question. |
| Wang et al [73] | ChatGPT-3.5 and ChatGPT-4 with unclear settings | February 14, 2023, for ChatGPT-3.5 and May 14-16, 2023, for ChatGPT-4 | 300 questions | The exact prompts were provided. Chinese and English were used. The prompts were enhanced though role play. |
| Wang et al [74] | ChatGPT-3.5 with unclear settings | March 5-10, 2023 | Not clear | Chinese (Mandarin) and English were used. Examples of prompts were provided. |
| Zhou et al [75] | ChatGPT-3.5 with unclear settings | April 24-25, 2023 | Single case and multiple poll questions | The exact prompts were provided in English. |