Table 3.

Classification of the included studies based on the features of data used to generate queries for testing on the generative artificial intelligence–based models.

Authors	Transparency	Range	Randomization
Al-Ashwal et al [42]	Full description using 2 tools for assessment: Micromedex, a subscription-based drug-drug interaction screening tool, and Drugs.com, a free database	Narrow; Drug-drug interaction prediction	Nonrandom; Purposeful selection of the drugs by a clinical pharmacist; 5 drugs paired with the top 51 prescribed drugs
Alfertshofer et al [43]	Full description using the question bank AMBOSS, with official permission for the use of the AMBOSS USMLE step 2CK practice question bank for research purposes granted by AMBOSS	Broad	Randomly extracted
Ali et al [44]	Developed by the researchers and reviewed by a panel of experienced academics for accuracy, clarity of language, relevance, and agreement on correct answers; Evaluation of face validity, accuracy, and suitability for undergraduate dental students	Narrow intersubject (dentistry); Broad intrasubject in restorative dentistry, periodontics, fixed prosthodontics, removable prosthodontics, endodontics, pedodontics, orthodontics, preventive dentistry, oral surgery, and oral medicine	Not clear
Aljindan et al [45]	Saudi Medical Licensing Exam questions extracted from the subscription CanadaQBank website	Broad in medicine, with 30% of the questions from medicine, 25% from obstetrics and gynecology, 25% from pediatrics, and the remaining 20% from surgery	Randomized through 4 researchers to ensure comprehensive coverage of questions and eliminate potential bias in question selection
Altamimi et al [46]	Snakebite management information guidelines derived from the World Health Organization, Centers for Disease Control and Prevention, and the clinical literature	Narrow	Not clear
Baglivo et al [47]	The Italian National Medical Residency test	Narrow; Vaccination-related questions from the Italian National Medical Residency Test	Not clear
Biswas et al [48]	Constructed based on the frequently asked questions on the myopia webpage of the Association of British Dispensing Opticians and the College of Optometrists	Narrow involving 9 categories: 1 each for disease summary, cause, symptom, onset, prevention, complication, natural history of untreated myopia, and prognosis, and 3 involving treatments	Not clear
Chen et al [49]	BoardVitals, which is an online question bank accredited by the Accreditation Council for Continuing Medical Education	Neurology-based; Broad intrasubject: basic neuroscience; behavioral, cognitive, psychiatry; cerebrovascular; child neurology; congenital; cranial nerves; critical care; demyelinating disorders; epilepsy and seizures; ethics; genetic; headache; imaging or diagnostic studies; movement disorders; neuro-ophthalmology; neuro-otology; neuroinfectious disease; neurologic complications of systemic disease; neuromuscular; neurotoxicology, nutrition, metabolic; oncology; pain; pharmacology; pregnancy; sleep; and trauma	Not clear
Deiana et al [50]	Questions concerning vaccine myths and misconceptions by the World Health Organization	Narrow on vaccine myths and misconceptions	Not clear
Fuchs et al [51]	Digital platform self-assessment questions tailored for dental and medical students at the University of Bern’s Institute for Medical Education	Broad with multiple-choice questions designed for dental students preparing for the Swiss Federal Licensing Examination in Dental Medicine, and allergists and immunologists preparing for the European Examination in Allergy and Clinical Immunology	Not clear
Ghosh & Bir [52]	Department question bank, which is a compilation of first and second semester questions from various medical universities across India	Biochemistry	Random without details of randomization
Giannos [53]	Specialty Certificate Examination Neurology Web Questions bank	Neurology and neuroscience	Not clear
Gobira et al [54]	National Brazilian Examination for Revalidation of Medical Diplomas issued by Foreign Higher Education Institutions (Revalida)	Preventive medicine, gynecology and obstetrics, surgery, internal medicine, and pediatrics	Not clear
Grewal et al [55]	Not clear	Radiology	Not clear
Guerra et al [56]	Questions released by the Congress of Neurological Surgeons in the self-assessment neurosurgery exam	Neurosurgery across 7 subspecialties: tumor, cerebrovascular, trauma, spine, functional, pediatrics, and pain or nerve	Not clear
Hamed et al [57]	Guidelines adapted from Diabetes Canada Clinical Practice Guidelines Expert Committee, the Royal Australian College of General Practitioners, Australian Diabetes Society position statement, and the Joint British Diabetes Societies	Management of diabetic ketoacidosis	Not clear
Hoch et al [58]	Question database of an online learning platform funded by the German Society of Oto-Rhino-Laryngology, Head and Neck Surgery	Otolaryngology with a range of 15 distinct otolaryngology subspecialties, including allergology, audiology, ear, nose and throat, tumors, face and neck, inner ear and skull base, larynx, middle ear, oral cavity and pharynx, nose and sinuses, phoniatrics, salivary glands, sleep medicine, vestibular system, and legal aspects	Not clear
Juhi et al [59]	A list of drug-drug interactions from previously published research	Narrow on drug-drug interaction	Not clear
Kuang et al [60]	Not clear	Neurosurgery	Not clear
Kumari et al [61]	Designed by experts in hematology-related cases	Hematology with the following intrasubject aspects: case solving, laboratory calculations, disease interpretations, and other relevant aspects of hematology	Not clear
Kung et al [62]	Orthopedic In-Training Examination	Orthopedics	Not clear
Lai et al [63]	The United Kingdom Medical Licensing Assessment, which is a newly derived national undergraduate medical exit examination	Broad in medicine with the following aspects: acute and emergency, cancer, cardiovascular, child health, clinical hematology, ear, nose and throat, endocrine and metabolic, gastrointestinal including liver, general practice and primary health care, genetics and genomics, infection, medical ethics and law, medicine of older adults, mental health, musculoskeletal, neuroscience obstetrics and gynecology, ophthalmology, palliative and end of life care, perioperative medicine and anesthesia, renal and urology, respiratory, sexual health, social and population health, and surgery	Not clear
Lyu et al [64]	Chest computed tomography and brain magnetic resonance imaging screening reports collected from the Atrium Health Wake Forest Baptist clinical database	Radiology	Not clear
Moise et al [65]	Statements published in the latest American Academy of Otolaryngology–Head and Neck Surgery Foundation Clinical Practice Guideline: Tympanostomy Tubes in Children (Update)	Narrow involving otolaryngology	Not clear
Oca et al [66]	Not clear	Narrow involving only queries on accurate recommendation of close ophthalmologists	Not clear
Oztermeli & Oztermeli [67]	Turkish medical specialty exam, prepared by the Student Selection and Placement Center	Broad: basic sciences including anatomy, physiology-histology-embryology, biochemistry, microbiology, pathology, and pharmacology; clinical including internal medicine, pediatrics, general surgery, obstetrics and gynecology, neurology, neurosurgery, psychiatry, public health, dermatology, radiology, nuclear medicine, otolaryngology, ophthalmology, orthopedics, physical medicine and rehabilitation, urology, pediatric surgery, cardiovascular surgery, thoracic surgery, plastic surgery, anesthesiology and reanimation, and emergency medicine	Not clear
Pugliese et al [68]	Expert selection of 15 questions commonly asked by patients with nonalcoholic fatty liver disease	Narrow involving nonalcoholic fatty liver disease aspects	Not clear
Sallam et al [69]	Panel discussion of experts in health care education	Broad on health care education, medical, dental, pharmacy, and public health	Not clear
Seth et al [70]	Devised by 3 fellows of the Royal Australasian College of Surgeons, with experience in performing rhinoplasty and expertise in facial reconstructive surgery	Narrow involving technical aspects of rhinoplasty	Not clear
Suthar et al [71]	Quizzes from the Case of the Month feature of the American Journal of Neuroradiology	Narrow involving radiology	Not clear
Walker et al [72]	Devised based on the Global Burden of Disease tool	Narrow involving benign and malignant hepatopancreaticobiliary-related conditions	Not clear
Wang et al [73]	Medical Exam Help. A total of 10 inpatient and 10 outpatient medical records to form a collection of Chinese medical records after desensitization	Clinical medicine, basic medicine, medical humanities, and relevant laws	Not clear
Wang et al [74]	The Taiwanese Senior Professional and Technical Examinations for Pharmacists downloaded from the Ministry of Examination website	Broad involving pharmacology and pharmaceutical chemistry, pharmaceutical analysis and pharmacognosy (including Chinese medicine), pharmaceutics and biopharmaceutics, dispensing pharmacy and clinical pharmacy, therapeutics, pharmacy administration, and pharmacy law	Not clear
Zhou et al [75]	A single clinical case from OrthoBullets, a global clinical collaboration platform for orthopedic surgeons; Written permission to use their clinical case report	Very narrow involving a single orthopedic case	Not clear