Table 3.
A summary of the main conclusions of the included records representing preprints.
| Author(s) [Record] | Design, Aims | Applications, Benefits | Risks, Concerns, Limitations | Suggested Action, Conclusions |
|---|---|---|---|---|
| Wang et al. [61] | An arXiv preprint 1; investigating ChatGPT effectiveness to generate Boolean queries for systematic literature reviews | Higher precision compared to the current automatic query formulation methods | Non-suitability for high-recall retrieval; many incorrect MeSH 11 terms; variability in query effectiveness across multiple requests; a black-box application | A promising tool for research |
| Borji [20] | An arXiv preprint; to highlight the limitations of ChatGPT | Extremely helpful in scientific writing | Problems in spatial, temporal, physical, psychological and logical reasoning; limited capability to calculate mathematical expressions; factual errors; risk of bias and discrimination; difficulty in using idioms; lack of real emotions and thoughts; no perspective for the subject; over-detailed; lacks human-like divergences; lack of transparency and reliability; security concerns with vulnerability to data poisoning; violation of data privacy; plagiarism; impact on the environment and climate; ethical and social consequences | Implementation of responsible use and precautions; proper monitoring; transparent communication; regular inspection for biases, misinformation, among other harmful purposes (e.g., identity theft) |
| Cotton et al. [62] | An EdArXiv 2 preprint on the academic integrity in ChatGPT era | - | Risk of plagiarism; academic dishonesty | Careful thinking of educational assessment tools |
| Gao et al. [63] | A bioRxiv 3 preprint comparing the scientific abstracts generated by ChatGPT to original abstracts | A tool to decrease the burden of writing and formatting; it can help to overcome language barriers | Misuse to falsify research; risk of bias | The use of ChatGPT in scientific writing or assistance should be clearly disclosed and documented |
| Polonsky and Rotman [64] | An SSRN 4 preprint on listing ChatGPT as an author | ChatGPT can help to accelerate the research process; it can help to increase accuracy and precision | Intellectual property issues if financial gains are expected | AI 12 can be listed as an author in some instances |
| Aczel and Wagenmakers [65] | A PsyArXiv 5 preprint as a guide of transparent ChatGPT use in scientific writing | - | Issues of originality, transparency issues | There is a need to provide sufficient information on ChatGPT use, with accreditation and verification of its use |
| De Angelis et al. [66] | An SSRN preprint discussing the concerns of an AI-driven infodemic | ChatGPT can support and expedite academic research | Generation of misinformation and the risk of subsequent infodemics; falsified or fake research; ethical concerns | Carefully weigh ChatGPT possible benefits with its possible risks; there is a need to establish ethical guidelines for ChatGPT use; a science-driven debate is needed to address ChatGPT utility |
| Benoit [67] | A medRxiv 6 preprint on the generation, revision, and evaluation of clinical vignettes as a tool in health education using ChatGPT | Consistency, rapidity and flexibility of text and style; ability to generate plagiarism-free text | Clinical vignettes’ ownership issues; inaccurate or non-existent references | ChatGPT can allow for improved medical education and better patient communication |
| Sharma and Thakur [68] | A ChemRxiv 7 preprint on ChatGPT possible use in drug discovery | ChatGPT can help to identify and validate new drug targets; to design new drugs; to optimize drug properties; to assess toxicity; and to generate drug-related reports | Risk of bias or inaccuracies; inability to understand the complexity of biologic systems; transparency issues; lack of experimental validation; limited interpretability; limited handling of uncertainty; ethical issues | ChatGPT can be a powerful and promising tool in drug discovery; however, its accompanying ethical issues should be addressed |
| Rao et al. [69] | A medRxiv preprint on the usefulness of ChatGPT in radiologic decision making | ChatGPT showed moderate accuracy to determine appropriate imaging steps in breast cancer screening and evaluation of breast pain | Lack of references; alignment with user intent; inaccurate information; over-detailed; recommending imaging in futile situations; providing rationale for incorrect imaging decisions; the black box nature with lack of transparency | Using ChatGPT for radiologic decision making is feasible, potentially improving the clinical workflow and responsible use of radiology services |
| Antaki et al. [70] | A medRxiv preprint assessing ChatGPT’s ability to answer a diverse MCQ 8 exam in ophthalmology | ChatGPT currently performs at the level of an average first-year ophthalmology resident | Inability to process images; risk of bias; dependence on training dataset quality | There is a potential of ChatGPT use in ophthalmology; however, its applications should be carefully addressed |
| Aydın and Karaarslan [71] | An SSRN preprint on the use of ChatGPT to conduct a literature review on digital twin in health care | Low risk of plagiarism; accelerated literature review; more free time for researchers | Lack of originality | Expression of knowledge can be accelerated using ChatGPT; further work will use ChatGPT in citation analysis to assess the attitude towards the findings |
| Sanmarchi et al. [72] | A medRxiv preprint evaluating ChatGPT value in an epidemiologic study following the STROBE 9 recommendations | ChatGPT can provide appropriate responses if proper constructs are developed; more free time for researchers to focus on experimental phase | Risk of bias in the training data; risk of devaluation of human expertise; risk of scientific fraud; legal issues; reproducibility issues | Despite ChatGPT possible value, the research premise and originality will remain the function of human brain |
| Duong and Solomon [73] | A medRxiv preprint evaluating ChatGPT versus human responses to questions on genetics | Generation of rapid and accurate responses; easily accessible information for the patients with genetic disease and their families; it can help can health professionals in the diagnosis and treatment of genetic diseases; it could make genetic information widely available and help non-experts to understand such information | Plausible explanations for incorrect answers (hallucination); reproducibility issues | The value of ChatGPT will increase in research and clinical settings |
| Yeo et al. [74] | A medRxiv preprint evaluating ChatGPT responses to questions on cirrhosis and hepatocellular carcinoma | Improved health literacy with better patient outcome; free availability; increased efficiency among health providers; emulation of empathetic responses | Non-comprehensive responses; the limited knowledge up to 2021; responses can be limited and not tailored to specific country or region; legal issues | ChatGPT may serve as a useful aid for patients besides the standard of care; future studies on ChatGPT utility are recommended |
| Bašić et al. [75] | An arXiv preprint on the performance of ChatGPT in essay writing compared to masters forensic students in Croatia | - | Risk of plagiarism; lack of originality; ChatGPT use did not accelerate essay writing | The concerns in the academia towards ChatGPT are not totally justified; ChatGPT text detectors can fail |
| Hisan and Amri [76] | An RG 10 preprint on ChatGPT use medical education | Generation of educational content; useful to learn languages | Ethical concerns; scientific fraud (papermills); inaccurate responses; declining quality of educations with the issues of cheating | Appropriate medical exam design is needed, especially for practical skills |
| Jeblick et al. [77] | An arXiv preprint on ChatGPT utility to simplify and summarize radiology reports | Generation of medical information relevant for the patients; moving towards patient-centered care; cost efficiency | Bias and fairness issues; misinterpretation of medical terms; imprecise responses; odd language; hallucination (plausible yet inaccurate response); unspecific location of injury/disease | Demonstration of the ability of ChatGPT simplified radiology reports; however, the limitations should be considered. Improvements of patient-centered care in radiology could be achieved via ChatGPT use |
| Nisar and Aslam [78] | An SSRN preprint on the assessment of ChatGPT usefulness to study pharmacology | Good accuracy | Content was not sufficient for research purposes | ChatGPT can be a helpful self-learning tool |
| Lin [79] | A PsyArXiv preprint to describe ChatGPT’s utility in academic education | Versatility | Hallucination (inaccurate information that sounds scientifically plausible); fraudulent research; risk of plagiarism; copyright issues | ChatGPT has a transformative long-term potential; embrace ChatGPT and use it to augment human capabilities; however, adequate guidelines and codes of conduct are urgently needed |
1 arXiv: A free distribution service and an open-access archive for scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics, materials on arXiv are not peer-reviewed by arXiv, available from: https://arxiv.org/, accessed on 18 February 2023; 2 EdArXiv: A preprint server for the education research community, available from: https://edarxiv.org/, accessed on 19 February 2023; 3 bioRxiv: A free online archive and distribution service for unpublished preprints in the life sciences, available from: https://www.biorxiv.org/, accessed on 19 February 2023; 4 SSRN: Social Science Research Network repository for preprints, available from: https://www.ssrn.com/index.cfm/en/, accessed on 19 February 2023; 5 PsyArXiv: Psychology archive for preprints, available from: https://psyarxiv.com/, accessed on 18 February 2023; 6 medRxiv: Free online archive and distribution server for complete but unpublished manuscripts (preprints) in the medical, clinical, and related health sciences, available from: https://www.medrxiv.org/, accessed on 18 February 2023; 7 ChemRxiv is a free submission, distribution, and archive service for unpublished preprints in chemistry and related areas, available from: https://chemrxiv.org/engage/chemrxiv/public-dashboard, accessed on 18 February 2023; 8 MCQ: Multiple choice exam; 9 STROBE: Strengthening the reporting of observational studies in epidemiology; 10 RG: ResearchGate: A commercial social networking site for scientists and researchers, available from: https://www.researchgate.net/about, accessed on 19 February 2023; 11 MeSH: Medical Subject Headings; 12 AI: Artificial intelligence.