Commentary to Corti et al.: Exploring the utility and limitations of ChatGPT in scientific literature searches

We read with interest the Commentary by Corti and colleagues¹ in the first issue of this journal, entitled ‘Exploring the utility and limitations of ChatGPT in scientific literature searches’. Although discussing the topic of artificial intelligence (AI) and large language models (LLMs) is timely, the commentary by Corti et al.¹ reveals an incomplete appreciation of the usability of these tools. As with any other tool, we as users must learn about the principles of the tool, its capabilities, and its limitations. Finally, we can adjust our expectations and acquire the skills to harness the full potential of the tool of interest.

ChatGPT, trained on a large number of texts, generates responses to questions (prompts) by predicting the next probable word based on its training and general language rules.² Herein lies the key to understanding the tool: it has no intention, no reasoning: it is only putting symbols together. Using words such as ‘forge’ and ‘fabricate’ to describe its output seems like an inappropriate anthropomorphizing which serves to cause skepticism. However, we agree with Corti et al.¹ that users must remain vigilant in their use of ChatGPT output.

At the time when the paper by Corti et al.¹ was written and published, ChatGPT did not have the ability to search the internet. Hence, asking for factual information, such as precise scientific references, would be futile. However, during mid-autumn 2023, this feature was added to ChatGPT-4, albeit in the plus version.³ This serves as an example of the need to know the principles of the tool one wants to use and keep up with the fast-evolving development of AI technology.

The adequate use of ChatGPT in research can enhance efficiency in several ways. ChatGPT can assist researchers in brainstorming and expanding upon initial concepts, fostering creative solutions and novel research angles. For statistical analysis, ChatGPT’s proficiency in programming languages such as R and Python enables it to aid in the coding of statistical models, interpreting outputs, and troubleshooting code-related issues.² One main issue, however, is the opportunity for individuals to use AI and LLMs to forge several steps of the research process, including creating fake datasets and research papers.^4,5 ChatGPT is well-suited for language editing: correcting grammatical errors, improving sentence structure, and ensuring clarity of expression.² Its ability to work with texts, including summarizing lengthy documents, transforming narratives into bullet points, and tailoring content to suit diverse audiences, is particularly useful. Importantly, researchers are always responsible for their own statements, and plagiarizing, as in presenting text from any source as their own, is unacceptable.

Importantly, ChatGPT is only a machine. It does not know your role, your setting, or your intention unless you provide that information. Hence, practicing your prompting skills is imperative. In summary, we recommend researchers to make themselves familiar with AI tools and keep up to date. Researchers must adjust their expectations according to the capabilities of the tools and share their reflections on practical and ethical aspects.