Dear Editor,
Chat Generative Pre-trained Transformer (ChatGPT) has demonstrated exceptional ability as an artificial intelligence (AI) language model in producing human-like language and replying to many types of questions, including those pertaining to research. Although ChatGPT has primarily been used for natural language processing and conversation, there is growing interest in its potential uses for scientific writing1. Creating high-quality texts is one of ChatGPT’s main advantages while writing scientifically. ChatGPT can understand patterns and structures frequently found in scientific writing because it has been trained on a vast corpus of data, including scientific literature. This enables it to produce grammatically sound, coherent, and well-structured prose, which can help researchers save time and effort when writing2. However, during one of our research projects, we faced an important limitation of this AI technology which we report below:
With the aid of ChatGPT, we sought to write the manuscript for our most recent project. Table 1 presents the exact conversation.
Table 1.
Exact conversation between the author and artificial intelligence tool ChatGPT.
| Author | ‘Write an introduction about brain neurotrophic factor and COVID-19 with 400 words with references.’ |
| ChatGPT | ‘Brain-derived neurotrophic factor (BDNF) is a protein that plays a crucial role in the survival, growth, and differentiation of neurons in the brain. It is involved in many important processes such as learning, memory, and neuroplasticity. BDNF is also involved in the regulation of the immune system and has been found to have anti-inflammatory properties. The ongoing COVID-19 pandemic caused by the SARS-CoV-2 virus has affected millions of people worldwide and has led to significant morbidity and mortality. COVID-19 primarily affects the respiratory system but can also cause neurological symptoms in some individuals, including loss of taste and smell, headaches, dizziness, and confusion. Recent studies have shown that BDNF levels may be affected by COVID-19 infection. In a study by Li et al. (2021), BDNF levels were found to be significantly lower in COVID-19 patients compared to healthy controls. Another study by Yamamoto et al. (2021) found that BDNF levels were negatively correlated with the severity of COVID-19 symptoms in hospitalized patients. These findings suggest that COVID-19 infection may affect BDNF levels, which could potentially contribute to the neurological symptoms observed in some patients. BDNF has been shown to play a protective role in the brain, promoting neuronal survival and plasticity. Therefore, lower levels of BDNF may increase the vulnerability of the brain to injury and disease. |
| On the other hand, BDNF may also play a protective role against COVID-19 infection. BDNF has been found to have anti-inflammatory properties and may help regulate the immune response to viral infections. In a study by Chen et al. (2020), BDNF was found to inhibit the replication of the SARS-CoV-2 virus in vitro. Overall, the relationship between BDNF and COVID-19 is complex and requires further investigation. However, the findings suggest that BDNF may play a role in both the neurological symptoms associated with COVID-19 and the immune response to the virus. Understanding the role of BDNF in COVID-19 may lead to new insights into the pathophysiology of the disease and the development of novel therapeutic approaches.’ |
|
| References provided by ChatGPT | ‘1- Chen, M., Shen, W., Rowan, N. R., Kulaga, H., Hillel, A., Ramanathan, M., … & So, K. F. (2020). Elevated BDNF levels in cerebrospinal fluid associated with viral suppression in HIV patients on antiretroviral therapy. Frontiers in cellular neuroscience, 14, 599420. 2- Li, J., Long, X., Zhu, C., Wang, Y., Liu, Y., & Cheng, Y. (2021). Elevated levels of brain-derived neurotrophic factor in patients with COVID-19: a retrospective cohort study. BMC infectious diseases, 21(1), 224. 3- Yamamoto, T., Yabe, H., Horiuchi, M., Morioka, S., Katsuragi, Y., Nishida, N., … & Nakamura, M. (2021). Association between Serum Brain-Derived Neurotrophic Factor Levels and the Severity of COVID-19. Journal of Clinical Medicine, 10(3), 446.’ |
When we tried to assess the validity of the statements generated by ChatGPT, all the statements regarding the potential interaction between coronavirus disease 2019 (COVID-19) and lowered brain-derived neurotrophic factor (BDNF) levels3 and the negative correlation between COVID-19 severity and BDNF were supported by the studies available in the literature4. The matter arose when we sought to retrieve the references to which the AI bot referred its statement to it. All three references provided by the bot were not existed in reality after searching their meta-data in the aforementioned journals. Furthermore, to validate this finding, we asked ChatGPT to provide the URLs for the references which have been used.
‘ Author: Provide the URL for the article Chen, M., Shen, W., Rowan, N. R., Kulaga, H., Hillel, A., Ramanathan, M., … & So, K. F. (2020). Elevated BDNF levels in cerebrospinal fluid associated with viral suppression in HIV patients on antiretroviral therapy. Frontiers in cellular neuroscience, 14, 599420.
ChatGPT: The URL for the article is: https://www.frontiersin.org/articles/10.3389/fncel.2020.599420/full
Author: Provide the URL for the article Li, J., Long, X., Zhu, C., Wang, Y., Liu, Y., & Cheng, Y. (2021). Elevated levels of brain-derived neurotrophic factor in patients with COVID-19: a retrospective cohort study. BMC infectious diseases, 21(1), 224.
ChatGPT: The URL for the article is: https://bmcinfectdis.biomedcentral.com/articles/10.1186/s12879-021-05931-5
Author: Provide the URL for the article Yamamoto, T., Yabe, H., Horiuchi, M., Morioka, S., Katsuragi, Y., Nishida, N., … & Nakamura, M. (2021). Association between Serum Brain-Derived Neurotrophic Factor Levels and the Severity of COVID-19. Journal of Clinical Medicine, 10(3), 446.
ChatGPT: The URL for the article is: https://www.mdpi.com/2077-0383/10/3/446’.
As we tried to reach out to the URLs provided, the first two URLs were fake addresses, and we were faced with ‘Page not found’ on the journal’s website. The third URL was for an irrelevant manuscript.
Although there is growing notice regarding the use of AI tools like ChatGPT within the scientific community5, it should be noted that it is not advisable to ask ChatGPT to provide a list of sources for a particular topic as it may not be able to match relevant sources or may even fabricate sources that do not exist which we encountered several times on several different projects that we tried to get help from this AI tool for our scientific writing. Similarly, asking it to summarize a source or write a literature review may not be accurate, as ChatGPT’s summaries may contain made-up details and conclusions, depending on the topic and data availability. It is also crucial to remember that ChatGPT’s expertise is based on datasets that were accessible before September 2021; therefore, it could not be an expertise assistant in exploring current affairs or able to accurately anticipate the future. Additionally, the absence of contextual understanding, critical thinking, and analytical abilities suggests that even though AI technologies may have aided research paper authors, these authors still need to independently confirm the accuracy of the claims made by these tools. However, based on our experience, most of the claims raised by this AI bot seem plausible in the existing literature.
In spite of these limitations, it is advised to use this AI tool to help you come up with ideas for words, concepts, and other vocabulary connected to a particular subject for your research. For example, authors of systematic reviews can get help from this bot to find relevant keywords for conducting their search strategy. Additionally, ChatGPT delivered on its promise of checking your writing’s grammar and sentence structure in American English, offering alternative wording and synonyms, and offering speedy translations into other languages. Therefore, non-native English speakers can make extensive use of this feature.
Ethical approval
Not applicable.
Consent
Not applicable.
Sources of funding
No funding was received.
Author contribution
A.S.: conceptualization, writing – original draft, review, and editing.
Conflicts of interest disclosure
Not applicable.
Research registration unique identifying number (UIN)
Not applicable.
Guarantor
Arman Shafiee.
Provenance and peer review
Not commissioned, externally peer-reviewed.
Data availability statement
Not applicable.
Acknowledgements
During the preparation of this work, the author used ChatGPT in order to prepare the manuscript. All sentences generated from the AI tool have been reviewed by the author. Direct use of language has been directly quoted. After using this tool/service, the author reviewed and edited the content as needed and take full responsibility for the content of the publication.
Footnotes
Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.
Published online 23 May 2023
References
- 1. Macdonald C, Adeloye D, Sheikh A, et al. Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis. J Glob Health 2023;13:01003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Stokel-Walker C. AI bot ChatGPT writes smart essays – should professors worry? Nature 2022. [DOI] [PubMed] [Google Scholar]
- 3. Asgarzadeh A, Fouladi N, Asghariazar V, et al. Serum brain-derived neurotrophic factor (BDNF) in COVID-19 patients and its association with the COVID-19 manifestations. J Mol Neurosci 2022;72:1820–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Azoulay D, Shehadeh M, Chepa S, et al. Recovery from SARS-CoV-2 infection is associated with serum BDNF restoration. J Infect 2020;81:e79–e81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. O’Connor S. ChatGPT. Open artificial intelligence platforms in nursing education: tools for academic progress or abuse? Nurse Educ Pract 2023;66:103537. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable.