The use of ChatGPT in medical research: do we need a reporting guideline?

ChatGPT (Chat Generative Pre-trained Transformer) is an artificial intelligence chatbot that utilizes natural language processing to generate conversational dialogue that closely resembles human interactions¹. This language model can respond to queries and compose diverse written content, spanning from articles, social media posts, and essays to code and emails¹. It has also been widely applied in medical research, such as in surgery² and radiology³.

Researchers have employed ChatGPT to draft scientific papers, which have been published^3,4. The use of such tools has increased the researchers’ efficiency and productivity to some extent. However, there may also be some issues and limitations associated with this approach. First, the data used to train large language models like ChatGPT may have a time lag, meaning the generated content may not be up-to-date or accurate. Second, the quality of the questions posed to ChatGPT greatly affects the quality of its responses, and even slightly different prompts can yield inconsistent results⁵. Because of these challenges, authors should not rely on ChatGPT or similar tools alone and should always confirm and critically revise the generated content.

In order to tackle these problems, the World Health Organization Collaborating Centre for Guideline Implementation and Knowledge Translation proposes to develop reporting guidelines for the use of ChatGPT and similar conversational tools in medical research writing. When utilizing ChatGPT or similar chatbots in writing papers or conducting medical research, it is important to adhere to reporting standards and provide detailed information about the process. Our research team has developed a proposed draft list of items and explanations for such guidance based on a review of the literature, as outlined in Table 1. In the next step, we plan to establish an international multidisciplinary expert group to develop a reporting standard for the application of ChatGPT in medical research, abbreviated as CHEER (CHatGPT usEd in mEdical Research), aiming to enhance transparency in the use of chatbots like ChatGPT in medical research.

Table 1.

Potential reporting items for the use of ChatGPT and other similar chatbots in medical research.

1. Did you use ChatGPT or similar software/tools to assist in writing the manuscript?

Explanation: If ChatGPT or similar tools were used during the writing of a research paper, it is suggested to report the details to enhance the transparency of the study.

2. Please specify the AI tool and its version that was used in writing the manuscript.

Explanation: Authors should disclose in the relevant sections of the paper the name (e.g. ChatGPT, Claud, Bard, or NewBing) and the version (e.g. ChatGPT 3.5 or 4.0) of the AI tool(s) that were used.

3. Please report the specific section of the manuscript where ChatGPT or similar tools were used.

Explanation: Authors should explicitly report in the article for which paragraphs or sections ChatGPT (or other similar tool) was used to assist readers in better understanding and assessing the content and value of the paper.

4. Please describe the role of ChatGPT in assisting the manuscript writing process.

Explanation: In addition to reporting which paragraphs were generated with the assistance of ChatGPT, authors should specify the exact role of ChatGPT in the process. This may include, for example, language refinement, outlining ideas, or generating content.

5. Please report whether the content generated by ChatGPT was verified and/or modified. If not, please explain the reason.

Explanation: Content directly generated by ChatGPT may contain false or exaggerated information, so it is recommended to manually proofread or verify the generated content to ensure its accuracy and reliability and revise it if necessary. For example, if ChatGPT was used solely for language refinement, further modifications may be necessary. If no verification was performed, the reason should be clearly stated.

6. Please report whether and how the ChatGPT-generated content can be expected to have influenced the overall conclusions and accuracy of the research.

Explanation: Authors should report whether and how the content generated by ChatGPT may have influenced the results. If ChatGPT was used for language editing correctly, it generally should not affect the content. However, if ChatGPT is used to directly generate content, it may also have an impact on the results.

AI, artificial intelligence; ChatGPT, Chat Generative Pre-trained Transformer.

Ethical approval

Not applicable/not required. This correspondent does not require human or animal subjects to acquire such approval.

Consent

None.

Sources of funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author contributions

X.L.: drafted the initial manuscript; J.E. and Y.C.: provided mentorship and editorial support. All authors read and approved the final draft before submission.

Conflicts of interest disclosure

The authors declare no conflicts of interest.

Research registration unique identifying number (UIN)

1. Name of the registry: not required.

2. Unique identifying number or registration ID: not required.

3. Hyperlink to your specific registration (must be publicly accessible and will be checked): not required.

Guarantor

Yaolong Chen (corresponding author) takes full responsibility for the work and/or the conduct of the study, has access to the data, and controls the decision to publish.

Data availability statement

All data generated or analyzed during this study are included in this published article.

Provenance and peer review

Not commissioned, externally peer-reviewed.