Perception of Chat Generative Pre-trained Transformer (Chat-GPT) AI tool amongst MSK clinicians

Abstract

Background

Chat Generative Pre-trained Transformer (ChatGPT); an open access artificial intelligence (AI) tool has been in the limelight with its ability to respond to prompts, analyse data information using algorithms to augment efficiency in day-to-day activities across a spectrum of human activities including MSK/Orthopaedic science.

Purpose of the study

The purpose of this cross-sectional survey has been to analyse the knowledge, understanding of the role of Chat Generative Pre-trained Transformer (ChatGPT) and its implications in clinical practice as well as research in medicine.

Material & methods

An online cross-sectional survey of 10 questions (multiple choice and free text) was circulated amongst orthopaedic surgeons, musculoskeletal radiologists and Rheumatologists in India and UK, to evaluate perception of Chat Generative Pre-trained Transformer (ChatGPT) AI Tool.

Results

We had 125 responses with majority being aware of ChatGPT though a minority had used it. There was consensus that its going have detrimental effect on workforce with majority of the opinion that they would be used to create radiology reports. Mixed responses were noted regarding the quality of research and role of ChatGPT being an anonymous author.

Conclusion

There is a considerable debate amongst clinicians of orthopaedic, radiology and rheumatology -specialities. The attitudes are mixed but mainly positive, although there are many concerns about the still-evolving new technology.

Level of study

Diagnostic Study level 4.

1. Introduction

Technological innovations led by the industry 4.0 and 5.0 Revolution (2011) through the integration of Artificial Intelligence (AI) and Human Intelligence are transforming healthcare delivery to make us more productive and efficient.^1,2

Chat Generative Pre-Trained Transformer (ChatGPT) is an artificial intelligence (AI)-based chatbot developed by Open AI, which uses Language Processing Models to generate responses to prompts in a colloquial way.³ It has been designed on the pre-existing OpenAI's GPT-3 family of language processing model. This AI based technology can answer queries in a text or written output following Data acquisition from various sources (e.g., Websites, books etc.), analysis and fine-tuning, using both supervised and reinforcement learning algorithms.

In the field of medical science and healthcare delivery, ChatGPT has been shown to streamline patient discharge summaries and preparation of AI-generated Radiology reports.^4,5 The aims of these applications have been to facilitate productivity, improve efficiency, and free doctors and clinicians from automated tasks. However, the study involving ChatGPT-generated radiology reports showed errors and oversight of significant radiological findings. Therefore, patient discharge summaries may come with the omission of vital medical follow-up requirements.

We have conducted an online survey amongst Orthopaedic surgeons, Radiologists, Rheumatologists and other specialty clinicians about their knowledge and understanding of the roles of ChatGPT and its implications in clinical practice as well as research in medicine.

2. Material and methods

2.1. Study design

This cross-sectional study (online questionnaire survey) was conducted over a 2-week period amongst Orthopaedic surgeons, Radiologists, Rheumatologists and other speciality clinicians.

An online questionnaire was developed, with both multiple choice and free text questions, using the Google Forms platform to maximise ease of administration, data handling, and facilitate fast participant response through electronic mail (Supplementary file 1). The target population were clinicians involved in Orthopaedic, Radiology and Rheumatological practice. The survey was kept open for response submission for 3 weeks, with a reminder notification sent halfway through, after which submitted responses were analysed and summarised accordingly.

2.2. Inclusion and exclusion criteria

The questionnaire was circulated using social media platforms in orthopaedic, rheumatology and radiology forums. Non clinicians were excluded from the study.

2.3. Data collection

The responses submitted were checked for duplication, pooled, analysed, and summarised.

2.4. Evaluation points

The focus of the survey was on the following points.

• 1)Knowledge/Usage of Chat Generative Pre-trained Transformer (ChatGPT)
• 2)Role of Chat Generative Pre-trained Transformer (ChatGPT) amongst clinicians
• 3)Effect on Chat Generative Pre-trained Transformer (ChatGPT Scientific & Academic Publishing
• 4)Effect Chat Generative Pre-trained Transformer (ChatGPT) on Value and quality of Research
• 5)Work force implications
• 6)Concerns

2.5. Statistical methods

Excel data sheet was used for data collection. Descriptive analysis was performed to summarise the data.

2.6. Funding of study

No funding was received from any individual or institution for this study.

2.7. Ethics statement

Institutional review board approval was not required for this survey-based study.

3. Results

The survey was completed by 125 clinicians. All responses received were used in the analysis. The predominant responses were received from the Radiologists (52.5%) follower by orthopaedic surgeon and rheumatologists. The majority of clinicians (82.7%) who responded to the survey admitted being aware of ChatGPT, AI tool with 28.7% of the respondents having used the ChatGPT in their daily practice (Fig. 1, Fig. 2). More than half (56.6.%) felt that ChatGPT will be increasingly used to generate Radiology Reports in the future (Fig. 3). Nearly half of the surveyed clinicians (43.8% saying ‘Yes’ whilst 44.6% unsure) felt that Chat Generative Pre-trained Transformer (ChatGPT) will feature as anonymous co-authors in the future Scientific & Academic Publishing. There was an almost even split with 52.5% supporting the fact that quality of research would be decreased and 47.5.2% felt that the quality of research would be enhanced. (Fig. 4, Fig. 5). There was a feeling amongst the respondents that ChatGPT will have effect on work force planning with implication of job security in the health care sector (Fig. 4). Some of the respondents had strong views with ChatGPT effecting write paper and alteration of author guidelines. There were concerns by several respondents about the inaccuracies of the articles and answers generated using the tool. Some also raised concerned about the legality around the reports and documents created.

Fig. 1.

Graph A showing the specialty of the respondent. Graph B showing awareness of ChatGPT.

Fig. 2.

Graph A showed the number of respondents who use ChatGPT. Graph B showing the responses when asked if ChatGPT would play a crucial role in medicine.

Fig. 3.

Graph A shows the spread of results if tool would be used to create radiology reports and graph B showing is ChatGPT would be anonymous author.

Fig. 4.

Graph A showing the results if ChatGPT would decrease work force or the value of research (graph B).

Fig. 5.

Graph showing the results of the if ChatGPT would enhance quality of research.

4. Discussion

The cross-sectional survey suggests that the recently introduced ChatGPT by the Open AI organisation has generated significant interest amongst Radiology, Orthopaedics and Rheumatology medical sub-specialities clinicians. This emphasizes the influence of Innovative and newer Digital technology in the delivery of healthcare and the awareness amongst the clinicians about emerging technologies.

Data from the respondents show that ChatGPT will find increasing use in Radiology. This is no surprise as Artificial Intelligence (AI) has been shown useful in other areas of medicine. For example, in Surgery, it has been projected that AI could find a practical application in the development of intelligent surgical robots that can assist surgeons in highly complex surgeries of the nervous and cardiovascular systems.⁶

While opinions are divided on the future incorporation of ChatGPT into academic authorship, concerns have been previously expressed about citing ChatGPT as a co-author. Although there is currently no consensus by medical or scientific publishers or authors, it seems the bulk of the criticisms against the use of non-human authors like ChatGPT rests on the fact that they cannot tick all the boxes traditionally required for authorship to be merited.^7,8 ChatGPT will likely influence or force some shift in the academic publishing landscape, much like open access has done to traditional scientific publishing.

The generality of opinions is that ChatGPT will reduce the quality of academic research. There is no doubt that ChatGPT could aid in medical writing,⁹ but this aid could come at the cost of not helping future clinicians and researchers to develop the human capacity to analyse a problem and come up with the appropriate research questions. For example, when junior faculties and residents were asked about their most significant challenge, they responded that they lacked knowledge about where to start their research writing. We could have a future where better “skilled” AI will replace less skilled humans, leading to job losses. Indeed, many respondents in this survey believed that ChatGPT could lead to job losses, particularly in the health sector.

5. Conclusion

Our study shows that orthopaedic surgeons, radiologists, rheumatologists, and other speciality clinicians are knowledgeable about ChatGPT. In addition, the attitudes are primarily positive, although there are many concerns about the still-evolving new technology.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:KP and RB are on editorial board of JCOT

Appendix A. Supplementary data

The following are the Supplementary data to this article: