Abstract
Objective
Large language models and artificial intelligence (AI) based chatbots have brought new insights in healthcare, but they also raise major concerns. Their applications in vascular surgery have scarcely been investigated to date. This international survey aimed to evaluate the perceptions and feedback from vascular surgeons on the use of AI chatbots in vascular surgery.
Methods
This international open e-survey comprised 50 items that covered participant characteristics, their perceptions on the use of AI chatbots in vascular surgery, and their user experience. The study was designed in accordance with the Checklist for reporting Results of Internet E-Surveys and was critically reviewed and approved by international members of the European Vascular Research Collaborative (EVRC) prior to distribution. Participation was open to self reported health professionals specialised (or specialising) in vascular surgery, including residents or fellows.
Results
Of the 342 individuals who visited the survey page, 318 (93%) agreed to participate; 262 (82.4%) finished the survey and were included in the analysis. Most were consultants or attending physicians (64.1%), most declared not having any training or education related to AI in healthcare (221; 84.4%), and 198 (75.6%) rated their knowledge about the abilities of AI chatbots between average to very poor. Interestingly, 95 participants (36.3%) found that AI chatbots were very useful or somewhat useful in clinical practice at this stage and 229 (87.4%) agreed that they should be systematically validated prior to being used. Eighty participants (30.5%) had specifically tested it for questions related to clinical practice and 59 (73.8%) of them experienced issues or limitations.
Conclusion
This international survey provides an overview of perceptions of AI chatbots by vascular surgeons and highlights the need to improve knowledge and training of health professionals to better evaluate, define, and implement their use in vascular surgery.
Keywords: Artificial intelligence, Chatbot, Large language model, Natural language processing, Survey, Virtual assistant
Highlights
-
•
Artificial intelligence (AI) chatbots have brought new insights into vascular surgery.
-
•
This international open survey investigated feedback from vascular surgeons.
-
•
Surveyed participants highlighted interests but also current limitations.
-
•
Training is mandatory to help define the use of AI chatbots in clinical practice.
Introduction
Large language models (LLM) correspond to a field of artificial intelligence (AI) aiming to mimic human language processing abilities and have been used to empower and develop new generations of chatbots. These are computer programs that process language and allow humans to interact with digital devices as if they were communicating with another human being.1 The release of AI chatbots with open access to the public has become a highly scrutinised topic in the media due to its potential use as a virtual assistant or source of knowledge. However, it raises many concerns in healthcare.1,2 There are various potential applications of LLM and chatbots in healthcare and vascular surgery in particular, including: applications in patient care by providing medical knowledge, facilitating patient empowerment, and assistance in writing, translating, and summarising medical content.2 It can also be used for education, with interactive learning and opportunities to develop personalised education, and research by facilitating access to scientific knowledge, writing, communication, or the production of scientific content.1,3,4 At the same time, this generates new challenges with respect to safety, ethical and legal concerns. This international survey aimed to evaluate the perceptions and feedback from vascular surgeons on the use of chatbots in vascular surgery.
Methods
Recruitment of participants
This international open survey was designed in accordance with the Checklist for reporting Results of Internet E-Surveys.5 Participation was open to self reported health professionals specialised (or specialising) in vascular surgery, including residents or fellows. The anonymous e-survey was accessible via the web based company SurveyMonkey (www.surveymonkey.com). Participants were contacted and informed about the possibility of participating through mailing lists using the European Vascular Research Collaborative (EVRC) network via links posted on social media (from authors and EVRC members on social media accounts on LinkedIn and X [formerly Twitter]). The link to participate in the e-survey was accessible between 28 November 2023 and 28 February 2024, and three calls for participation were launched within this three month period. All participants were informed about the aim of the survey, its design, data collected, and expected benefits. Informed consent was obtained before completing the survey. Participation was voluntary and anonymous and the identity of participants was not collected.
Content of the survey
The questionnaire was designed by experts in AI and in vascular surgery (JR, FL, MDO) after review of current knowledge and literature on the use of Chatbots and Natural Language Processing (NLP) in vascular surgery and healthcare.2,6, 7, 8 The survey was critically reviewed by international members of the EVRC and final approval by EVRC members was obtained prior to distribution of the survey. The survey comprised 50 items covering three main fields: information on participant characteristics, perceptions on the use of AI chatbots in vascular surgery, and user experience with this technology. The questions explored the perception and use of AI chatbots by specialists in vascular surgery in the context of clinical practice and medical research and education. The survey was written in English and also translated into French. The English version of the survey was systematically used, except for participants working in France who used the French version. The time to complete the survey was estimated between 5 – 10 minutes. Adaptive questioning was used, and participants had the ability to review and change their answers before final validation. Whenever it was possible, questions with one response option were used.
Data analysis
Descriptive statistics were used to analyse the results. Categorical variables were expressed using frequency distributions and proportions. Consistency and completeness of the responses to the survey were checked and data analysis was adjusted following handling of incomplete questionnaires. The participation rate was defined as the ratio of unique visitors who agreed to participate/unique first survey page visitors, and completion rate as the ratio of users who finished the survey/users who agreed to participate. The IP address of each respondent's computer was used to identify any potential duplicate entries from the same user. Criteria used to identify a same user were pre-defined as duplicate entries having the same IP address within <24 hours, with the same characteristics in terms of age, sex, and number of years of practice in vascular surgery. Stratification by baseline characteristics was performed to identify potential trends or differences between participants who had used AI chatbots and those who had not. Group differences were analysed using Fisher's exact test and p < .05 was considered statistically significant. Statistical analyses were performed using GraphPad Prism® software (version 10.00, La Jolla, CA, USA).
Results
Characteristics of participants
Three hundred and forty-two individuals visited the survey page and 318 agreed to participate in the survey, leading to a participation rate of 93.0%. Of the 318 participants, 262 finished the survey (completion rate of 82.4%) and no duplicate entry was identified. The cohort comprised 185 men (70.6%) and most participants were consultants or attending physicians (64.1%); 72.1% worked in a university hospital (Table 1). The distribution regarding the number of years of vascular surgery practice showed that all categories were represented among participants, from 0 – 5 to >20 years of experience (Table 1). The participants came from 23 different countries and the most represented countries were France (79 participants; 30.2%), Portugal (25; 9.5%), Italy (22; 8.4%), and United Kingdom (20; 7.6%). Most of the participants declared not having had any training or education related to AI in healthcare (221; 84.4%). A total of 198 (75.6%) participants rated their knowledge about the abilities of AI chatbots between average to very poor.
Table 1.
Characteristics of the participants in the international survey on the interest and use of artificial intelligence (AI) chatbots in vascular surgery.
| Characteristics of participants (n = 262) | n (%) |
|---|---|
| Age – years | |
| <30 | 32 (12.2) |
| 31–40 | 124 (47.3) |
| 41–50 | 59 (22.5) |
| 51–60 | 31 (11.8) |
| >60 | 16 (6.1) |
| Sex | |
| Male | 185 (70.6) |
| Female | 77 (29.4) |
| Position | |
| Consultant or attending physician | 168 (64.1) |
| Fellow | 67 (25.6) |
| Resident doctor | 20 (7.6) |
| Not specified | 7 (2.7) |
| Country | |
| France | 79 (30.2) |
| Portugal | 25 (9.5) |
| Italy | 22 (8.4) |
| United Kingdom | 20 (7.6) |
| Austria | 19 (7.3) |
| Netherlands | 18 (6.9) |
| Finland | 12 (4.6) |
| Sweden | 12 (4.6) |
| Germany | 10 (3.8) |
| Other | 45 (17.2) |
| Number of years of practice of vascular surgery (starting from the beginning of residency) – years | |
| 0–5 | 71 (27.1) |
| 6–10 | 62 (23.7) |
| 11–15 | 51 (19.5) |
| 16–20 | 34 (13.0) |
| >20 | 44 (16.8) |
| Type of health institute∗ | |
| University hospital | 189 (72.1) |
| Regional hospital | 52 (19.8) |
| Private healthcare institution | 41 (15.6) |
| Past training or course related to AI in healthcare∗ | |
| University degree or university course in AI | 5 (1.9) |
| Research activity in AI | 20 (7.6) |
| Attendance to conferences in AI | 20 (7.6) |
| No training or course | 221 (84.4) |
| Rate your knowledge about the abilities of AI chatbots | |
| Very good | 8 (3.1) |
| Good | 56 (21.4) |
| Average | 92 (35.1) |
| Poor | 79 (30.2) |
| Very poor | 27 (10.3) |
| Rate how experienced you are in the use of AI chatbots | |
| Very experienced | 7 (2.7) |
| Somewhat experienced | 41 (15.6) |
| Average | 77 (29.4) |
| Somewhat inexperienced | 76 (29.0) |
| Very inexperienced | 61 (23.3) |
Multiple answers possible.
Perceptions of AI chatbots by surveyed vascular surgeons
The participants evaluated which aspects of vascular surgery and research they thought could benefit from the use of AI (Table 2). Interestingly, 95 participants (36.3%) believed that AI chatbots are already very useful to somewhat useful for clinical practice, and 212 (80.9%) believed that they will be very useful to somewhat useful in the future. In addition, 213 participants (81.3%) thought that AI chatbots could help save time; 198 respondents (75.6%) thought it likely or very likely that patients should use AI chatbots for their current medical disease; 161 participants (61.5%) believed that the use of AI chatbots could very probably or probably cause harm to patients (Table 2), and 128 participants (48.9%) believed that AI chatbots could partially replace human expertise, while 106 (40.5%) did not (Table 2). A total of 229 participants (87.4%) thought that AI chatbots should be systematically validated prior to being used for clinical practice. In the field of research, participants believed that AI chatbots could be mainly useful for literature searches and writing tasks such as translating or correcting grammar or typography (Table 2).
Table 2.
Responses of the participants regarding their perception of artificial intelligence (AI) chatbots in vascular surgery.
| Responses (n = 262) | n (%) |
|---|---|
| What aspects of vascular surgery do you think could benefit from the use of AI chatbots?∗ | |
| Diagnostic | 141 (53.8) |
| Treatment | 86 (32.8) |
| Follow up of patients | 148 (56.5) |
| Education of patients | 205 (78.2) |
| Research | 154 (58.8) |
| Administrative tasks | 191 (72.9) |
| Telemedicine | 93 (35.5) |
| None | 1 (0.4) |
| What is your opinion on the usefulness of AI chatbots for clinical practice at this moment? | |
| Very useful | 21 (8.0) |
| Somewhat useful | 74 (28.2) |
| Neutral | 105 (40.1) |
| Somewhat useful | 33 (12.6) |
| Not useful at all | 29 (11.1) |
| Currently, what is your level of trust in the ability of AI chatbots to provide accurate clinical information? | |
| Very high | 6 (2.3) |
| Somewhat high | 65 (24.8) |
| Neutral | 92 (35.1) |
| Somewhat low | 73 (27.9) |
| Very low | 26 (9.9) |
| How likely do you think patients are to use AI chatbots for their current medical disease (i.e., for self diagnosis, self medication, or self treatment)? | |
| Very likely | 84 (32.1) |
| Likely | 114 (43.5) |
| Neutral | 30 (11.5) |
| Unlikely | 27 (10.3) |
| Very unlikely | 7 (2.7) |
| How likely is it that the use of AI chatbots could cause harm to patients? | |
| Very likely | 54 (20.6) |
| Likely | 107 (40.8) |
| Neutral | 86 (32.8) |
| Unlikely | 12 (4.6) |
| Very unlikely | 3 (1.1) |
| How likely do you think it is that patients will accept the use of AI chatbots by health professionals? | |
| Very likely | 28 (10.7) |
| Likely | 126 (48.1) |
| Neutral | 67 (25.6) |
| Unlikely | 39 (14.9) |
| Very unlikely | 2 (0.8) |
| For what purpose in research do you think AI chatbots could be useful?∗ | |
| Generate ideas for research | 117 (44.7) |
| Write manuscript | 129 (49.2) |
| Correct grammar or typography | 219 (83.6) |
| Translation to other languages | 220 (84) |
| Literature research | 219 (83.6) |
| Do you think that AI chatbots might somehow replace human expertise? | |
| Yes totally | 8 (3.1) |
| Yes partially | 128 (48.9) |
| No | 106 (40.5) |
| Not sure | 20 (7.6) |
| Do you think that AI chatbots should be systematically validated prior to being used for clinical practice? | |
| Strongly agree | 146 (55.7) |
| Agree | 83 (31.7) |
| Neutral | 16 (6.1) |
| Disagree | 5 (1.9) |
| Strongly disagree | 12 (4.6) |
Multiple answers possible.
User experience of AI chatbots by surveyed vascular surgeons
Regarding user experience, 187 participants (71.4%) had used or tested an AI chatbot; the main software explored were ChatGPT (OpenAI) (89.8%), Bing (Microsoft) (22.5%), and Bard (Google) (12.3%). Of the 187 participants, 80 (30.5%) had specifically tested it for questions related to clinical practice (Table 3); among them, 59 (73.8%) experienced issues or limitations, which are detailed in Table 3. When comparing characteristics between participants who had tested or used an AI chatbot for questions related to clinical practice and those who had not, a higher proportion of affiliation to a university hospital was observed in users (81.3% vs. 68.1%, p = .036) (Table 4). In addition, the proportion of participants who had previous training related to AI (e.g., attended conferences on AI, research activity in AI), was statistically significantly higher among users compared with non-users (Table 4). No statistically significant difference was observed regarding age or number of years of practice in vascular surgery between the two groups (Table 4).
Table 3.
Participant responses regarding their user experience of artificial intelligence (AI) chatbots in vascular surgery. Only participants who have ever used or tested an AI chatbot for the specific indications were included in the analysis.
| Responses | n (%) |
|---|---|
| Have you ever used or tested an AI chatbots? | |
| Yes | 187 |
| Which AI chatbot have you ever used or tested?∗ | |
| ChatGPT (OpenAl) | 168 (89.8) |
| Bing (Microsoft) | 42 (22.5) |
| Bard (Google) | 23 (12.3) |
| Med-Palm (Google) | 3 (1.6) |
| Other | 11 (5.9) |
| Have you ever used or tested an AI chatbots for questions related to clinical practice? | |
| Yes | 80 |
| When did you use or test an AI chatbot for questions related to clinical practice? | |
| Before 2021 | 1 (1.3) |
| 2021 | 8 (10.0) |
| 2022 | 25 (31.3) |
| 2023 | 45 (56.3) |
| 2024 | 1 (1.3) |
| How often do you use AI chatbots in your clinical practice? | |
| Daily | 3 (3.8) |
| Weekly | 11 (13.8) |
| Monthly | 6 (7.5) |
| Rarely | 46 (57.5) |
| Never | 14 (17.5) |
| Have you experienced any issues or limitations using AI chatbots for clinical practice? | |
| Yes | 59 (73.8) |
| If yes, what kind of issues or limitations have you experienced?∗ | |
| Inappropriate answers from the AI chatbots | 35 (43.8) |
| Inaccurate answers from the AI chatbots | 36 (45) |
| Lack of specificity of the answers | 51 (63.8) |
| Difficulty interpreting responses | 8 (10) |
| Variability in the quality of responses | 38 (47.5) |
| Answers unavailable | 11 (13.8) |
| Lack of transparency on where the information came from | 28 (35) |
| Have you ever used or tested an AI chatbot for scientific or academic writing? | |
| Yes | 64 |
| What is your opinion on the usefulness of AI chatbots for scientific or academic writing? | |
| Very useful | 8 (12.5) |
| Somewhat useful | 21 (32.8) |
| Neutral | 13 (20.3) |
| Somewhat not useful | 13 (20.3) |
| Not useful at all | 9 (14.1) |
| Have you experienced any issues or limitations using AI chatbots for scientific or academic writing? | |
| Yes | 57 (89.1) |
| If yes, what kind of issues or limitations have you experienced?∗ | |
| Concerns about plagiarism | 29 (45.3) |
| Concerns about the quality or accuracy of the scientific content generated | 55 (85.9) |
| Concerns about lack of transparency | 21 (32.8) |
| Concerns about authorship and accountability | 33 (51.6) |
| Concerns about dependence on technology over human expertise | 18 (28.1) |
| Have you ever used or tested an AI chatbot for searching bibliography or scientific content? | |
| Yes | 64 |
| What is your opinion on the usefulness of AI chatbots for literature search? | |
| Very useful | 11 (17.2) |
| Somewhat useful | 27 (42.2) |
| Neutral | 9.0 (14.1) |
| Somewhat not useful | 12 (18.8) |
| Not useful at all | 5.0 (7.8) |
% are expressed per subgroups.
Multiple answers possible.
Table 4.
Analysis of differences between participants who had ever tested or used an artificial intelligence (AI) chatbot for questions related to clinical practice and those who had not.
| Characteristics | Users (n = 80) | Non-users (n = 182) | p value |
|---|---|---|---|
| Age – years | |||
| <30 | 10 (12.5) | 22 (12.1) | >.99 |
| 31–40 | 39 (48.8) | 85 (46.7) | .79 |
| 41–50 | 19 (23.8) | 40 (22.0) | .75 |
| 51–60 | 8.0 (10.0) | 23 (12.6) | .68 |
| >60 | 4.0 (5.0) | 12 (6.6) | .78 |
| Sex | |||
| Male | 69 (86.3) | 116 (63.7) | <.001 |
| Female | 11 (13.8) | 66 (36.3) | <.001 |
| Position | |||
| Consultant or attending physician | 50 (62.5) | 118 (64.8) | .78 |
| Fellow | 22 (27.5) | 45 (24.7) | .65 |
| Resident doctor | 6 (7.5) | 14 (7.7) | >.99 |
| Not specified | 2 (2.5) | 5 (2.7) | >.99 |
| Number of years of practice of vascular surgery (starting from the beginning of residency) – years | |||
| 0–5 | 23 (28.8) | 48 (26.4) | .76 |
| 6–10 | 18 (22.5) | 44 (24.2) | .87 |
| 11–15 | 17 (21.3) | 34 (18.7) | .61 |
| 16–20 | 10 (12.5) | 24 (13.2) | >.99 |
| >20 | 12 (15.0) | 32 (17.6) | .72 |
| Type of health institute∗ | |||
| University hospital | 65 (81.3) | 124 (68.1) | .036 |
| Regional hospital | 13 (16.3) | 39 (21.4) | .40 |
| Private healthcare institution | 9 (11.3) | 32 (17.6) | .27 |
| Past training or course related to AI in healthcare∗ | |||
| University degree or university course in AI | 3 (3.8) | 2.0 (1.1) | .17 |
| Research activity in AI | 13 (16.3) | 7 (3.8) | .002 |
| Attendance to conferences in AI | 13 (16.3) | 7 (3.8) | .006 |
| No training or course | 54 (67.5) | 167 (91.8) | <.001 |
| Rate your knowledge about the abilities of AI chatbots | |||
| Very good | 6 (7.5) | 2.0 (1.1) | .011 |
| Good | 28 (35) | 28 (15.4) | <.001 |
| Average | 31 (38.8) | 61 (33.5) | .48 |
| Poor | 14 (17.5) | 65 (35.7) | .003 |
| Very poor | 1.0 (1.3) | 26 (14.3) | <.001 |
| Rate how experienced you are in the use of AI chatbots | |||
| Very experienced | 6 (7.5) | 1 (0.5) | <.001 |
| Somewhat experienced | 24 (30.0) | 17 (9.3) | <.001 |
| Average | 27 (33.8) | 50 (27.5) | .31 |
| Somewhat inexperienced | 20 (25) | 56 (30.8) | .38 |
| Very inexperienced | 3 (3.8) | 58 (31.9) | <.001 |
Multiple answers possible.
Sixty four participants (24.4%) had tested AI chatbots for searching bibliographies or scientific content; among them, 39 (60.9%) found it very useful to somewhat useful (Table 3). Sixty four participants (24.4%) had also tested AI chatbots for scientific or academic writing, of whom 29 (45.3%) found it very useful to useful (Table 3). However, 57 (89.1%) experienced issues or limitations when using AI chatbots in that setting, including concerns about the quality or accuracy of the generated content, concerns about plagiarism, authorship, accountability, lack of transparency, and concerns about dependence on technology over human expertise (Table 3).
Discussion
This international European survey is one of the first to address surgeons’ views regarding the use of AI chatbots in vascular surgery. While the results indicate that most surveyed vascular surgeons believed that AI chatbots could be useful for clinical practice and medical research, they also highlighted concerns regarding limitations and the risk that it could be harmful for patients if this technology is not used appropriately.
The potential use of chatbots in healthcare raises important ethical concerns.7, 8, 9 The first one relates to the accuracy of the generated content, as LLMs are dependent on the training dataset that is used.10,11 Inaccurate, biased, or outdated sources can lead to errors in the generated content.10,11 In addition, neural hallucination can sometimes be observed, meaning that the models can generate output that is inaccurate or nonsensical despite appearing reliable.12, 13, 14 In this survey, health professionals highlighted some main issues when using AI chatbots for clinical practice, including: inappropriate or inaccurate answers of the AI chatbots, lack of specificity, difficulty interpreting responses, variability in the quality of responses, answers unavailable, and lack of transparency on the source of the information. This study highlights the importance of human expertise from health professionals and the need to accompany and warn patients about potential pitfalls that could be harmful in case of inacurracy or misunderstanding of the information provided. More than 85% of the surveyed participants agreed that such tools should be systematically validated before being implemented in clinical practice.
Similar work in other medical specialties has recognised the potential of AI chatbots and LLMs in clinical research, but has also raised similar concerns regarding ethics and patient acceptance.15 In the field of vascular surgery, a recent pilot study aimed to evaluate the accuracy of two commonly used chatbots for common questions that an aortic patient might ask.16 The authors found significant risk of inaccurate or incomplete responses and underlined that care should be taken to ensure that such innovation is used safely for the patient's best interests. There is certainly a strong need for regulations to ensure appropriate use, and measures should be taken to establish rules and guidelines to maximise safety and efficiency when using this novel technology.17 Evaluation and validation of AI chatbots for medical use is extremely challenging and methods to standardise it are still to be defined.2,17, 18, 19, 20
In addition to accuracy and safety, other major ethical concerns are at stake, including questions related to health data protection, equity and fairness, transparency, responsibilities and accountability, and acceptability and integration by patients and healthcare professionals of such innovation.7,8 This study showed that 80% of surveyed vascular surgeons believed that AI chatbots will be very useful to somewhat useful for clinical practice in the future. However, at the moment, 75% of the participants rated their knowledge about the abilities of AI chatbots between average to very poor and 30% had tested or used on AI chatbot for questions related to clinical practice. These results highlight a current gap between future implementation and application of AI chatbots in clinical practice and current training of health professionals. This should encourage the building of educational programs that are accessible to all vascular specialists in public and private health institutions.
In the field of research and academic work, LLMs and AI chatbots have brought new techniques to enable information retrieval that could be leveraged to facilitate literature searches and perform systematic reviews.2,19,21 In this survey, 83.6% of participants believed that such innovation could be useful for literature searches. The use of AI based software for literature searches in vascular surgery has been scarcely investigated so far. A pilot study recently tested its use for literature searches on a topic specifically related to vascular surgery and suggested its potential interest to enable a fast and focused selection of relevant articles.21 Although further studies are required, such an approach could be complementary to the traditional literature search performed by humans and could contribute to dissemination of scientific knowledge. Finally, LLMs have also been proposed for scientific writing to assist various tasks including editing, summarising, translating, or generating scientific content.13,22 However, as highlighted by the surveyed participants, this raises major ethical concerns related to accuracy, transparency, accountability, and authorships. Most scientific journals have now stipulated that AI cannot be cited as co-authors and that authors should declare and acknowledge if AI was used for manucript preparation.
Finally, it should be noted that LLMs are rapidly evolving over time. The Pre-training Transformer (GPT) model designed by OpenAI (ChatGPT) has been highly mediatised and was used by almost 90% of the participants who had tested or used an AI chatbot. Nevertheless, various other LLMs have been developed over the past few years, some of them specifically designed for medical purposes.23 This offers new insights that could change the vascular surgeon's perspectives.
This study had several limitations. Selection bias could have affected the representivity of the sample. Calls for participation were communicated through mailing lists and posted on social media; therefore, they were internationally accessible. Although participants came from 23 different countries, most of them originated from European countries and were working in university hospitals. There was also the possiblity that respondents to the survey may have been more interested or more prone to use AI chatbots than non-respondents. Nevertheless, the participation and completion rates were within acceptable ranges and this survey provides an overview of current knowledge and perception on the use of AI chatbots in vascular surgery, which has been scarcely investigated to date.
Conclusion
This international open survey has highlighted the potential interest of AI chatbots in vascular surgery but also shown concerns raised by vascular surgeons. The study also underlined the need to improve knowledge and training of health professionals on AI based tools to help them evaluate interests and pitfalls in order to define their use for clinical practice and medical research.
Conflict of interest
None.
Funding
This work was supported by the French government through the National Research Agency (ANR) with the reference number ANR-22-CE45-0023-01 and through the 3IA Côte d'Azur Investments in the Future project, managed with reference number ANR-19-P3IA-0002.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.ejvsvf.2024.07.037.
Contributor Information
Fabien Lareyre, Email: fabien.lareyre@gmail.com.
Study group collaborators: European Vascular Research Collaborative (EVRC):
Florian Enzmann, Gert J. de Borst, Joel Ferreira Sousa, Lewis Meecham, Liliana Domingos, Martin Teraa, Petar Zlatanovic, Salome Weiss, Stefano Ancetti, Albert Busch, Bergrós Jóhannesdóttir, Alexander Gombert, Katariina Noronen, Robert Hinchliffe, Alexandru Predenciuc, Panagiotis Doukas, Leszek Kukulski, Qasam Ghulam, Angelos Karelis, Maram Darwish, Mohammad Esmaeil Barbati, Markvard Møller, Matt Spreadbury, Willemien van de Water, Desiree van den Hondel, Harm Ebben, Alexander Croo, Gilles Uijtterhaegen, Adina Trusca, Ryan Gouveia Melo, Vaiva Dabravolskaite, Paolo Spath, Vishal Amlani, Aoife Kiernan, and Christian Zielasek
Appendix A. Supplementary data
The following is/are the supplementary data to this article:
References
- 1.Hosseini M., Gao C.A., Liebovitz D.M., Carvalho A.M., Ahmad F.S., Luo Y., et al. An exploratory survey about using ChatGPT in education, healthcare, and research. medRxiv. 2023 doi: 10.1371/journal.pone.0292216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lareyre F., Nasr B., Chaudhuri A., Di Lorenzo G., Carlier M., Raffort J. Comprehensive review of Natural Language Processing (NLP) in vascular surgery. EJVES Vasc Forum. 2023;60:57–63. doi: 10.1016/j.ejvsvf.2023.09.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel) 2023;11:887. doi: 10.3390/healthcare11060887. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Khan R.A., Jawaid M., Khan A.R., Sajjad M. ChatGPT – reshaping medical education and clinical management. Pak J Med Sci. 2023;39:605–607. doi: 10.12669/pjms.39.2.7653. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Eysenbach G. Improving the quality of web surveys: the checklist for reporting results of Internet E-surveys (CHERRIES) J Med Internet Res. 2004;6:e34. doi: 10.2196/jmir.6.3.e34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Reading Turchioe M., Volodarskiy A., Pathak J., Wright D.N., Tcheng J.E., Slotwiner D. Systematic review of current natural language processing methods and applications in cardiology. Heart. 2022;108:909–916. doi: 10.1136/heartjnl-2021-319769. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Harrer S. Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicine. EbioMedicine. 2023;90 doi: 10.1016/j.ebiom.2023.104512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Li H., Moon J.T., Purkayastha S., Celi L.A., Trivedi H., Gichoya J.W. Ethics of large language models in medicine and medical research. Lancet Digit Health. 2023;5:e333–e335. doi: 10.1016/S2589-7500(23)00083-3. [DOI] [PubMed] [Google Scholar]
- 9.Marchandot B., Matsushita K., Carmona A., Trimaille A., Morel O. ChatGPT: the next frontier in academic writing for cardiologists or a Pandora's box of ethical dilemmas. Eur Heart J Open. 2023;3 doi: 10.1093/ehjopen/oead007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Baumgartner C. The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med. 2023;13 doi: 10.1002/ctm2.1206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Sarraju A., Bruemmer D., Van Iterson E., Cho L., Rodriguez F., Laffin L. Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model. JAMA. 2023;329:842–844. doi: 10.1001/jama.2023.1044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.https://arxiv.org/abs/2202.03629
- 13.Janssen B.V., Kazemier G., Besselink M.G. The use of ChatGPT and other large language models in surgical science. BJS Open. 2023;7 doi: 10.1093/bjsopen/zrad032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Alkaissi H., McFarlane S.I. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023;15 doi: 10.7759/cureus.35179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Eppler M., Ganjavi C., Ramacciotti L.S., Piazza P., Rodler S., Checcucci E., et al. Awareness and use of ChatGPT and large language models: a prospective cross-sectional global survey in urology. Eur Urol. 2024;85:146–153. doi: 10.1016/j.eururo.2023.10.014. [DOI] [PubMed] [Google Scholar]
- 16.Melissano G., Tinelli G., Soderlund T. Current artificial intelligence based chatbots may produce inaccurate and potentially harmful information to patients with aortic disease. Eur J Vasc Endovasc Surg. 2024;67:683–684. doi: 10.1016/j.ejvs.2023.10.042. [DOI] [PubMed] [Google Scholar]
- 17.Lareyre F., Raffort J. Ethical concerns regarding the use of large language models in healthcare. EJVES Vasc Forum. 2024;61:1. doi: 10.1016/j.ejvsvf.2023.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lareyre F., Poggi E., Raffort J. Moving forward: evaluation of artificial intelligence chatbots in vascular diseases. Eur J Vasc Endovasc Surg. 2024;67:691. doi: 10.1016/j.ejvs.2023.11.044. [DOI] [PubMed] [Google Scholar]
- 19.Tian S., Jin Q., Yeganova L., Lai P.T., Zhu Q., Chen X., et al. Opportunities and challenges for ChatGPT and large language models in biomedicine and health. Brief Bioinform. 2023;25 doi: 10.1093/bib/bbad493. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Yu P., Xu H., Hu X., Deng C. Leveraging generative AI and large language models: a comprehensive roadmap for healthcare integration. Healthcare (Basel) 2023;11:2776. doi: 10.3390/healthcare11202776. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Roumengas R., Di Lorenzo G., Salhi A., de Buyer P., Chaudhuri A., Lareyre F., et al. Natural language processing for literature search in vascular surgery: a pilot study testing an artificial intelligence based application. EJVES Vasc Forum. 2023;60:48–52. doi: 10.1016/j.ejvsvf.2023.09.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Johnson D., Goodman R., Patrinely J., Stone C., Zimmerman E., Donald R., et al. Assessing the accuracy and reliability of AI-generated medical responses: an evaluation of the ChatGPT model. Res Sq. 2023 [Google Scholar]
- 23.Cascella M., Semeraro F., Montomoli J., Bellini V., Piazza O., Bignami E. The breakthrough of large language models release for medical applications: 1-year timeline and perspectives. J Med Syst. 2024;48:22. doi: 10.1007/s10916-024-02045-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.