Comprehensive Review of Natural Language Processing (NLP) in Vascular Surgery

Abstract

Objective

The use of Natural Language Processing (NLP) has attracted increased interest in healthcare with various potential applications including identification and extraction of health information, development of chatbots and virtual assistants. The aim of this comprehensive literature review was to provide an overview of NLP applications in vascular surgery, identify current limitations, and discuss future perspectives in the field.

Data sources

The MEDLINE database was searched on April 2023.

Review methods

The database was searched using a combination of keywords to identify studies reporting the use of NLP and chatbots in three main vascular diseases. Keywords used included Natural Language Processing, chatbot, chatGPT, aortic disease, carotid, peripheral artery disease, vascular, and vascular surgery.

Results

Given the heterogeneity of study design, techniques, and aims, a comprehensive literature review was performed to provide an overview of NLP applications in vascular surgery. By enabling identification and extraction of information on patients with vascular diseases, such technology could help to analyse data from healthcare information systems to provide feedback on current practice and help in optimising patient care. In addition, chatbots and NLP driven techniques have the potential to be used as virtual assistants for both health professionals and patients.

Conclusion

While Artificial Intelligence and NLP technology could be used to enhance care for patients with vascular diseases, many challenges remain including the need to define guidelines and clear consensus on how to evaluate and validate these innovations before their implementation into clinical practice.

Highlights

• •Natural language Processing (NLP) has brought new insights in vascular surgery.
• •NLP can help to extract information of patients with vascular diseases.
• •NLP can be used as virtual assistants by patients and health professionals.
• •NLP applications need to be validated before implementation in surgical practice.
• •Standardisation and guidelines are required to evaluate NLP applications.

Introduction

Natural Language Processing (NLP) is a field of Artificial Intelligence (AI) that aims to provide computer programs with the ability to analyse, understand, and interpret human language including oral and written language.¹ Using text or voice as input data, NLP addresses several aspects of language including the syntax (the contextual arrangement of words and phrases) and the semantics (the meaning), and uses a combination of various techniques such as rule based methods, statistical, machine learning (ML), and deep learning (DL) models to process human language and provide the output.¹

NLP has a wide range of applications that are already used in daily life (i.e., voice operated systems, speech to text dictation software, or customer services chatbots generating automatic answers to user questions). Its potential applications in healthcare have attracted increased attention over the past decade.¹

Advances in computer technologies including development of electronic health records (EHR), widespread use of the internet, and integrated networks have led to an exponential increase in the generation of health data. Health data are diverse and heterogeneous, including physiological, biological, imaging, administrative, billing data, and medical notes, as well as social and environmental data that can be collected through the use of digital devices.^2,3 They include structured data, defined by a standardised format and structure (i.e., diagnosis codes using standardised classification, biological values with pre-defined units, and also unstructured data, which do not have standardised organisation and formatting (i.e., clinical notes or reports). Much of the knowledge and medical information is still stored in free text format. Identification and extraction of relevant healthcare information at a large scale to enable high quality research is therefore challenging and NLP has been proposed as a useful tool to facilitate such tasks.⁴ A survey on clinical NLP projects over the past 15 years in the United Kingdom confirms a substantial increase in its use in healthcare, with the total budget allocated in the last three years of the survey period being 80 times that of the budget in the first three years.⁴

In parallel to the use of NLP to facilitate the extraction of health information, large language models (LLM) correspond to a field of AI aiming to mimic human language processing abilities. Using DL algorithms such as neural networks, these models are trained on vast amounts of data provided from various sources and can develop the ability to perform various and complex tasks such as recognising, summarising, translating, predicting, and even generating text and other content.⁵ LLM have been used to empower and develop new generations of chatbots (computer programs that process language and allow humans to interact with digital devices as if they were communicating with another human person).⁶ The release of chatGPT by OpenAI in November 2022 attracted a lot of media attention. Its potential use as a virtual assistant raises major concerns in healthcare.^7,8

A few studies have addressed the use of NLP, chatbots, and virtual assistants in vascular surgery.⁹ Vascular specialists should be aware of ongoing research on the topic to better evaluate the potential benefits and limitations for clinical practice and are likely to face increasing demands and legitimate questions from patients and society regarding such innovations.

The aim of this comprehensive review is to summarise the current knowledge of NLP applications in vascular surgery, from applications to enhance management of health data, to the development of virtual assistants for health professionals and patients. The main challenges and limitations that remain are addressed.

Methods

A literature search was performed on the MEDLINE database in April 2023 using a combination of keywords to identify studies reporting the use of NLP or chatbots in vascular surgery, focusing on three main vascular diseases including carotid, aortic, and peripheral artery disease. Keywords included Natural Language Processing, chatbot, chatGPT, aortic disease, carotid, peripheral artery disease, vascular, and vascular surgery. Two authors (J.R., F.L.) screened titles and abstracts collected from the search strategy. As the aim was to perform a comprehensive review of current and potential future applications of NLP and chatbots in vascular surgery, the literature search screened for all types of articles including original articles, literature reviews, editorials, letters, or case reports published in the English language. Exclusion criteria were articles published in languages other than English. The full texts of relevant articles were obtained and reviewed. The reference lists of each manuscript were scanned for other potentially relevant studies.

Natural language processing for identification and extraction of information on patients with vascular diseases

NLP based technology enables extraction of information from various structured and unstructured text documents such as EHR, medical notes, and reports. Such tools have shown applicability in automatic data extraction and identification of patients with various vascular diseases including aortic disease, peripheral artery disease, and carotid stenosis.

Aortic disease

NLP algorithms have been used to identify patients with abdominal aortic aneurysms (AAA) and aortic stenosis from radiology reports or health records.10, 11, 12, 13, 14, 15 Such applications offer opportunities to improve best practice based care across a large health system. As an example, some investigators used a commercially available NLP system to identify AAAs from the EHR in a tertiary hospital over an 11 year retrospective period.¹¹ In total, 6 340 505 imaging reports were processed by the NLP application and 6 495 patients with AAA were identified. Among them, 2 937 cases were reviewed and closed, 1 183 were placed undergoing surveillance, and 2 375 were awaiting review.¹¹ By allowing identification of patients who did not have appropriate follow up, such an application can be very useful in terms of public health by supporting evaluation of current practice and providing a pragmatic tool to identify these patients and propose appropriate prevention and surveillance.

In addition to identifying patients with AAA, NLP also enabled automatic extraction of the characteristics of the AAA such as the size, and therefore can help to classify and evaluate the severity of the disease.^14,15 NLP has the potential to be used as a technology to develop aid decision support systems. By enabling analysis of both numerical and text information from medical charts, some investigators tested the use of a decision support system in emergency care that alerted physicians by predicting aortic dissection.¹⁶ The authors reported the area under the curve (AUC) of the model as 0.901 (95% CI 0.840–0.962). By allowing automatic and early identification of patients with aortic disease from health records, a NLP system can generate flags or alarms that could be useful to coordinate care and facilitate early orientation of patients. These systems can be helpful in emergency and radiology departments, especially in the context of incidental findings, to coordinate management of patients and refer them to specialists when needed. Also, NLP techniques could be added within screening and surveillance programs to facilitate communication and organisation of care across medical and surgical departments and help to optimise the follow up of patients.^17,18

Peripheral artery disease (PAD)

Several studies have highlighted use of NLP to identify patients with PAD from medical reports.¹⁹ In a cohort of 6 861 patients, a study reported that the NLP system enabled identification of patients with PAD from clinical notes in the EHR with an AUC of 0.88.²⁰ The method performed better compared with a traditional structured data based approach (least absolute shrinkage and selection operator).²⁰ Another study investigating a NLP approach to identify PAD from radiology reports showed an accuracy of 0.93 compared with the gold standard defined manually by human operators.²¹ Other studies confirmed the potential interest of NLP to automatically identify patients with PAD from the EHR as well as identification of related complications such as critical limb ischaemia (CLTI).22, 23, 24 Such a tool can have applications in public health, by enhancing study of the epidemiology of the disease, the treatment, and the outcomes of patients. For instance, some authors addressed the compliance to guideline recommendations for secondary prevention and used NLP in the EHR to identify patients with PAD.²⁵ They found that most of the patients did not have an optimal guideline recommended therapeutic strategy.²⁵

Carotid stenosis

The use of NLP to identify carotid stenosis from imaging reports has also been addressed. Using index cases with 76 276 ultrasound (US) and 19 620 computed tomography angiograms (CTA) or magnetic resonance angiograms (MRA), a NLP based approach was used to categorise non-haemodynamically significant carotid stenosis, moderate or severe stenosis, and occlusion.²⁶ Interestingly, the NLP algorithm demonstrated good performance to identify non-haemodynamically significant carotid stenosis, with a positive predictive value of 99% for US and 96.5% for CTA and MRA.²⁶ Such application could help to optimise triage of patients who may require closer follow up than others, and facilitate referral to vascular specialists. Other studies confirmed interest in NLP to identify patients with carotid stenosis from radiology reports and neurology reports, as well as discharge notes.^27,28

NLP is thus a technology that can be used to extract information from health data to identify patients with vascular diseases and provide information on the severity and outcomes of patients, with new tools to enhance public health surveillance and large scale epidemiological studies. Combined with the extraction of other relevant information such as patients’ comorbidities, treatment, and outcomes, it can facilitate analysis of large datasets such as national health data to identify patterns and contribute to building aid decision support systems to enhance precision medicine. In addition, it provides new tools to optimise screening strategies, orientation and organisation of patient care.

Current challenges and limitations

The quality and accuracy of NLP models relies on input data. Large datasets are required for training and the performance of the models can be hindered in case of class imbalance.⁴ Task formulation should be considered carefully in creation of a NLP model to answer the desired clinical question.²⁹ Configuring domain knowledge and ontologies in clinical settings is challenging as formal knowledge resources are not clearly defined and formats and conventions of writing clinical documents are heterogeneous.⁴ In addition, labelled data are required to train supervised models. The annotation process requires medical expertise to evaluate clinical information and is a tedious and complex task that most often necessitates multiple annotators to limit interoperator variations and reach the ground truth.⁴ Access to medical information must comply with current regulations to guarantee patient privacy, data protection, and confidentiality. Implementation of AI algorithms to analyse health data raises major ethical concerns regarding security, equity, trustworthiness, and fairness. Several ethical principles and commitments related to AI have therefore been defined by various groups of experts including the EU's AI High level expert group.^30,31 The current legal framework in Europe requires that access to and use of medical data complies with the General Data Protection Regulation (GDPR).³² The EU's AI Act aims to provide clear guidelines and regulation regarding the use and implementation of AI techniques in healthcare. The first draft was published in 2021 by the European Commission, and it is expected that the law will come into force in late 2023 or early 2024.³³ Such a regulation might bring guidelines and standardisation to guarantee ethics and trustworthy AI. Meanwhile, access and use of medical data raises many challenges regarding data governance and organisation with heterogeneous frameworks between institutions and countries.^1,4,29 Adapted infrastructures are required to provide computational resources and to seek appropriate solutions for interoperability of the systems.

While the results of the above studies show a proof of concept that NLP could be used to facilitate identification of patients with vascular diseases, further validation is required before implementation in clinical practice. A main issue is the explainability of AI models, as understanding how ML models generate outputs is challenging. The portability and reliability of the models must be evaluated as performances of NLP algorithms can differ between institutions and information systems due to differences in formats, documents structures, and expression. Altogether, these current limitations may slow adoption of the models into clinical practice and efforts should be oriented towards external validation. Most of the previous studies were retrospective. Prospective studies and trials would help to evaluate whether evidence based practice and care provided to patients can be enhanced using AI and NLP driven tools. Cross border collaboration remains a key point to address these challenges to build efficient and transposable models.^1,4,29,34

Natural language processing as virtual assistants for health professionals

Administrative and clinical work

Chatbots can perform tasks for writing assistance that could be helpful for health professionals, including vascular surgeons. By performing autostructuring, autocompletion, or translations, this has potential as a digital scribe system to help writing clinic letters, medical reports, or discharge summaries under the supervision of clinicians.35, 36, 37, 38, 39 Although its use has not yet been evaluated in vascular surgery, chatGPT could help clinicians by alleviating administrative and repetitive writing tasks, with the output being systematically checked and kept under the responsibilities of health professionals. Being used appropriately, such innovative tools could help to reduce burden, save time, improve workflow, and enhance communication and effectiveness within healthcare. However, use of chatbots in this setting would require efficient and transparent systems to guarantee data protection, security, and confidentiality. The use of such software for healthcare would require compliance with current regulations such as the GDPR used for European countries and the EU's AI Act expected to soon be applicable.

Research and academic work

Due to its ability to generate text content, chatGPT has attracted increasing attention for its assistance with several steps of scientific writing, reviewing, and editing. The application is able to generate suggestions on the syntax of the manuscript (i.e., vocabulary, grammar), but also suggestions on the scientific content (i.e., format, structure, titles, references, citations, translations, text summarisation for abstracts).40, 41, 42, 43 It has also been suggested that chatGPT could provide information for data analysis, although the software might be limited to simple statistical tests and cannot replace the expertise of statisticians and epidemiologists.40, 41, 42, 43 In addition to scientific writing assistance, some NLP driven applications can also be used as sources of information. As an example, several applications propose use of AI to facilitate bibliographic searches.^7,44 Finally, it has also been suggested that chatGPT could have the potential to generate novel hypotheses or find ideas for systematic reviews or meta-analyses although this may be limited by the output being dated as the application was only trained on data up to 2021.^41,45

Medical and surgical education

AI has provided new tools and techniques that can be helpful for the training and education of vascular surgeons.⁴⁶ The launch of chatGPT raises the question of whether this NLP technology could also be applied for medical and surgical education, and several applications have been suggested.^8,47,48 ChatGPT could be used for teaching assistance under the responsibility of the supervisors to extract and or generate adapted content such as quizzes, exercises, case studies, or scenarios.⁴⁸ It could also help to enable personalised learning through the generation of content adapted to the student, personalised study plans, and feedback.⁴⁸ Students have access to large amounts of data and can be confronted with an overload of information. NLP applications could be useful in research assistance, saving students the time required to find and organise appropriate content. Finally, NLP could also be used to facilitate evaluation and follow up of students’ work through implementation of new tools such as automatic scoring.⁴⁸

Pitfalls and challenges

The potential use of NLP driven virtual assistants such as chatGPT by health professionals raises important ethical concerns.⁴⁰ Evaluation of the accuracy of the output information is challenging and clinicians should bear in mind that some information can be outdated.⁴⁹ While the application has been proposed for writing assistance, there remain major limitations as it lacks creativity, originality, and critical thinking. It should therefore be kept under supervision, with final responsibility lying with humans. In addition, as the software is learnt on various sources including previous publications, there is a potential risk of plagiarism by generating similar text or content.^35,40,43 In addition, it can be difficult to detect whether a manuscript or abstract has been generated by AI. An evaluation revealed that even scientists can be fooled, and that one third of the AI generated abstracts were erroneously rated as having been generated by humans.^50,51 There have even been cases in which chatGPT has been cited as a co-author.⁵² AI output detector programs are currently being developed and many editorial offices are planning to implement them in their process in addition to plagiarism detectors.^53,54 Guidelines and policies to define the use of AI generated text in academic research are currently being built. The International Committee of Medical Journal Editors highlighted criteria that chatGPT and AI applications cannot fulfil to be cited as co-author. These include the lack of final approval of the work, agreement to be held accountable, and inability to identify the specific contribution to the manuscript. Therefore, many editorial offices agree that chatGPT should not be cited as a co-author, and instead recommend mentioning its use in the acknowledgements. The field is in its infancy and the use of NLP, virtual assistants, and chatGPT in academic research raises ethical considerations on good scientific practice that need to be carefully evaluated.

Natural language processing as virtual assistants for patients

Natural Language Processing as a source of information and support assistant for patients

Chatbots such as chatGPT or Med-PaLM 2 developed by Google Health represent a wide source of information on health, diseases, and treatments that can be easily available for patients.

NLP technology has also enabled development of virtual assistants that can be helpful as support systems to assist and optimise patient care. NLP driven virtual assistants can help patients for planning, follow up, or appointment scheduling. In addition, various applications for patients with vascular diseases are currently being developed to enable telemedicine including telecoaching, telesurveillance, or telemonitoring.³ These applications can provide support assistance for follow up, medication management, or engagement of patients for lifestyle changes such as promoting physical activity or health diet.⁹ As an example, various telemedicine enhanced exercise programs have been developed for patients with LEAD to enhance education and patient adherence.55, 56, 57 NLP technology can be used and embedded in a wide range of applications related to telemedicine to assist patients in various tasks and contribute to mental encouragement and support all along care.

Pitfalls and challenges

Although the applications of NLP and chatbots are interesting, their use to provide medical information to patients raises questions regarding evaluation and validation of the accuracy and quality of such information. Patients should remain cautious of the information provided as the quality of information depends on the training datasets. Bias in a training dataset can lead to generation of misleading information or outdated content.^37,49 Besides, the information needs to be provided in an appropriate and clear manner to ensure adequate understanding by patients. Answers provided by chatbots can sometimes be repetitive or stereotypical, with information being vague or inaccurate.³⁷ One of the main issues with LLM models is the potential occurrence of neural hallucinations, meaning that sometimes the model can generate content output that is incorrect or nonsensical, despite appearing reliable.^43,58,59

The use of chatbots by patients incurs several ethical issues, not only linked to the accuracy of the information provided, but also regarding data privacy, security, and concerns on responsibilities on how to guarantee patient safety.^37,40,43 Although chatGPT passed the exams of the United States Medical Licensing Exam (USMLE),⁶⁰ clinical practice is a complex process and patients cannot be considered as standardised question items. Decision making is currently based on clinical guidelines and evidence based medicine, but it also relies on the expertise of health professionals taking into consideration patient comorbidities, clinical examination, and results from biological and imaging investigations. While chatbots can assist and provide information and suggestions on diagnoses and treatments, they cannot replace medical expertise provided by health professionals who will also take into consideration results from medical data as well as the patients wishes and their environmental, social, and psychological conditions. NLP driven applications offer perspectives to be used by both patients and professionals, but they cannot replace humans, the patient–doctor relationship, the expertise, and the role of support of health professionals for patients and their families.

Conclusion

NLP offers a wide range of applications in vascular surgery by enabling the identification and extraction of information on patients with vascular diseases. The technology can be used to help analyse data from health information systems to provide feedback on current practice and develop support systems to optimise care and orientation of patients. In parallel, the development of chatbot and NLP driven virtual assistants can support health professionals in clinical, scientific, and academic work, but also patients by providing medical information and assistance. While these innovations are encouraging, challenges and limitations remain, and these must be faced to ensure efficient and safe use in clinical practice. The major need is to define specific guidelines and standards to guarantee ethics and validate applications. AI and NLP technologies raise cultural concerns over the acceptability and perception of the technology by health professionals, patients, and society. Such applications should be used to enhance and augment patient care without replacing human expertise and responsibility. Education, information, and involvement of vascular specialists in the field is of utmost importance to anticipate and shape the future of medical and surgical vascular practice.

Funding

This work was supported by the French government through the National Research Agency (ANR) with the reference number ANR-22-CE45–0023–01 and through the 3IA Côte d’Azur Investments in the Future project managed with the reference number ANR-19-P3IA-002.

Conflict of interest

None.