Large language models in clinical trials: applications, technical advances, and future directions

Abstract

Background

As clinical trials scale up and grow more complex, researchers are facing mounting challenges, including inefficient participant recruitment, complex data management, and limited risk monitoring. These issues not only increase the workload for clinical researchers but also compromise trial reliability and safety, potentially elevating the risk of trial failure. Large language models (LLMs), as an emerging technology in natural language processing (NLP), exhibit notable advantages across various tasks, such as information extraction and relation classification.

Main text

With domain-specific pre-training and fine-tuning, LLMs present promising potential in clinical trial tasks such as automated patient-trial matching and the extraction and processing of trial data, which are anticipated to reduce time and financial costs. Additionally, they offer valuable insights for scientific rationale, medical decision-making, and trial endpoint prediction. In this context, an increasing number of studies have begun to explore the applications of LLMs in the design and conduct of clinical trials.

Conclusion

This paper provides a review of LLM applications in clinical trials with an emphasis on real-world integration. Comparative advantages over traditional NLP models, technical limitations, and future implementation challenges are also discussed. This narrative review aims to highlight the potential of LLMs in clinical trial workflows and clarify key challenges and future directions.

Background

As the most rigorous and widely adopted referent standard for evaluating medical interventions, clinical trials play an indispensable and pivotal role in several key areas, including advancing medical progress, verifying the efficacy of treatment regimens, and ensuring the safety of medications for patients. Since 1747, when James Lind conducted the world’s first systematically documented randomized controlled trial (RCT) on scurvy prevention using vitamin C, the standardized protocol design and conduct of clinical trials have not only become a cornerstone of evidence-based medicine but have also provided a solid and reliable scientific foundation for modern medical innovations [1]. By implementing rigorously controlled study designs, standardized data collection processes, and methodologically sound statistical analyses, clinical trials can minimize research bias and provide robust medical evidence to support evidence-based decisions in clinical practice. In the twenty-first century, the number of clinical trials has increased dramatically. According to data from ClinicalTrials.gov, the number of registered trials increased from 2119 in 2000 to 477,220 in 2023, representing a compound annual growth rate of over 9% [2].

With ongoing innovation and rapid advancements in experimental drugs and therapeutic regimens [3–6], the challenges faced by clinical trials are becoming increasingly complex, drawing widespread attention from both the medical community and society at large. One of the most prominent challenges is recruiting qualified participants in compliance with the trial protocol within the preset timeline and ensuring their continued participation [7]. A systematic study of 151 RCTs in the UK showed that up to 44% of trials did not achieve their intended participant recruitment goals, and a median of 11% of participants discontinued follow-up for various reasons [8]. Accurately matching clinical trials with potential participants also presents challenges. A study on cancer clinical trials demonstrated that over half (55.6%) of patients were ineligible for participation at the healthcare institution where they received treatment, and an additional 21.5% were excluded for failing to meet enrollment criteria [9]. Assessing treatment effects based on clinical indicators, particularly distinguishing the effects of therapeutic interventions from the natural progression of the disease, represents another challenge in clinical trials [10]. This requires researchers to establish multidimensional data integration and analysis systems, as well as systematically tracking and monitoring the long-term clinical performance of participants. Additionally, there are also several critical challenges in clinical trials that urgently require resolution, including establishing a trial risk assessment system, real-time monitoring of adverse events (AEs) [11–13], the standardized management of vast clinical data, and the safeguarding of participants’ private data, all of which demand innovative solutions from next-generation information technologies.

In recent years, artificial intelligence (AI) technologies, including machine learning (ML) [14, 15], deep learning (DL) [16], computer vision (CV), and the Internet of Medical Things (IoMT), have made substantial progress in the field of clinical trials. These advances include the use of computer vision for cellular imaging analysis to assess the immunological response of subjects following drug treatment [17], the application of DL to analyze retinal angiography images in clinical trials to evaluate hemodynamic features [18], and the use of random forest algorithms to predict the approval likelihood of drugs in clinical trials for chronic obstructive pulmonary disease [19]. As one of the essential branches of AI, natural language processing (NLP) technology uses computational models to process large-scale text, enabling both semantic understanding and generation of natural language [20]. Researchers have applied NLP technology to clinical documents, such as electronic health records (EHRs), to transform unstructured data into clinical insights [21]. Since the advent of generative pre-trained transformer-1 (GPT-1) and bidirectional encoder representations from transformers-base (BERT-base) in 2018, large language models (LLMs), as a class of pre-trained models characterized by a large number of parameters, have had a revolutionary impact on the AI field. Based on the Transformer architecture and large-scale data training, LLMs exhibit emergent abilities in contextual understanding, the ability to follow instructions, and deductive reasoning, demonstrating exceptional interactive capabilities and professional proficiency in NLP tasks such as named entity recognition (NER), relation extraction (RE), or relation classification (RC) [22, 23].

Foundational LLMs, exemplified by GPT, BERT, and LLaMa, following domain-specific pre-training and subsequent medical fine-tuning, have been widely applied in clinical practice, such as medical inquiry, clinical decision support, and the interpretation of medical imaging and electrocardiogram reports [24–33]. However, their systematic evaluation and integration in the clinical trials remain in early stages. Current researches mainly focus on limited domains such as subject screening and participant data extraction, lacking exploration and optimization efforts in other aspects of clinical trials. A comprehensive review is therefore necessary to understand their potential contributions across the various stages of clinical trials.

This review presents the relevant applications of LLMs in the design and conduct stages of clinical trials, technical advances, and future challenges, across four main dimensions: (1) the applications of LLMs in clinical trial design, specifically involving three key aspects: extracting and analyzing research elements and reasons for termination to inform scientific rationale and protocol development, optimizing eligibility criteria and evaluating the association between criteria complexity with trial termination risk, and enhancing research ethics and informed consent processes; (2) the deployment of LLMs in the conduct of clinical trials, covering four key stages including participant recruitment and screening, data acquisition and management, safety monitoring, and trial outcome prediction; (3) the advantages of LLMs in comparison with traditional NLP approaches, their existing technical limitations, and the corresponding technical strategies; (4) the challenges LLMs encounter in clinical trial settings, including data privacy, quality and blinding control, model transparency and trustworthiness, legal and regulatory concerns, standardization of evaluation and application, as well as corresponding potential solutions. This study comprehensively summarizes the current applications and projected scenarios in clinical trial workflows. This review aims to help clinical researchers better understand the potential applications of LLMs across various stages of clinical trials, while the discussion of technical characteristics and implementation challenges provides valuable insights for the promotion and future development of LLMs in this context.

LLMs in clinical trial design

In this section, we focus on the applications of LLMs in three key aspects: (1) protocol design and scientific rationale, (2) recruitment and exclusion criteria, and (3) ethics and informed consent. We outline how LLMs can assist in extracting and synthesizing essential trial components, such as PICO elements, thereby supporting the construction of scientifically grounded protocols. The discussion then turns to the optimization of eligibility criteria, emphasizing the potential of LLMs to analyze and flag overly complex or exclusionary terms linked to trial failure, and retrieve historical patterns to support recruitment design. Furthermore, we explore the potential of LLMs to identify ethical risks and enhance the customization and clarity of informed consent materials, particularly by simplifying technical language for laypersons (Fig. 1).

Fig. 1.

Applications of large language models in clinical trial design. The left boxes illustrate three steps of clinical trial design and their specific contents, in the following top-to-bottom order: establishment of the research background and objectives, protocol development, and ethical approval with informed consent. The right boxes demonstrate how large language models (LLMs) can assist researchers in optimizing and accelerating specific tasks in each design phase

Scientific rationale and protocol design

Extracting and analyzing previous research elements

During clinical trial design, researchers typically review previous studies to extract key information, including Patient, Intervention, Comparison, and Outcome (PICO) elements, efficacy and safety data, eligibility criteria, and reasons for trial termination, in order to inform hypothesis generation and protocol development. However, with the exponential growth of medical literature in recent years, researchers are increasingly challenged to stay abreast of contemporary advancements [34]. Additionally, extracting key trial information from abstracts or full texts of research reports often proves to be both time-consuming and labor-intensive [35]. Several applications, such as RobotReviewer [36], ResearchScreener [37], DistillerSR [38], and Abstrackr [39], have already been developed to extract information from scientific articles or abstracts, assess research quality, and draw inferences regarding treatment effects. However, these technologies are primarily based on traditional semi-automated frameworks (e.g., logistic regression, support vector machines), which face limitations such as limited flexibility and challenges in generalizability.

As an emerging NLP technology, LLMs are trained on large-scale datasets using unsupervised learning techniques and are able to extract and analyze key elements from prior studies with higher efficiency and accuracy. For example, Lee et al. developed a GPT-4-based pipeline, named SEETrials, which is capable of automatically extracting safety and efficacy data from the abstracts of multiple myeloma clinical trials [40]. Whitton and Hunter utilized a fine-tuned BERT model for the automatic extraction of key findings from RCT reports and the automatic generation of corresponding tables [41].

The PICO framework serves as a fundamental component in formulating clinical questions. The capability of LLMs to automatically extract PICO information from clinical trial reports not only accelerates comprehensive analyses of previous studies but also facilitates the formation of new research hypotheses. Mutinda et al. utilized BioBERT to extract PICO information, which is then standardized using the Unified Medical Language System (UMLS), and ultimately converts the information into structured data, facilitating subsequent statistical analyses and visualization [42]. Utilizing domain-specific training datasets can notably enhance the completeness and accuracy of models in PICO information extraction tasks. Wang et al. compared general-purpose and mixed-domain pretrained BERT models; those pretrained specifically for the medical domain exhibited significant advantages in accuracy of PICO extraction [43]. Few-shot learning (FSL) has been demonstrated to be an effective approach in low-resource settings. Ghosh et al. developed AlpaPICO, a PICO information extraction model that boosts in-context learning capabilities with minimal prompting. Unlike traditional methods that require additional training, researchers input annotated context into AlpaCare to bolster its performance in low-resource environments, thereby bypassing the need for complex supervised training and fine-tuning [44].

Guiding research direction through evidence synthesis

In the frontier areas of medicine where consensus has yet to be reached, contradictory research findings frequently arise. Building an evidence-based medical system rooted in diverse evidence helps guide researchers in identifying gaps in medical knowledge [45]. Traditionally, identifying and resolving such contradictions requires extensive manual review of the literature. LLMs can assist researchers in capturing differences in conclusions across studies on similar topics, while offering high-quality, multidimensional evidence-based medical insights. Xie et al. employed the GPT-4–1106-preview model to evaluate ChatGPT’s effectiveness in identifying conflicting arguments. Tests conducted on a PubMed dataset demonstrated that the model achieved a recall rate of 0.903 in detecting inconsistencies in ternary assertions. Notably, ChatGPT’s basic reasoning logic in interpreting medical evidence is not influenced by manual annotation; rather, it relies on its inherent logical reasoning capabilities to assess the relevance of each claim to the core hypothesis of the issue [46]. This study highlights ChatGPT’s objectivity and critical approach in evaluating conflicting medical evidence, offering novel methodological support for clinical trials grounded in evidence-based rationale.

Learning from historical trial terminations

LLMs systematically analyze various factors influencing trial continuation and summarize termination reasons from historical trials, providing valuable insights for clinical researchers to optimize the design phases of trials, thereby improving completion or success rates. Wang et al. devised an innovative approach by applying the sciBERT-NN model to a large dataset comprising 76,950 clinical trial records from ClinicalTrials.gov and MEDLINE, aiming to predict the likelihood of clinical trial publication and analyze characteristics influencing trial completion [47]. Razuvayevskaya et al. employed a fine-tuned BERT model to perform a comprehensive analysis and classification of 28,561 prematurely terminated clinical trials from the ClinicalTrials.gov database. This model systematically summarized the reasons for clinical trial failure—99% of prematurely terminated trials could be grouped based on their reasons for failure within a classification framework comprising six primary categories and 15 subcategories. Results revealed a critical finding: drug trials terminated at various stages generally lacked robust genetic support [48]. The analysis of common features of terminated trials using LLMs encouraged researchers to more precisely explore and validate potential targets at the molecular level, offering novel insights into clinical trial design.

Recruitment and exclusion criteria

Designing appropriate eligibility criteria is one of the most challenging tasks in clinical trial planning. Researchers typically define the target population based on study objectives, disease characteristics, and the nature of the intervention. At the same time, individuals with potential safety risks or factors that may interfere with outcome assessment must be excluded to ensure internal validity and control for confounding variables. This process often involves referencing eligibility criteria from prior similar studies and clinical practice guidelines and is usually refined through multidisciplinary discussions [49]. Moreover, trial designers need to strike a delicate balance between scientific rigor and practical feasibility, especially in the absence of established precedents, since overly restrictive criteria may inadvertently exclude patients who could otherwise benefit from participation, reducing the generalizability of findings and contributing to recruitment delays or even trial terminations.

On the one hand, LLMs can analyze the complexity of eligibility criteria and their association with trial termination risks and identify specific terms within recruitment criteria that are strongly linked to trial failure. Peterson et al. applied ScispaCy to analyze the complexity of eligibility criteria across thousands of clinical trials. They found that from 2008 to 2018, the median number of unique words in eligibility criteria increased by 95%, while trial termination rates rose by 17.6%. ScispaCy also uncovered linguistic patterns associated with high termination risks. For example, exclusion criteria in neuropsychiatric trials containing medical terms such as “brain,” “central nervous system,” and “psychiatric” were strongly linked to higher trial failure rates [50]. This insight highlights that LLMs have the potential to offer early warnings about the risks of overly complex or exclusionary criteria, enabling investigators to proactively revise them before trial initiation.

On the other hand, LLMs can automate the extraction of recruitment criteria from prior similar studies, offering trial designers structured, experience-based insights that help standardize eligibility criteria across similar trials. Datta et al. introduced a GPT-4-based model, AutoCriteria, which automates the extraction of recruitment eligibility criteria from clinical trial reports with high accuracy [51]. Lim et al. employed an LLM-assisted pipeline to systematically analyze phase III clinical trials involving heart failure with reduced ejection fraction (HFrEF). The quantitative analysis demonstrated that 69% of trials used the New York Heart Association functional classification as the primary inclusion criterion, while acute coronary syndrome and valvular heart disease were the most common exclusion criteria [52]. This finding holds significant reference value for future clinical studies related to HFrEF.

Ethics and informed consent

Identifying potential ethical issues

All clinical trials involving human subjects must undergo review by an institutional review board (IRB), which is responsible for safeguarding participant rights and evaluating the ethical acceptability of trial protocols [53]. Constrained by lengthy review timelines [54] and the limited oversight capacity within IRBs [55], clinical trial investigators can utilize LLMs to conduct ethical self-assessments of trial designs prior to IRB submission. LLMs can learn and understand complex ethical issues, automatically assess the compliance of trials with local regulations, and provide decision support. Sridharan and Sivaramakrishnan conducted a pioneering study, which evaluated four LLMs (Google Bard, Claude, GPT-3.5, and GPT-4) on seven validated cases that covered multiple key ethical concerns, including recruitment eligibility criteria, concerns related to vulnerable populations, the disclosure of information in the informed consent, risk–benefit assessments, and the rationale for using placebos. The results of the study showed that all the evaluated LLMs were able to respond to all the questions posed in the given cases upon receiving prompts. Moreover, LLMs produced more specific and actionable suggestions when given more detailed prompts [56]. This study highlighted that, when given detailed prompts, LLMs may complement early-stage protocol development by helping researchers identify areas that could raise concerns during formal IRB review.

Optimizing informed consent

Obtaining fully informed consent from participants is the cornerstone of protecting their rights and safety in clinical trials. When drafting informed consent forms (ICFs), researchers often need to establish the framework including the study objectives, participant responsibilities, and potential risks or benefits based on the clinical trial protocol. However, due to the cognitive gap between patients and professionals, it is often challenging for participants to fully comprehend the medical terms and concepts presented in the ICFs [57, 58]. Furthermore, a patient’s racial and religious background may also influence their perception of certain clinical trials [59]. Therefore, after establishing the core framework of the ICFs, researchers often need to conduct a readability check of the drafted version and develop different versions or supplementary materials tailored to different populations.

Recent studies have highlighted the potential of LLMs to assist researchers in drafting personalized ICFs tailored to specific cases and to facilitate participants’ comprehension of abstract medical concepts and principles in ICFs. Sridharan and Sivaramakrishnan examined the ability of four LLMs to generate ICFs across seven ethically challenging cases. The results showed that all tested LLMs were not only capable of identifying key ethical risks, but also successfully generated tailored ICFs that included risk disclosure and participant rights with adaptation to specific cases [56]. By annotating complex technical terms, adopting concise phrasing, and simplifying paragraph structure, LLMs might effectively reduce the difficulty participants face in understanding ICFs. Campillos-Llanos et al. employed the GPT-3.5 model to optimize the clinical trial announcements for Spanish patients. Although 28.9% of the definitions still required further refinement, GPT-3.5 generally demonstrated satisfactory performance in defining and simplifying medical terminology [60]. Ali et al. applied GPT-4 to reduce the comprehension difficulty of the ICF from the college freshman level to the eighth-grade level. After optimization, the frequency of technical terms and passive voice in the simplified text notably decreased, and complex long sentences were effectively split into several shorter sentences. All simplified versions passed strict review by legal and medical professionals [61].

LLMs in clinical trial conduct and operations

In this section, we review the applications of LLMs in the conduct and operational management of clinical trials, with a particular emphasis on four key stages: (1) participant recruitment and screening, (2) data collection and management, (3) safety monitoring, and (4) trial outcome prediction. We first review how LLMs facilitate automated patient-trial matching and eligibility screening with high accuracy and reduced costs, by transforming unstructured documents and aligning them with structured criteria. We then explore the potential of LLMs to assist in collecting and structuring trial data, improving data quality through automated processing and cleaning, and enabling data mapping across heterogeneous datasets. In the safety domain, LLMs enhance adverse event detection and drug-drug interaction tracking through natural language understanding and prompt-based inference. Moreover, we highlight their emerging role in predicting trial outcomes, including the simulated generation of trial scenarios (Fig. 2).

Fig. 2.

Applications of large language models in the conduct and operations of clinical trials. The four boxes at the top depict the main steps and related content of clinical trial conduct, in the following left-to-right order: participant recruitment, follow-up and data collection, safety monitoring, and data management and analysis. The boxes below outline specific tasks that large language models (LLMs) can be deployed to automate at each of these stages

Participant recruitment and eligibility screening

Insufficient patient recruitment is a primary cause of clinical trial termination [3]. Previous efforts, such as enhancing social media outreach [62] and providing incentives to participants [63], have partially addressed the insufficient enrollment. However, the effectiveness of these methods remains uncertain. Patient recruitment typically begins with eligibility screening, a process in which researchers compare trial inclusion/exclusion criteria against patient information, most of which is stored as unstructured data in electronic health records (EHRs). This process is labor-intensive and time-consuming—on average, screening a single patient takes approximately 45 min [64, 65]. Against this backdrop, previous studies have attempted to leverage traditional AI to assist with patient–trial matching [66, 67]. However, there is a notable discrepancy between the unstructured free-text data in electronic EHRs and the structured eligibility criteria. This inconsistency in data format severely hampers the efficiency of trial matching. In recent years, with the widespread adoption of standardized medical coding systems such as the International Classification of Diseases (ICD) [68], Current Procedural Terminology (CPT) [69], and Unified Medical Language System (UMLS) [70], the standardization of expressions of medical concepts substantially enhances the ability of LLMs to parse, understand, and reconcile various terms in EHR information and trial eligibility criteria, laying the foundation for efficient and precise automated patient-trial matching and eligibility screening.

Recent advancements in LLM-based trial-matching frameworks have demonstrated promising gains in matching accuracy and data security. TrialGPT developed by Jin et al. precisely scored various patient metrics, demonstrated superior accuracy in ranking and filtering potential trials when compared to traditional linear aggregation analysis [71]. Yuan et al. developed an innovative LLM-PTM matching model with privacy enhancement methods. Compared to baseline models, LLM-PTM, through its analysis of de-identified patient data, demonstrated 6% and 8.4% improvement in F1-scores for patient-eligibility matching and patient-trial matching, respectively [72]. This advancement not only enhanced matching accuracy but also effectively safeguarded patient privacy, providing a viable solution to data security concerns in clinical trial recruitment.

The impressive performance of LLMs in reducing the time and economic costs of patient-trial matching also warrants attention. Beattie et al. utilized GPT-4 to assess whether 74 patients with head and neck cancer met the inclusion criteria for a phase II clinical trial. Through the continuous refinement of LLM-driven EHR analysis under different prompts, GPT-4 maintained good accuracy while reducing the average matching time to 7.9–12.4 min per patient, with costs lowered to just $0.15–$0.27 per case [73].

The integration of LLMs with oncogenomics databases is spearheading a new direction in cancer patient-drug trial matching studies, paving the way for precision oncology therapies. Wu et al. developed MonoMiner based on the OMIM knowledge base of 4461 monogenic pathogenic genes. MonoMiner retrieves disease-gene pairs from EHRs using specific trigger words and ranks them by similarity to known entries in the OMIM database [74]. Xu et al. developed OncoCTMiner, an innovative clinical oncology database that incorporates an automated patient-trial matching function. OncoCTMiner can intelligently match patients’ genomic mutation profiles and other “omics” data with the enrollment criteria of clinical trials in the database. Additionally, users can filter the matching results based on multidimensional information such as the patient’s clinical treatment history and trial recruitment status, ultimately producing an optimized list of recommended clinical trials [75]. Such integrations mark a promising step toward more targeted enrollment in oncology trials at the genetic level.

Data collection and management

Data acquisition and standardization

In traditional RCTs, clinical data collection is highly structured and typically predefined during the trial design through case report forms (CRFs) [76] and data management plans (DMPs) [77]. These documents specify variables, timing, coding standards, and access control. During phases such as baseline data collection, intervention administration recording, and follow-up data collection, researchers often need to manually enter and cross-check large amounts of free-text information, which must then be converted into structured data. These processes are not only time-consuming and labor-intensive, but also prone to input errors, inconsistent terminology, and inter-site variability, all of which can compromise data quality. Conventional methods for information extraction typically rely on rule-based or lexicon-based encoding approaches [78], as well as conditional random fields (CRFs) [79] and neural networks (NNs) [80]. However, these techniques often suffer from limited generalizability and poor performance on long or complex textual inputs. Against this backdrop, collecting and transforming clinical text into electronic structured data automatically has become an urgent challenge.

The utilization of multimodal LLMs (M-LLMs) to extract and standardize baseline information or participant responses from unstructured text is an emerging solution. For instance, M-LLMs integrated with optical character recognition (OCR) technology can efficiently extract baseline characteristics from handwritten physician notes, paper-based reports, or non-interactive databases, and convert them into editable and searchable electronic text [81]. Laique et al. utilized OCR to convert cancer genomics data stored in PDF format from The Cancer Genome Atlas (TCGA) into structured text, and subsequently applied a fine-tuned ClinicalBERT model to identify and classify genomic entities, offering a novel pathway for processing cancer research data [82]. Similarly, M-LLM-based approaches have been explored for extracting structured data from pathology documents [83] and unstructured medical records [84]. However, the application of LLMs for automated extraction or standardization of trial intervention data, such as dosage and timing, as well as group-specific differences between treatment and control arms, remains largely undeveloped. Future work should expand LLMs’ applications in these areas, integrating real-time data capture and remote monitoring to help researchers efficiently interpret trial data characteristics.

Data processing and cleaning

After trial data have been collected and standardized, further data processing and cleaning are typically required. Specifically, validation rules predefined in the DMPs are applied to identify missing values, outliers, and inconsistencies within the structured dataset. Researchers then issue data queries to investigate these anomalies and implement corrections. The revised rules are employed to remove erroneous data, impute missing values, and rectify records that do not meet the established standards, thereby ensuring data accuracy and completeness [85]. Throughout this process, LLMs, by leveraging contextual modeling, can efficiently detect and correct anomalous data within the dataset, enhancing the automation and quality of data cleaning. As a context-aware data processing pipeline, LLMClean is capable of constructing semantic dependency models based on context, thereby identifying and correcting invalid, anomalous, or inconsistent entries in clinical datasets [86]. Databonsai is a data preprocessing tool that integrates multiple LLMs. Leveraging the contextual awareness of LLMs and unique batch processing capabilities, Databonsai can automatically perform tasks such as annotation, classification, anomaly detection, and standardization across multiple tabular datasets based on user-defined requirements. It is particularly well-suited for real-world data and auxiliary clinical trial databases [87]. In practice, the deployment of LLMs for data cleaning must follow the pre-specified DMPs to ensure compliance with privacy safeguards, data quality, and blinding control, which are discussed in detail in the Data quality, privacy, and blinding control section.

Data mapping and integration

Clinical trial data typically require mapping and integration into large datasets to facilitate analysis and sharing. This process commonly involves terminology standardization, variable alignment, and format conversion, whereby these steps are guided by variable naming conventions and target data structures specified in DMPs [88]. LLM-based comprehensive architectures have demonstrated considerable potential for cross-dataset mapping. Kimura et al. combined BioBERT, GPT-3.5, and retrieval-augmented generation (RAG) techniques to map drug names to the standardized RxNorm terminology system [89]. Adams et al. integrated GPT-3 embeddings with vector matching methods to align clinical trial data with public health registry data, subsequently mapping these data to the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) [90]. This framework not only maintained high-precision matching but also reduced data mapping time to a matter of hours, thereby substantially enhancing the capacity of small teams to participate in data sharing platforms.

Safety monitoring

Tracking adverse events

In clinical trials involving investigational drugs, devices, or novel therapies, tracking adverse events (AEs) is a critical component of safety assessment and subject protection [91]. Active surveillance of AEs typically relies on regular follow-up assessments, laboratory testing, and review of data within participants’ EHRs or trial records. AEs, especially severe AEs, are often required to be reported and entered into the system within 24 h [92]. Given the time-sensitiveness of this process, an increasing number of studies have begun to explore the potential of LLMs to detect AEs from trial records automatically. Li et al. demonstrated the strong performance of LLMs in extracting influenza vaccine-related AEs from annotated reports [93], while Hu et al. investigated the impact of various prompt strategies on the performance of LLMs in identifying vaccine-associated AEs [94]. Sivarajkumar et al. further advanced this application by using LLMs to automatically adjudicate cardiovascular AEs from real-world, unstructured clinical trial records. Their LLM-based pipeline first extracts relevant information from free-text documents and then applies expert guidelines and a Tree-of-Thoughts reasoning framework to identify cardiovascular events [95]. Currently, LLMs are deployed primarily for the detection and adjudication of AEs, serving a diagnostic rather than prognostic role. Future efforts should focus on integrating such LLM-driven analytic frameworks into electronic data capture systems or mobile applications to enable real-time, proactive AE monitoring throughout the course of clinical trials.

Detecting drug-drug interactions

In clinical trials, managing drug-drug interactions (DDIs) is a critical aspect of trial design and patient safety. LLMs offer promising tools to enhance this process by automatically detecting potential DDIs, particularly those related to pharmacokinetics and synergistic effects. For example, Zirkle et al. developed the BioBERT-directional DDI model based on 325 manually annotated FDA drug labels. This model successfully identified pharmacokinetic-drug interactions (PK-DDI) with an accuracy of 0.82 and achieved 100% accuracy in identifying target drugs [96]. Li et al. utilized CancerGPT based on GPT-2 to predict the synergistic effects of paired drugs across different organ sites after few-shot prompting. The results showed that for areas with rich prior knowledge, such as the endometrium and bone tissue, data-driven models exhibited better predictive performance. However, in prediction tasks for areas lacking sufficient external information, such as soft tissue and the urinary tract, CancerGPT’s FSL performance notably surpassed that of data-driven models [97]. This study highlighted the value of few-shot prompting in situations where the types and characteristics of real-world data differ from training data. In the future, researchers can leverage LLMs to uncover drug combinations with greater efficacy or risk, and tailor protocol adaptations for specific subgroups affected by DDIs, enhancing therapeutic outcomes while maintaining trial-level generalizability.

Trial outcome prediction

In feasibility assessments and interim analyses, leveraging LLMs to predict trial outcomes (such as primary/secondary endpoint and drug efficacy) can provide early insights for protocol optimization and resource allocation, promoting more evidence-based medical decision-making. Dougherty et al. applied BART to predict the outcomes of clinical trials involving COMP360, a drug developed for patients with treatment-resistant depression. The research team transcribed recordings of conversations between participants and their psychotherapists into text. BART then assigned emotion and arousal scores to these transcripts and predicted participants’ treatment responses at 3 and 12 weeks to assess the overall efficacy of COMP360 [98]. Agarwal et al. used TRANSMED to predict the duration of hospitalization and improvements in pulmonary ventilation in patients receiving a novel COVID-19 treatment [99]. This study highlighted that integrating static information (e.g., patient baseline characteristics) with dynamic datasets (e.g., longitudinal imaging or laboratory results) may enhance model performance, particularly in rare or novel diseases.

Recently, researchers have proposed an innovative approach using multi-LLM pipelines to simulate clinical trials for drug efficacy evaluation. Goldenholz et al. developed a pipeline composed of three different LLMs to simulate randomized trials of cenobamate for seizure control. The pipeline employed LLaMa 2 to generate clinical notes based on real-world data; the Mistral model then summarized these notes, and Claude 2 aggregated the summaries and predicted the 50% response rate between the placebo and treatment groups [100]. Future studies could explore integrating both trial simulation and data analysis into a single LLM, thereby simplifying the entire pipeline.

It is noteworthy that most RCTs rely on blinding to minimize bias. Unrestricted use of LLMs to predict trial outcomes may inadvertently reveal critical intervention information or influence assessors’ judgments, thereby compromising the integrity of the blinding mechanism. Therefore, such applications must be carefully contextualized and confined to appropriate settings, such as non-blinded or open-label trials, retrospective analyses, or simulation frameworks.

Comparison between LLMs and traditional NLP models in clinical trials

Based on the Transformer architecture and large-scale pre-training, LLMs demonstrate better performance than traditional NLP models. This section elucidates four major performance advantages of LLMs over traditional NLP models: contextual understanding, few-shot learning, dynamic text generation, and generalization/multitask capability. However, challenges including hallucinated outputs, prompt sensitivity, and limited capacity for knowledge updates may compromise their reliability. By examining the technical advantages that LLMs currently demonstrate over traditional approaches alongside their remaining limitations in the context of clinical trial scenarios, this analysis informs trial investigators to better understand and evaluate the outputs of LLMs (Fig. 3).

Fig. 3.

Performance advantages, technical limitations, and solutions for large language models in clinical trials. The upper four sections illustrate the major performance advantages of large language models (LLMs) compared to traditional natural language processing models, and explain how these advantages support LLM applications in clinical trials. These advantages are, from left to right: contextual understanding, few-shot learning, dynamic text generation, and generalization with multitask capability. The middle sections present the current technical limitations of LLMs with the integration of trial scenarios, including output hallucination, prompt sensitivity, and challenges in updating the knowledge base. The lower sections list potential technical solutions to mitigate these limitations

Performance advantages

Context understanding

Compared to traditional NLP models (e.g., dictionary-based model, N-gram model, hidden Markov model, support vector machine) constrained by fixed window sizes and linear, sequence-based text input, the Transformer architecture of LLMs demonstrates notable advantages in capturing long-range dependencies and semantic connections, namely, superior context understanding [101, 102]. In the long-term follow-up period, trial records often chronicle the progression of illness and the medication history of subjects over months or even years. Superior context understanding allows LLMs to detect AEs or functional changes in post-treatment subjects, and establish long-range connections with baseline characteristics, assisting researchers in evaluating treatment efficacy and identifying potential response patterns over time.

Few-shot learning

Few-shot learning (FSL) is a learning approach designed for LLMs that leverages a limited number of annotated instances as prompts to learn specific tasks [103]. In the absence of sufficient external training data to serve as ground truth, traditional NLP models demand extremely high precision in textual representation, and the subsequent refinement processes are often highly complex, resulting in reduced flexibility [104]. FSL enables LLMs to perform specific tasks using only a small number of labeled examples, without requiring extensive architectural modifications or additional supervised training. This allows for rapid adaptation to trial-related tasks in low-resource settings, such as extracting clinical information of subjects with rare or novel diseases, or developing more precise informed consent materials for children, older adults, or minority populations.

Dynamic text generation

Leveraging the unique “emergent” ability [105], LLMs can generate text coherently, adjusting in real time based on immediate context. This dynamic adaptability offers practical value in clinical trial documentation. Medical writing often involves complex terminology and potential ambiguity. LLMs help ensure clarity and coherence by adjusting word choice and writing style based on contextual meaning and specific trial settings. For example, in the analysis of drug efficacy informed by participant data, LLMs typically adopt a precise and objective narrative style, often augmenting the use of specialized terminology. In contrast, when aiding participants in understanding informed consent documentation, LLMs employ a more accessible language, replacing technical terms with simplified lexis to improve text comprehensibility.

Generalization and multitask capability

LLMs trained on general-purpose datasets can transfer the experience and knowledge acquired in specific tasks to other related tasks, showcasing exceptional generalization abilities within condensed timeframes. In contrast, traditional NLP models often require different model parameters to be set for different scenarios, leading to reduced flexibility. Additionally, their unified Transformer architecture supports efficient knowledge sharing and parallel processing, enabling LLMs to perform multiple tasks concurrently [106]. As a near “plug-and-play” tool, LLMs might perform baseline features extraction and participant eligibility screening at the same time, reducing the burden on researchers when designing complex pipelines.

Technical limitations

Output hallucination

Hallucination often refers to the generation of unfounded details to support a viewpoint [107]. This error arises from the presence of false or highly redundant information in the training data [108], the use of high-uncertainty sampling algorithms [109], as well as exposure bias between text generation and inference [110]. In clinical trials, hallucinated outputs can distort trial outcome predictions, misguide eligibility assessments, or even fabricate patient data to fit expected conclusions. Mitigation strategies include credibility-based data weighting, controllable generation methods [111], and retrieval-augmented generation (RAG) [112].

Prompt sensitivity

LLMs may generate markedly different responses to similar prompts for the same task, such as when synonyms are used or the prompt style is adjusted, indicating that the quality of the model’s output is highly sensitive to prompt variations [113]. Considering the diverse medical record formats and rich free-text information in clinical trials, reducing the prompt sensitivity in a controlled manner will assist LLMs in adapting to different user input styles or various trial categories. Future research may explore soft prompts with learnable vectors and prompt calibration, in which meta-prompts are iteratively scored and adjusted based on user intent to improve LLM adaptability to downstream tasks [114, 115].

Challenges in knowledge update

After deployment, static LLMs lack the ability to automatically update their knowledge base, while frequent retraining often incurs substantial costs. Furthermore, LLMs may encounter catastrophic forgetting when learning new tasks [116]. These limitations might hinder LLMs from continuously learning and integrating external data or expert consensus, which might eventually impair the timeliness and accuracy of LLMs’ recommendations of clinical trials matched to participants, or the analysis involving rare diseases or advanced medications. The current strategy for updating the knowledge base mainly involves leveraging external retrieval information to override outdated content in the model’s output, such as data augmentation and internet augmentation [117].

Implementation challenges and potential solutions

In this section, we discuss the potential challenges and related solutions of deploying LLMs in clinical trials based on real scenarios, including (1) data quality control, privacy protection, and blinding requirements; (2) limited transparency in decision-making, biases in data and algorithms, and difficulty in reliability and explainability assessment; (3) legal and regulatory hurdles; and (4) standardization of LLMs evaluation and application. This discussion will offer insights for the promotion and future applications of LLMs in clinical trials (Fig. 4).

Fig. 4.

Challenges and potential solutions in the applications of large language models in clinical trials. This figure highlights four major challenges associated with the application of large language models (LLMs) in clinical trials: requirements of data quality, privacy, and blinding control; legal and administrative impediments; the imperative for standardization in evaluation and practice; and concerns surrounding model credibility and interpretability. Furthermore, the figure delineates techniques or cross-sector collaboration approaches, presenting potential solutions corresponding to each of the listed challenges

Data quality, privacy, and blinding control

During the protocol development of clinical trials, investigators typically establish DMPs in advance to address practical challenges in data management, such as quality control, privacy protection, and blinding control. (i) A primary concern for researchers is ensuring that LLMs used for data management do not become new sources of data contamination. As discussed in the Comparison between LLMs and traditional NLP models in clinical trials section, LLMs may compromise data quality in clinical trials by generating inconsistent, inaccurate, or biased outputs due to their sensitivity to prompts and potential misinterpretation of context. Embedding predefined rules into prompts or implementing contextual constraint mechanisms are key strategies to guide LLMs in generating outputs that meet established quality standards [118]. (ii) The private information of participants may be unintentionally leaked during the output process, potentially being maliciously exploited by cybercriminals. For data privacy safeguarding, the deployment of LLMs should incorporate multiple data encryption techniques, such as symmetric/asymmetric encryption [119] and k-anonymity [120]. In recent years, the emergence of fast fully homomorphic encryption (TFHE) has enabled LLMs to perform computation directly on encrypted data, balancing privacy protection with processing efficiency [121]. (iii) LLM-generated outputs for data management may inadvertently reveal group allocation to researchers, compromising the blinding of the trial. Therefore, researchers should implement role-based access control (RBAC) to restrict LLM access to randomization data [122] and establish output auditing mechanisms to prevent the generation of any information that could indirectly reveal intervention assignments.

Model credibility, interpretability, and bias

Compared to traditional NLP models, LLMs often lack transparency in their reasoning and decision-making processes, a challenge commonly referred to as the “black box” [123]. The limited interpretability of LLM frameworks may undermine the credibility of their outputs, particularly when LLMs are applied to tasks such as trial protocol optimization or patient stratification. Current research efforts, including model distillation [124], hybrid architectures [125], and explainable AI (XAI) techniques [126, 127], have proven effective in enhancing interpretability. However, most of these methods are developed for general tasks and have yet to be systematically validated in clinical trial settings. Integrating multiple XAI techniques to support causal reasoning and decision traceability may help build greater trust in LLM-assisted analysis in the future.

LLMs applied in clinical trials are vulnerable to algorithmic and data biases, potentially leading to unequal trial outcomes, skewed patient recruitment, and diminished credibility and fairness of recommendations [128]. Such biases may stem from underrepresented populations in historical datasets, subjective endpoint annotations, or limited language coverage [129]. To mitigate issues of data and algorithmic bias, future efforts could focus on improving the quality of training data by leveraging real-world data for pre-training, or employing advanced algorithms such as fairness-aware training or adversarial debiasing to mitigate biased samples [130].

Legal and regulatory concerns

The application of LLMs in clinical trials entails various complex legal concerns. (i) Data privacy protection: The training and fine-tuning of LLMs require large volumes of trial data, which contains a substantial amount of sensitive personal information. Legal and regulatory frameworks across the globe, such as the European Union’s General Data Protection Regulation (GDPR) and the US Health Insurance Portability and Accountability Act (HIPAA), impose stringent requirements on the protection of patients’ personal information during LLMs training, transmission, and storage [131]. (ii) Intellectual property disputes: Some developers use medical databases to train LLMs without proper authorization, infringing on the intellectual property rights of the data owners. Additionally, the copyright and originality issues surrounding the use of LLMs’ outputs are also a topic of ongoing debate. (iii) Assignment of responsibility: How to address the medical disputes and responsibility issues stemming from LLM-mediated decisions is an urgent challenge. The amendment to Sect. 1557 of the US Affordable Care Act stipulates that physicians or medical decision-makers “liable for medical decisions made in reliance on clinical algorithms,” indicating that LLMs do not yet possess independent diagnostic and decision-making authority [132]. (iv) Misinformation and academic integrity: The advanced ability of LLMs to closely mimic human writing might lead to the creation of inaccurate or even false information, resulting in misleading conclusions that could undermine academic integrity [133].

Recent legislative developments, such as the European Union’s Artificial Intelligence Act (2024), have initiated the classification of artificial intelligence systems based on risk levels; however, numerous challenges persist. Current legal frameworks frequently lack specific guidance for clinical trial applications, while inconsistencies in global regulatory approaches may hinder the achievement of unified oversight [134, 135]. To keep pace with the rapid advancement of LLMs, regulatory frameworks must be adaptive, evidence-based, and capable of maintaining equilibrium between safety and innovation. Mechanisms such as regulatory sandboxes may offer viable solutions by facilitating controlled clinical testing within clearly defined legal parameters.

Standardization of evaluation and practice

To date, regulatory agencies such as the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) have not issued specific guidelines for evaluating or implementing LLMs in clinical trials. Establishing unified standards and practical guidelines is therefore essential for facilitating their integration into the workflows. Given that manual assessment remains the primary approach for evaluating LLMs in the medical field [136], a “human-in-the-loop” strategy provides a practical framework for standard evaluation through human–machine collaboration. In this approach, clinical experts prompt models to measure the consistency between the LLMs’ outputs and professional opinions, and subsequently provide improvement recommendations. Moreover, quantitative metrics, including ROC curves and domain-specific checklists such as STAGER and QUEST, have been proposed to assess the LLMs’ output quality [137, 138]. Industry-specific benchmarks are equally important; for example, CTBench provides a comprehensive test suite for evaluating GPT-4’s ability to extract baseline characteristics of clinical trial participants [139]. Future guideline development should incorporate feedback from real-world clinical trial design and conduct, alongside expert consensus, while balancing potential benefits, performance limitations, and application risks. Additionally, open platforms, shared toolkits, and structured feedback mechanisms are needed to facilitate best-practice dissemination and collaborative progress in trial workflows.

Conclusion

The integration of LLMs in clinical trials encompasses a range of critical processes, including protocol development, informed consent, patient recruitment, safety monitoring, data management, and outcome prediction. Currently, some applications, such as trial data mapping and simulation-based evaluations, remain at the exploratory stage with limited validation, while applications targeting specific tasks like real-time adverse event monitoring remain underdeveloped. Going forward, clinical researchers must ensure that LLM deployment aligns with both the trial protocol and conduct management plan, and adheres to relevant legal and regulatory requirements. The establishment of standardized industry benchmarks, enhancement of transparency in model outputs, and active mitigation of data and algorithmic biases will be essential for unlocking the full potential of LLMs for broader and safer integration into clinical trials.

Abbreviations

RCT

Randomized controlled trial

AE

Adverse event

DL

Deep learning

NLP

Natural language processing

EHR

Electronic health record

GPT

Generative pre-trained transformer

BERT

Bidirectional encoder representations from transformer

LLM

Large language model

NER

Named entity recognition

DDI

Drug-drug interaction

PICO

Patient, Intervention, Comparison, Outcome

FSL

Few-shot learning

IRB

Institutional review board

ICF

Informed consent form

DMP

Data management plan

RAG

Retrieval-augmented generation

M-LLM

Multimodal large language model

OCR

Optical character recognition

XAI

Explainable artificial intelligence

Authors' contributions

Writing-original draft, A.Q.L., Z.H.W., A.M.J.; Conceptualization, Q.C., B.F.T., P.L.; Investigation, A.Q.L., W.Z.H.; Writing-review and editing, A.Q.L., W.Z.H., A.M.J., L.C., C.Q., L.X.Z., W.M.M., W.Y.G., D.Q.Z., M.J.X., G.D.C., S.K.P., H.Z.W., L.Z., H.G.Z., X.P.D., Y.X.W., J.Z., Q.C., B.F.T., P.L.; Visualization, A.Q.L, W.Z.H.. All authors have read and agreed to the published version of the manuscript.

Authors' twitter handles

X: @PengL_Robin.

Funding

This work was supported by grants from the Science and Technology Innovation Program of Hunan Province. (No. 2023RC3074).

Data availability

No datasets were generated or analysed during the current study.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.