Artificial intelligence for optimizing recruitment and retention in clinical trials: a scoping review

Abstract

Objective

The objective of our research is to conduct a comprehensive review that aims to systematically map, describe, and summarize the current utilization of artificial intelligence (AI) in the recruitment and retention of participants in clinical trials.

Materials and Methods

A comprehensive electronic search was conducted using the search strategy developed by the authors. The search encompassed research published in English, without any time limitations, which utilizes AI in the recruitment process of clinical trials. Data extraction was performed using a data charting table, which included publication details, study design, and specific outcomes/results.

Results

The search yielded 5731 articles, of which 51 were included. All the studies were designed specifically for optimizing recruitment in clinical trials and were published between 2004 and 2023. Oncology was the most covered clinical area. Applying AI to recruitment in clinical trials has demonstrated several positive outcomes, such as increasing efficiency, cost savings, improving recruitment, accuracy, patient satisfaction, and creating user-friendly interfaces. It also raises various technical and ethical issues, such as limited quantity and quality of sample size, privacy, data security, transparency, discrimination, and selection bias.

Discussion and Conclusion

While AI holds promise for optimizing recruitment in clinical trials, its effectiveness requires further validation. Future research should focus on using valid and standardized outcome measures, methodologically improving the rigor of the research carried out.

Introduction

Any computer program that shows characteristics indicative of intelligent behavior, such as problem-solving, learning, perception, and reasoning aimed at emulating, augmenting, and expanding human intelligence is considered to be a form of artificial intelligence (AI).¹ It encompasses a wide spectrum of realms, such as knowledge discovery, data mining, natural language processing (NLP), computer vision systems, speech recognition, and expert systems.¹ Collectively, these technologies form the backbone of AI, driving innovation and transformation across a multitude of industries and applications.² In recent years, modern AI technology has achieved notable advancements and is actively deployed across the medical domains.³ Medical services stand to gain significant benefits from the integration of AI across various domains such as patient administration, clinical decision support, patient monitoring, and healthcare interventions, thereby fostering the development of AI-augmented health systems.⁴ The latest frontier in AI’s positive impact on medicine lies in its application in the design, implementation, and analysis of clinical trials. AI technologies are becoming increasingly attractive to the clinical trial industry due to their automated nature, predictive power, and consequent expected increases in efficiency.⁵ AI has the capability to automatically identify meaningful patterns in large datasets, thereby expediting clinical trials through efficient participant match and comprehensive data analysis. The impact of AI technology on clinical trials has been steadily growing, leading to improved efficiency and reduced costs.⁶ AI also plays a crucial role in analyzing biomedical information and enhancing the decision-making process for patient recruitment in clinical trials.⁷ AI has the potential to address the shortcomings in traditional clinical trial design by harnessing the power of knowledge discovery, data mining, and NLP. These methods can be applied to diverse sources, such as electronic health records (EHRs) and medical literature, to improve patient-test matching and recruitment prior to trial initiation, and to enable automated and continuous patient monitoring throughout the trial process. Some recent tools have already shown promising results in terms of system information extraction. For example, researchers used NLP and other scalable methods to create the Clinical Trial Knowledge Base which summarized and standardized the free-text eligibility criteria from more than 350,000 trials registered in ClinicalTrials.gov.⁸ AI could provide a fast and facilitated approach to internal investigator ranking of patient eligibility, which would speed up site initiation and consequently positively impact recruitment.⁹ By leveraging AI in clinical trials, researchers can enhance their ability to identify and manage risks, thereby improving overall trial outcomes.

While significant progress is being made in the field of AI for clinical trials, it is crucial to acknowledge the responsibility that accompanies the opportunity to transform the clinical trial cycle through AI.¹⁰ The review of relevant regulatory documents indicates that AI is welcomed by regulators as an innovative technology with broad applications and disruptive potential but requires strategic planning for implementation.¹¹ Furthermore, the integration of AI in the recruitment and retention process of clinical trials can give rise to potential technical and ethical concerns.¹² For instance, incomplete data or selection bias in their data sources may hinder the accuracy of the AI tools and introduce bias into trial recruitment efforts.¹³ In light of these considerations, we have developed a comprehensive scoping review that aims to provide a detailed overview, description, and summary of the current status of AI implementation in recruitment and retention in clinical trials. Additionally, we aim to identify the benefits and drawbacks, ethical and technical issues with the application of AI in optimizing recruitment and retention in clinical trials. The ultimate goal of this review is to provide valuable insights and recommendations for the future development of AI in clinical trials.

Materials and methods

A comprehensive scoping review was conducted, encompassing both peer-reviewed and grey literature, to explore the use of AI in optimizing recruitment and retention in clinical trials. The review followed the methodological framework provided by Joanna Briggs Institute (JBI).¹⁴ To ensure transparency and adherence to reporting standards, the preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScRs) served as a guide for reporting the review¹⁵ (see Additional file S1).

Protocol registration

The study protocol was registered in Open Science Framework, an international platform for registration of reviews (https://osf.io/c9yhk/), and a paper was published in BMJ Open (doi: 10.1136/bmjopen-2023-080032).¹⁶

Stage 1: identifying the research question

Based on the objectives of this scoping review, we have formulated the following questions.

1. What specific types of clinical trials are AI tools focused on recruitment and retention predominantly applied to, and what are the design details (eg, study design) of those clinical trials?

2. What are the different types or names of AI tools that are currently available for assisting in optimizing the recruitment and retention process in the studies reviewed?

3. What are the outcomes, benefits, and drawbacks associated with the use of AI tools in assisting in optimizing recruitment and retention in clinical trials?

4. What are the ethical and technical issues encountered in the utilization of AI tools to assist in optimizing the recruitment and retention process in clinical trials?

Stage 2: identifying relevant studies

An electronic search was conducted in leading medical and computer science databases, including PubMed, Embase, Scopus, IEEE Xplore, and Web of Science Core Collection (SCI-EXPANDED, SSCI, A&HCI, and ESCI). We developed a comprehensive search strategy in the above databases (see Additional file S2). A search was performed on January 14, 2024 by X.L. English papers published with no time limits were included in this study. The search terms included artificial intelligence, AI, clinical trials, and recruitment and retention. Search terms were different according to different databases. Boolean terms were used to separate the keywords. To compile all relevant evidence sources, identify, remove duplicate records, and screen literature, Covidence was employed.

Stage 3: study selection

We established eligibility criteria to ensure that the selected studies included specific information relevant to our research question.

Inclusion criteria

Studies were included when they met the following criteria:

• The study focused on the application of AI technology.

• The study specifically addressed the use of AI technology to assist in recruitment or retention in clinical trials.

• The study was original research, descriptive study, randomized controlled trial, pilot study, or feasible or acceptable study.

Exclusion criteria

Studies were excluded when they were:

• Not available in English.

• Did not include the details of AI technology used.

• Just simply mentioned AI applications in recruitment and retention.

• Focused on clinical trials conducted on AI, rather than the application of AI to clinical trials.

• Not application of AI in recruitment and retention in clinical trials, but in other areas, such as AI in clinical trial design

Stage 4: charting the evidence

Covidence was used to identify and remove all duplicate articles. All publications identified according to these criteria went through title and abstract screening and then full-text screening, both of which were conducted independently by X.L. and C.Y. In case of disagreements or discrepancies, a consensus was reached through group discussions involving X.L., C.Y., and L.L.

Stage 5: extracting the data

A data extraction form was developed to systematically gather relevant information from the included articles. C.Y. and L.L. independently extracted the data from the selected studies using a pilot-tested form. In cases of discrepancies, X.L. was consulted to reach a consensus. Characteristics of included studies, including year of publication, country, scope of application, type of AI tools, study design, and aim of the study were extracted using the template in Additional file S3. We also extracted the funding sources of the included studies, presented in Additional file S4.

Stage 6: collating, summarizing, and reporting the results

The data collected were collated and summarized using a basic descriptive approach.¹⁷ The content analysis categorized the main outcomes, as well as the technical and ethical issues, benefits, and drawbacks that influence their development, implementation, and assessment.

Results

Selection of resources

The search yielded 5731 articles. A total of 1419 publications were found to be duplicates and were discarded, thus leaving 4312 articles, of which 4176 were excluded after evaluation of the title and abstract. After a full-text reading of the 136 selected articles, a total of 51 were included in the review (see Additional file S4 for key data extracted for each of the 51 studies and funding information), with a PRISMA flow diagram presented in Figure 1.

Figure 1.

The PRISMA-ScR flow diagram shown was used to illustrate the processes involved in conducting this scoping review. These processes mainly consisted of (1) identification: stated the number of articles identified and their respective sources, and provided the number of duplicates found and removed; (2) screening: provided the total number of articles screened and detailed information on the number of full-text articles that have been evaluated as eligible, the number of eligible studies, and the number of excluded studies; and (3) included: the number of studies provided in this section included following quality assessment, data extraction, and thematic analysis, respectively. PRISMA-ScR, preferred reporting items for systematic reviews and meta-analyses extension for scoping review.

Characteristics of included studies

The vast majority of publications had primary authors in the United States (n = 37),^9,13,18–52 others were conducted in China (n = 3),^53–55 Australia (n = 2),^56,57 United Kingdom (n = 2),^58,59 Canada (n = 2),^60,61 France (n = 2),^62,63 followed by Singapore (n = 1),⁶⁴ Japan (n = 1),⁶⁵ and Italy (n = 1),⁶⁶ and the most frequent clinical areas covered were oncology studies (n = 14), diversified clinical trials (n = 10) and Alzheimer’s disease (n = 9). While our literature search yielded publications between 2004 and 2023, most of them were published after 2020 (n = 35). The most frequent study designs were cohort studies (n = 14) and non-randomized controlled trials (n = 14). We classified case-control study, simulation study, comparative study, cross-sectional study, etc as other studies because the number of studies applying these methods is relatively small.

We categorized AI applications into the following 4 main types: (1) Medical tools, which were used here to refer to using a mature AI tool to assist recruitment in clinical trials. Medical tools like Watson for Clinical Trial Matching (WCTM) and Mendel.ai leveraged AI and machine learning (ML) to analyze diverse medical data sources for tasks such as patient selection. They offered features like predictive analytics and personalized medicine recommendations, thereby enhancing clinical trial recruitment workflows; (2) Traditional model-based approaches, which rely on established mathematical or statistical models, such as linear regression or decision trees, to make predictions or decisions. These approaches involved manually selecting features and fitting the model to the data; (3) Deep model-based approaches, modern deep approaches use complex neural network architectures to automatically learn patterns and relationships directly from the data. It does not require manual feature engineering and can capture intricate patterns that traditional models may miss; (4) Other, mixed methods, unknown methods, and unmentioned methods were grouped into this category.

The term “medical tools” in our article refers to commercially available tools like WCTM and Mendel.ai. These tools typically include a user interface designed for end-users, such as clinicians or researchers. They are used not only for clinical trials but also for a variety of other purposes in the medical field, such as patient data analysis and decision support. On the other hand, the “traditional and deep model-based” algorithms mentioned are primarily derived from recent academic research. These models are often in the experimental stage and have been tested in limited settings, such as small-scale pilot studies or controlled environments, rather than being widely deployed in real-world applications. Traditional approaches, deep methods, and medical tools all occasionally leveraged NLP to analyze clinical data. Four traditional methods utilized NLP in our review, where NLP extracted key information from unstructured clinical text, such as EHR, which was then used in statistical or ML models. Two deep methods employed NLP, with NLP being integrated into deep learning (DL) architectures to directly process raw text data, allowing the model to learn representations of the text’s meaning and context. Seven medical tools employed NLP to enhance various aspects of recruitment processes, such as eligibility screening and data extraction. These tools exploited NLP to automatically process and interpret unstructured clinical text from EHR and other medical documents, facilitating more efficient identification of potential trial participants. In all cases, NLP played a crucial role in extracting valuable insights from clinical text.

According to our research, all the AI applications included are solely used for recruitment and not for retention. The deployment of AI applications in real-world settings encompasses various types, including: (1) Implementing AI-based tools to enhance patients’ access to clinical trials. For instance, Jordan et al³⁶ utilized NLP to restructure trial information for efficient search and automated identification of relevant trials for individual patients. Patients interacted with the novel search tool through a free online website and completed a targeted questionnaire customized for each cancer type, focusing on key eligibility criteria that differentiated between trials to facilitate quick matching with relevant trials; (2) Utilizing AI tools for optimizing eligibility screening for clinical trials. For instance, WCTM consists of 3 main components: a trial data intake process, a patient data intake process, and a matching process. The matching process compares trial eligibility and exclusion criteria with patient data to generate a list of qualifying trials for patients. WCTM’s trial and patient data intake components use NLP to process unstructured information. In brief, the sponsor or client provides the trial protocol initially, which is ingested by the system. The output produced from WCTM’s trial intake process details criteria for the trial and provides additional information related to clinical concepts and values with associated WCTM attributes²²; (3) Enhancing patient identification. For example, Tissot et al⁵⁹ applied an algorithm to identify eligible patients using a moving 1-hour time window, they simulated the review of all patients in the intensive care unit (ICU) every hour, looking back on clinical data collected during the previous 24 hours to classify whether the patient had new onset septic shock, the patient was marked as eligible at the earliest timepoint that they fulfilled the selection criteria, and they were not considered eligible for future time points.

Table 1 lists a summary of the key characteristics of the included studies.

Table 1.

Key characteristics of the included studies.

Characteristics of included studies (N = 51)
Element in study	Categories	Frequency (%)
Publication year
2023		2 (4%)
2022		8 (16%)
2021		15 (29%)
2020		11 (21%)
2019		5 (10%)
2018		1 (2%)
2017		1 (2%)
2016		2 (4%)
2015		3 (6%)
2011		1 (2%)
2005		1 (2%)
2004		1 (2%)
Country
United States		37 (72%)
China		3 (6%)
Australia		2 (4%)
Canada		2 (4%)
France		2 (4%)
United Kingdom		2 (4%)
Italy		1 (2%)
Japan		1 (2%)
Singapore		1 (2%)
Scope of application
Alzheimer’s disease		8 (15%)
Diversified clinical trials		7 (14%)
Oncology studies		14 (27%)
Neuroprotective trials		1 (2%)
Stroke trial		2 (4%)
Acute respiratory distress syndrome		1 (2%)
Age-related macular degeneration		1 (2%)
Diabetes		1 (2%)
Cardiovascular trial		2 (4%)
Heart disease		3 (6%)
Knee osteoarthritis		1 (2%)
Critical care		1 (2%)
Non-alcoholic steatohepatitis (NASH) clinical trials		1 (2%)
Patients diagnosis of RA		1 (2%)
Emergency department		2 (4%)
People living with HIV (PLWH)		1 (2%)
Response-adaptive randomization (RAR) design		1 (2%)
Schizophrenia		2 (4%)
Transgender population		1 (2%)
Type of AI tools
Medical tools		10 (20%)
Traditional model-based approaches		22 (43%)
Deep model-based approaches		5 (10%)
Other		14 (27%)
NLP		13 (25%)
Study design
Cohort study		14 (27%)
Non-randomized controlled trial		14 (27%)
Other		14 (27%)
Descriptive research		4 (8%)
Pilot study		3 (7%)
Randomized controlled trial		2 (4%)

Abbreviations: AI, artificial intelligence; NLP, natural language processing.

Benefits and drawbacks

Table 2 summarizes the benefits and drawbacks that arise during the application of AI. AI systems can be integrated into recruitment in clinical trials in a variety of ways and bring convenience and benefits along with many shortcomings.

Table 2.

Description of benefits and drawbacks.

Main theme	Subtheme
Benefits of AI tools	Increasing efficiency, cost savings, and improving recruitment9,13,19,20–24,26,28–31,33–45,48,49,52–54,56–61,65,66
	Improving accuracy and predictive power9,19,20–29,31,33,36,38,39,41,42,44,48,49,51,57,59,61
	Create user-friendly interfaces and improve patient satisfaction58,60
	Allow remote access and monitoring45,55
Drawbacks of AI tools	Unreliability of results under data constraints9,13,19,22,24–27,29,31,32,34,35,37,39,43,44,47,51,56,57,59,64
	Limited generalizability13,18,21–24,29–31,33,35,37,42,44,45,48,49,54,57,61
	The complexity of algorithms and blind trust issues9,28,36,39,43,44,47,48,52,53,56,60,62,64
	Insufficient external validation data and lack of universal guidelines22,27,29,35,40,51,56
	Adds additional costs21,62

Abbreviations: AI, artificial intelligence.

One of the main drawbacks of implementing AI was the reliability of results under data constraints. Errors and incompleteness in the data sources used to train the AI algorithms and in the available data for potentially eligible individuals can lead to problematic results. Limited generalization capabilities were also a significant drawback; AI tools developed for specific purposes (eg, breast cancer lexicon³¹) may perform poorly in dealing with different populations or situations, and the lack of generalizability and transferability will limit the scalability and use of AI tools in clinical trials. It is also important to identify that the complexity of algorithms and blind trust in AI tools could hinder the recruitment process. Incomplete algorithm data training sets may lead to bias in patient inclusion selection,³⁷ and excessive trust tends to overlook the possibility of AI making mistakes. Insufficient external validation and lack of universal guidelines for AI tools, as well as additional costs, are also important considerations. Lack of content quality assessment or task-based evaluation could lead to inaccuracies in AI systems, and the lack of uniform standards for the application of AI to clinical trials may lead to nonstandard application of AI tools and confusion in clinical trial transformation, such as nonstandard informed consent notification.

On a positive note, we have identified some benefits for the effective implementation of AI tools. Increasing efficiency, cost savings, and improving recruitment were essential benefits that can help promote recruitment in clinical trials. We define efficiency as completing tasks with fewer resources,⁶⁷ which implies spending less time and incurring lower costs to obtain more patients who meet the enrollment criteria.⁶⁸ For instance, in an ongoing oncology trial, the utilization of Mendel.ai led to a 24%-50% increase in the number of patients accurately identified as potentially eligible compared to the standard practice. Furthermore, using Mendel.ai to determine potential eligibility only takes minutes, while it takes an average of 19 days for standard prescreening of breast cancer patients and 263 days for lung cancer patients.²¹ Improving accuracy and predictive power was also crucial for involving communities for better acceptance. Creating user-friendly interfaces and improving patient satisfaction were crucial for researchers and other team members in the recruitment process. Allowing remote access and monitoring could provide valuable improvements for clinical trials.

Technical issues

Facilitators

The AI system itself possesses numerous advantages that can promote its application in clinical trial recruitment, and we consider these advantages as its facilitators. Firstly, Mutasa et al³⁰ showed that AI systems allow the computer to automatically construct predictive statistical models, tailored to solve a specific problem subset. For example, the performance of Convolutional Neural Networks (CNNs) has been demonstrated to logarithmically increase with larger datasets, thereby enhancing its effectiveness in clinical trial recruitment. Additionally, utilizing tools like CogStack can potentially reduce the number of patient notes that need manual review by 85%, achieve rapid search and filtering, reducing the workload of manual audit.⁵⁹ Secondly, Cai et al⁴⁰ mentioned that by incorporating data from EHR, the algorithm significantly reduced the number of patients requiring manual chart review by 40.5%, while not excluding eligible patients, thereby allowing for a more accurate screening process. Thirdly, Kehl et al³⁹ showed that the heterogeneity of the training and testing data significantly strengthens the model to identify patients. Tomaszewski et al⁴⁹ showed similar results, by identifying an enriched subset (52%) of patients who had a significantly longer overall survival in cohorts treated with doxorubicin. Finally, the use of algorithms to identify eligible patients from EHR can streamline the recruitment process and reduce the necessity for manual chart review. Conducting consent, intervention, and follow-up remotely enables greater accessibility and participation, particularly for patients who are unable to travel to enroll sites.⁵¹

Barriers

However, AI itself also has several shortcomings that could hinder its use in clinical trial recruitment, such as the black box problem, and we consider these shortcomings as its barriers. For example, it should be considered that although using these methods may increase the efficient use of resources dedicated to clinical trial recruitment, unexpected biases may be introduced into the clinical trial cohort.¹³ Sato et al⁶⁵ mentioned that AI tools could limit the applicability of the A4-fitted models to younger participants. The limited quantity and quality of sample size (training data) was also an important issue. For example, Lanera et al⁶⁶ mentioned that the predictive performance of the proposed tool was not optimal for the case study considered due to the limited sample size. Besides, because training a CNN is an end-to-end process, it did not clearly reveal the reasoning behind the final result in a deterministic manner (ie, black box operation)³⁰ and Fink et al³² found that the system did not keep track of temporal changes in the data. Finally, integrating the framework into existing systems and ensuring its usability was also a challenge since it has not been applied in a real-world setting and consumer-oriented search engines for clinical trial registries and/or patient-trial matching systems, and the generalizability of the approach to other common diseases has not been investigated.⁴⁵

Ethical risks

Despite its many advantages and facilitators, the use of AI in clinical trial recruitment inevitably raised a host of ethical issues.

Privacy and data security were the main ethical issues associated with the use of AI in clinical trial recruitment; 15 studies included in this review mentioned them. For example, AI platforms often require access to sensitive patient data, such as medical images. It is crucial to protect the privacy and confidentiality of these data, ensuring that they are securely stored and only accessed by authorized personnel.²⁵

Informed consent was another ethical issue related to the use of AI tools. Kirshner et al⁵⁰ mentioned that ethical concerns arise regarding informed consent from patients for the use of their EHR data in research studies. Patients should be adequately informed about how their data will be used and have the option to opt-out if they do not wish to participate. Schwager et al²⁶ showed that EHRs could raise questions about who owns and controls the data. Patients should have control over their own health information and be able to make decisions about how it is used and shared. When using EHR or other health data sources, researchers should consider the ethical implications of data sharing and secondary use. They should obtain necessary permissions and adhere to data protection regulations to ensure that data are used responsibly and for legitimate purposes.⁵⁹

Transparency was also an important ethical consideration of AI. Studies by Vazquez et al¹³ and Chen et al¹⁸ have shown that it is crucial to ensure that AI-driven patient enrichment is implemented in a fair and responsible manner. Hassan et al²⁵ and 3 other studies^13,18,53 showed that the use of AI in clinical trial recruitment should be transparent, with clear communication about how the technology works and the role it plays in the decision-making process. There should also be mechanisms in place to address any concerns or complaints related to transparency and accountability of the AI system.

Eleven studies mentioned fairness, discrimination, and selection bias problems in AI tools. For example, ML models may inadvertently perpetuate biases present in the training data, leading to unfair or discriminatory outcomes.²⁷ Kirshner et al⁵⁰ mentioned that EHRs rely on accurate and up-to-date information to provide quality care. Ethical concerns arise when there are errors or discrepancies in the data, which can lead to incorrect diagnoses or treatments. Healthcare providers have an ethical responsibility to ensure the accuracy and integrity of the information entered into EHR.

Discussion

Predominant trends of AI application in clinical trial recruitment

The results of our scoping review of studies of the application of AI for optimizing recruitment and retention in clinical trials reveal several predominant trends. Firstly, we have observed a notable increase in publications since 2020, with 11 in 2020 compared to 5 in 2019, and over 80% of articles were published between 2019 and 2023. This indicates an increasing interest in using AI as an adjunct for clinical trial recruitment and this surge can be attributed to 2 main factors. First, the field of AI itself has undergone transformative advancements, particularly in ML and related DL methods, facilitated by hardware enhancements and extensive training datasets.⁶⁹ Additionally, medical data have become increasingly available in digital format due to technological advancements and public policy initiatives. The large-scale application of medical data, such as retrieval of patient information buried in the EHR, has facilitated the transformation of clinical trials to intelligence. For instance, medical data can support automatic or semi-automatic clinical trial screening, either through a patient-centered approach or in the form of decision support for clinicians, such as point-of-care alerts.⁵⁹ Second, the outbreak of COVID-19 has driven significant digital transformation within clinical trials.⁷⁰ There is a heightened interest in leveraging AI and big data analytics across clinical trials, especially for recruitment. This is evidenced by the increasing number of startups operating within this domain, rising agreements for drug development activities, and record levels of funding.⁷¹

Secondly, an AI technology needs to be working well before real-world testing and implementation. Good efficiency, accuracy, and patient satisfaction and so on have been demonstrated in our research findings. Based on our review, in comparison to other features, studies have frequently focused on the development of AI for identifying patients who are more suitable or interested in participating in clinical trials. This emphasis may stem from the potential simplification of patient identification through algorithmic automation, which can identify optimal trial participants and input their medical history and details into trial databases.⁷² Research by Chopra et al⁷¹ also demonstrates that AI’s application in patient identification is particularly prominent, especially in the analysis of social media information. AI has the capability to analyze online discussions from patient support groups to detect clusters of illnesses within specific regions. By analyzing population data, AI can aid in identifying individuals who would derive maximum benefit from participating in clinical trials.⁷³ Through evaluating hospital medical data and informing both physicians and patients about potential clinical trial opportunities, AI holds promise for expediting the process of finding suitable participants.⁷¹

Interaction between benefits, drawbacks, technical issues, and ethical issues

Cascini et al⁷⁴ conducted a scoping review of the current landscape of AI-based applications in clinical trials. Their study presented encouraging results on the implementation of AI-based applications to the development of clinical trials, showing that AI-based applications have a lot of potential. In comparison to their review, our research raised new questions, such as technical issues, ethical issues, benefits and drawbacks of AI tools, and so on, each of which were both interdependent and mutually reinforcing. Discrimination and selection bias, for instance, were noted ethical concerns when utilizing AI for recruitment in clinical trials, while also recognized drawbacks in limiting the generalizability because of potential data bias. The security of confidential patient data, in turn, was identified as critical for extending generalizability in the use of AI technology to recruitment and in clinical trials. In addition, the quality and diversity of training data significantly impact the accuracy and reliability of AI models, and insufficient data pertaining to uncommon diseases or conditions may impede the effectiveness of AI models within clinical trial environments.⁷¹ One suggestion offered to combat the threat of discrimination and selection bias in AI is the larger sample size and other cohorts, and a longer duration of follow-up or using different subsets.

Furthermore, acquiring informed consent from patients during the deployment of AI algorithms can pose challenges due to their limited comprehension regarding operational mechanisms and data utilization practices. Ensuring transparency and comprehensibility is essential for fostering patient confidence and upholding rigorous ethical standards in relation to these advanced technologies.⁷¹ Nevertheless, numerous types of AI such as DL neural networks are often labeled as “black box” models owing to their intricate internal processes that defy easy comprehension or explanation. This lack of interpretability could hinder widespread adoption given that explanations for specific predictions or recommendations made by an AI model are frequently ambiguous.⁷¹ It is therefore clear from our review that the aforementioned themes cannot be considered in isolation, but rather must be viewed in relation to one another issues when considering the application of AI in clinical trials.

Lack of application of AI in retention

None of the studies identified addressed retention of participants. An analysis involving 95 clinical trials revealed that close to 40% of patients discontinued their prescribed medication within the initial year.⁷⁵ The lack of adequate technical infrastructure to manage trial complexities without reliable adherence control and patient monitoring may result in underpowered trials,¹⁰ while elevated nonadherence rates pose substantial challenges for sponsors and researchers. Despite the seriousness of the retention problem, our review found no research on applying AI to it. Abiodun et al⁷⁶ developed a conceptual framework of remote health monitoring in a clinical trial using ML techniques, with various classifications to determine if a participant should be allowed to continue in the trial or not, but did not validate the effect of this framework.

In a recent review article by Blaschke et al,⁷⁷ various applications of AI were discussed in clinical trial settings. These applications encompass utilizing historical medical data for predicting potential dropouts among patients in order for medical professionals to provide timely intervention and employing AI technology for analyzing video recordings of patients while taking medications aimed at ensuring accurate dosages. Currently, there are few startups focusing specifically on preventing clinical trial dropouts as well as engaging participants, Brite Health stands out as a company actively addressing clinical trial dropout rates through its innovative product which processes both structured and unstructured patient data before delivering tailored messages and alerts designed to sustain participants’ involvement in trials while also identifying individuals at risk of dropping out for prompt intervention.⁷⁸ Additionally, solutions such as emocha Mobile Health and AiCure aim at enhancing medication adherence through digital methods known as directly observed therapy (DOT), where an AI system monitors patients’ drug intake.⁷⁸ Notably complex strategies have been necessary even for minor enhancements whereas simple measures often yield limited impact. Furthermore, without continuous reinforcement tactics, all intervention effects tend to diminish over time,⁷⁹ highlighting the need for multifaceted retention approaches incorporating patient-centric care along with education, dosage monitoring, counseling, prompt packaging, and reminders.⁸⁰ As patient dropout will likely continue to be a significant problem in the foreseeable future, this area has plenty of opportunities for innovation.

Classical models as dominant subfields, alongside a general transition toward deep models, medical tools, and NLP

The dominant subfields of AI identified by our review mirror trends in AI advances and align with other characteristics of the included studies. More than 40% of the studies reviewed used traditional model-based approaches, but the use of medical tools (20% of studies) and deep model-based approaches (5%) is increasing, reflecting a general shift in AI transitioning from smaller and structured datasets to massive and unstructured one. This trend also aligns with the current development of AI technology. Classical, or “non-deep” ML is more dependent on human intervention to learn; human experts determine the set of features to understand the knowledge within input data, usually requiring more expertise knowledge and structured data to learn.⁸¹ However, the driving force behind the maturity of AI research is the availability and quality of data, particularly after the shift toward data-driven deep models.⁸² Despite the potential for AI to revolutionize clinical trials, it still encounters numerous challenges, with unstructured data being a significant obstacle. For instance, in many clinical studies, faxed requests for patient records remain common, and hospitals often respond with PDFs or photos.⁸³ When a spreadsheet is faxed or converted to a read-only document, much of its original structure is lost. Researchers conducting clinical trials struggle to gather precise information required to assess a patient’s eligibility using the current manual approach.⁸⁴ Deep models enable systems to learn and integrate feedback on the quality of their analytic output into adapted underlying algorithms. Assistive systems utilizing these AI techniques or subsets thereof can automatically analyze EHR and clinical trial eligibility databases, identify matches between specific patients and recruiting trials, and recommend these matches to doctors and patients.⁸⁵

Additionally, NLP, a theory-driven range of computational techniques for automatic analysis and representation of human language from various unstructured data types,⁸⁶ has demonstrated its value in clinical trials. For instance, the use of the IBM Watson Health’s Clinical Trials Matching (CTM) system with screening coordinators facilitated an increase in clinical trial enrollment and promoted awareness of clinical trial opportunities within the lung oncology practice.³⁸ This system employs NLP to abstract patient and trial data from unstructured sources, aiding in matching patients to trials. It has shown positive results in increasing enrollment in clinical trials and raising awareness of clinical trial opportunities within the field of lung oncology. Other studies have demonstrated that the utilization of automated EHR text-mining for identifying suitable participants for cardiovascular trials could result in a 79.9% reduction in the number of patients needed to be screened.⁸⁷ Additionally, an information extraction approach to the EHR has proven effective in efficiently identifying and subphenotyping patients with heart failure with preserved ejection fraction and other disorders, achieving a sensitivity and positive predictive value of 0.95 and 0.86, respectively.⁸⁸ NLP integrates computational linguistics to empower computers to understand human language in both text and voice formats, comprehending its full meaning, including the speaker or writer’s intent and sentiment.

The utilization of AI-powered medical tools in clinical trial recruitment, such as WCTM and Mendel.ai, has been increasing, paralleling the trends in DL and NLP. These tools leverage AI to enhance various aspects of trial recruitment, including patient selection and eligibility screening, driven by the growing availability of large-scale digital medical data and the demand for more efficient and accurate recruitment processes. Advances in AI have enabled the development of more sophisticated tools capable of processing extensive datasets with higher precision. Combined with the digitization of healthcare records, regulatory acceptance, and the digital transformation accelerated by the COVID-19 pandemic, it has further fueled their adoption. As these trends continue, the deployment of AI-driven tools is expected to expand, contributing to the broader integration of advanced AI medical tools in clinical trial recruitment process.

Limitations

Although 51 studies were included in our research which described the brief current status of AI application in recruitment in clinical trials, the results of the interventions were too heterogeneous to perform a meta-analysis. By using search terms only in the English language, we might have not been able to identify articles in the national languages of different countries. In our review, no manual search was conducted and grey literature was not included. Due to the nature of this review, there were also heterogeneous populations, variable outcome measures, variable study methods, and methodological limitations, so we were not able to do a formal assessment of the predictive validity of the different AI models. The study found that current studies lack evidence of AI benefits and drawbacks, and it is difficult to draw conclusions and synthesize studies with inconsistent outcome measures, so there is an urgent need for a standardized measurement method to standardize measurement results. Finally, as most of the included studies were conducted in the United States, it is unclear whether the findings of these studies can be generalized to other countries, especially developing countries. However, our work could inform systematic reviews aimed at answering more focused research questions about the use of AI for specific diseases in clinical trials, including which AI techniques are most appropriate given different contexts.

Conclusion

To our knowledge, this review is the first comprehensive, interdisciplinary summary of research on AI application in recruitment and retention in clinical trials. Its 2 fundamental aims were to provide a comprehensive overview of the current status of the use of AI tools in the recruitment and retention process of clinical trials and to identify the benefits and drawbacks and ethical and technical issues that have influenced the application of these tools. Overall, the application of AI in clinical trials is in the early stages of maturity with not a lot of tools ready for widespread implementation. Its effectiveness needs to be further tested, requiring more obvious, higher-quality research evidence and clearer reporting than identified in this scoping review. The application of AI innovation can foster clinical trial transformation and improve clinical trial systems’ efficacy and quality. Future research should focus on using valid and standardized outcome measures, improving the methodological rigor of studies, formulating relevant policies, and issuing corresponding guidance documents to facilitate the engagement of clinical researchers and ethicists in AI-based clinical trials.

Author contributions

Xiaoran Lu led the drafting and writing of the manuscript, Chen Yang contributed to preparing the additional files and reviewing the article. Lu Liang assisted in developing the methodology of the protocol, designing the search strategy, and reviewing the article. Guanyu Hu and Ziyi Zhong provided feedback and critically revised the manuscript for important intellectual content. Zihao Jiang, the senior author, formulated the study conception with Xiaoran Lu, assisted in drafting the protocol, and reviewed the article. All authors read and approved the submitted copy.

Supplementary material

Supplementary material is available at Journal of the American Medical Informatics Association online.

Funding

This work was supported by the Central South University Graduate Independent Innovation Project (2022ZZTS0333). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Conflicts of interest

The authors declare no conflict of interest.

Data availability

The data underlying this article are available in the article and in its online supplementary material.