Embracing Generative Artificial Intelligence in Clinical Research and Beyond

Abstract

To explore threats and opportunities and to chart a path for safely navigating the rapid changes that generative artificial intelligence (AI) will bring to clinical research, the Duke Clinical Research Institute convened a multidisciplinary think tank in January 2024. Leading experts from academia, industry, nonprofits, and government agencies highlighted the potential opportunities of generative AI in automation of documentation, strengthening of participant and community engagement, and improvement of trial accuracy and efficiency. Challenges include technical hurdles, ethical dilemmas, and regulatory uncertainties. Success is expected to require establishing rigorous data management and security protocols, fostering integrity and trust among stakeholders, and sharing information about the safety and effectiveness of AI applications. Meeting insights point towards a future where, through collaboration and transparency, generative AI will help to shorten the translational pipeline and increase the inclusivity and equitability of clinical research.

Central Illustration

Highlights

• •Generative artificial intelligence will disrupt existing clinical research paradigms.
• •There are challenges to artificial intelligence's transformative potential; collaboration is crucial for future progress.
• •Regulatory guidelines and continuous monitoring, evaluation, transparency, knowledge sharing, and inclusivity should be established.

In the rapidly evolving field of artificial intelligence (AI), generative AI has emerged as a disruptor. Its unique capability to generate new data, from visual content to complex molecular structures, demonstrates a leap not only in the creative potential of AI but also in its capacity to address complex challenges previously believed to be beyond reach.1, 2, 3 Groundbreaking progress with AI has been made in critical areas such as drug discovery, patient diagnosis, and the customization of treatment strategies, paving the way for a new era of personalized medicine.4, 5, 6 The successful integration of generative AI into clinical research will mark a significant breakthrough, redefining traditional processes.

The current landscape for generative AI in clinical research is characterized by rapid expansion of efforts to eliminate administrative and process redundancies. Advances are increasingly acknowledged by agencies such as the Food and Drug Administration and the National Institutes of Health, which are crafting guidance on the use of generative AI for various clinical research practices.^7,8 While innovation has the potential to notably reduce the overall time and cost of therapeutic development, advances do not come without risks, and progress must be accompanied with careful mitigation of potential individual and societal harms, including misleading outputs, bias propagation, and loss of privacy.9, 10, 11, 12 In this context, a Duke Clinical Research Institute Think Tank was convened to explore the extensive implications of generative AI within clinical research.¹³ Based on the resulting observations, key themes and actionable items for the adoption of generative AI in clinical research are outlined, striving to leverage its benefits while ameliorating the associated challenges.

Methods

The 2-day think tank workshop in January 2024 gathered experts from the academic, government, and industry sectors to identify critical gaps and strategize innovative solutions for augmenting clinical research with generative AI. Participants engaged in 8 hours of moderated conversation and 4 hours of informal discussions on topics including automating clinical research processes using AI tools, engaging participants and communities directly with AI, and leveraging AI for measurements and analysis (Figure 1). Each moderated session began with prepared remarks from 3 to 5 experts, followed by in-depth discussions for a comprehensive exchange of ideas and experiences. The workshop was recorded and summarized in a meeting brief.¹⁴ The present manuscript was conceptualized, drafted, and revised based on the insights gained in the workshop, with drafts circulated among attendees for ongoing discussion and synthesis.

Figure 1.

Summary of the Workshop Session and Topics

The sessions were structured around several discussion points: efficiency, engagement, novel endpoints, and overarching impacts. We aimed to cultivate a consensus among the diverse participants on the scope of generative artificial intelligence, the practical and ethical concerns, the required validation, and regulatory landscape within each of the sessions. AI = artificial intelligence.

Results and discussion

The conversations highlighted the following: 1) the capabilities and potential impact of generative AI; 2) challenges and ethical considerations for generative AI adoption; 3) progress on guidelines and safeguards; 4) the need to ensure trust and transparency; and 5) actionable items for the adoption of generative AI in clinical research (Central Illustration).

Central Illustration.

Capacities, Impact, and Strategic Pathways of Generative Artificial Intelligence in Clinical Research

Generative artificial intelligence will facilitate both incremental and transformative changes. Collaboration and knowledge sharing will be key to navigating the challenges and ethical considerations inherent in its deployment. AI = artificial intelligence.

Capabilities and potential impact

Currently, clinical trials suffer from significant administrative burden, issues with participant recruitment and retention, burdensome data entry, and inadequate representation limiting trial generalizability, among other challenges.15, 16, 17 To address these challenges, numerous incremental process improvements, as well as more transformative changes to redefine the landscape of clinical research, were identified in the workshop (Table 1).

Table 1.

Potential Impacts of Generative AI on Clinical Research Processes

Clinical Research Process	Current Approach	Potential Changes Introduced by Generative AI
Incremental improvements
Document generation	Manual creation of research tools and documents.	Generation of drafts by AI, speeding up document creation and ensuring consistency.18
Participant engagement	Traditional methods for recruitment and dissemination of results.	Direct engagement through AI, from recruitment to results sharing, making the process more inclusive and efficient and improving retention.19,20
Potential transformative changes
Clinical trial design	Design based on historical data and standard practices.	AI-simulated scenarios and optimized protocols, increasing the success probability of clinical trials.21, 22, 23
Informed consent process	Standardized documents and procedures, often not fully tailored to individual comprehension levels.	Facilitation of a more personalized, engaging consent process through advanced technologies such as chatbots or avatars.24,25
Data collection, analysis, and synthesis	Reliance on conventional data gathering and analysis techniques.	New avenues for AI-driven data collection and synthesis, enhancing accuracy and depth of research findings.25, 26, 27
Ethical and bias mitigation	Policy and standard practice, with varying degrees of effectiveness.	Proactive efforts via AI to identify and correct biases in data and research practices, even within AI tools themselves, reinforcing equity and ethical integrity.28
Data management and security	Management through established IT protocols and practices, which involves challenges in data discoverability and reusability.	AI-enhanced data management, ensuring data quality, privacy, discoverability, and security, while upholding research integrity and reproducibility standards.29,30

Bullet points indicate more incremental improvements, and stars represent potential transformative changes.

AI = artificial intelligence.

While nongenerative AI can integrate complex inputs from clinical data to make predictions of patient outcomes or patients' potential to benefit from a specific therapy, generative AI extends this ability by using clinical data to create new content, including text and images.³¹ Generative AI can be harnessed to create synthetic data sets for simulations that further improve prediction performance.³² Generative AI can facilitate research by rapidly producing initial drafts of trial protocols and generating informed consent documents.^18,24,33 These advancements constitute incremental improvements by creating administrative efficiencies, which will result in faster trial development, more streamlined operations, and reduced costs that together could substantially shorten the overall drug development timetable.⁹

Beyond such incremental improvements, generative AI provides an opportunity for transformative change. AI-driven technologies can be used to reimagine clinical trial design by simulating potential clinical trajectories and optimizing protocols.21, 22, 23 Generative AI can revolutionize the consent process and disrupt current methods for data collection and analysis by directly engaging and communicating with individuals and communities.34, 35, 36, 37 Frequent bidirectional engagement with research participants throughout the clinical trial process can also create opportunities to establish novel endpoints, including from data as wide-ranging as those from imaging, wearables, and participant interactions with an app.^25,38,39 Such strategies could enable earlier identification of disease onset or progression, facilitating faster studies and therapeutic development. Furthermore, AI has the potential to provide new avenues for data collection and synthesis while also upholding research integrity and reproducibility standards by ensuring data quality, privacy, discoverability, and security.^26,27,29,40 These efforts could pair with the ability of generative AI to identify and correct biases in data and research practices to enhance the accuracy and depth of research findings while also reinforcing equity and ethical integrity.²⁸

Overall, generative AI holds the potential to propel clinical research into a new era of efficiency, precision, and discovery.

Challenges and ethical considerations

Generative AI is being rapidly adopted. The widely used ChatGPT had over 180 million active users at the time of the think tank, and a third of U.S. adults 18 to 64 years old now use generative AI every week.⁴¹ Generative AI is commonly being used for applications such as document generation. As these technologies mature, the use of generative AI is expected to increase significantly, becoming more prevalent in clinical settings and more familiar to research participants.^22,42,43 The pace of adoption underscores the urgency for organized, informed, and strategic integration of AI-informed tools into clinical research.⁴⁴ Before technologies become fully embedded, the full spectrum of benefits and harms that generative AI has within the clinical research domain should be systematically assessed. Without sharing experiences, both positive and negative, the field runs the risk of creating random practice without truly improving it. The speed of implementation must be carefully balanced with the need for rigor, safety, and caution.

Achieving the full potential of generative AI via safe and effective adoption must account for the unique risks of generative AI: generation of misleading or erroneous output (hallucinations), inappropriate sharing of protected health information, coercive engagement, perpetuation of bias, limitations in interpretability, inaccurate broad generalization from limited training data, and human overreliance on models.^45,46 Risks must be sufficiently mitigated so that the foundation for using generative AI in clinical research is trustworthy according to ethical, technical, and regulatory considerations.

The vast potential for generative AI to interact with research participants must be balanced with the imperative to preserve participant autonomy and maximize the generalizability of results. Chatbots that allow for personalized communication increase engagement, yet the risk is that communication might become coercive or manipulative.⁴⁷ This could reduce the voluntariness of consent and introduce bias in participant responses.⁴⁸ Generative AI's ability to recreate its training data, potentially revealing individual patient information, creates confidentiality risks.⁴⁶ Additionally, acceptable boundaries regarding which data are usable must be determined. Generative AI can train on vast amounts of input, including the entire digital footprint of participants such as social media patterns, which could help predict mental health status or cardiovascular risk.^49,50 However, the extent to which these data can be used for disease prediction or trial outcomes must be agreed upon across all stakeholders.

Beyond concerns related to the acceptable use of data, careful consideration of training data is critical to avoid perpetuation of bias. For example, the commonly used Generative Pretrained Transformer model propagates racial and gender stereotypes across clinical use cases, consequences of which have included overestimating the prevalence of HIV in Black patients and prioritizing a diagnosis of panic disorder for women, but not men, presenting with shortness of breath.⁵¹ Diagnostic accuracy is at risk, even when models reveal their biases.⁵² If uncorrected, such biases would have deleterious implications across the clinical research spectrum, including in how generative AI interacts with participants, how it is used to interrogate existing data, and how it is used to generate new data.⁵³ Most critically, bias in the clinical research process would systematically bake inequities into health care for generations to come and should be avoided at all costs.

From a technical perspective, generative AI is expected to achieve acceptable accuracy and relevancy over time. However, it has limitations, such as producing incorrect or misleading outputs (hallucinations), and its performance may degrade, particularly if trained on biased or outdated data.54, 55, 56, 57, 58 The dynamic performance of generative AI will necessitate continuous surveillance throughout the use lifecycle. Robust monitoring will require human oversight and may demand the development of additional AI tools to help assess model performance.⁵⁹ Further, new metrics for model evaluation and benchmarking are needed, especially given the nondeterministic nature of generative AI outputs.⁶⁰ Transparency in model development and the data used for training are crucial. Secure data sharing systems are essential to understand model limitations and ensure diverse patient data in model development.⁶¹ Additionally, while synthetic data created by generative AI can help simulate expected changes in population data, they must be used with caution to prevent unintentional leakage of personal health information.^30,62

Progress on guidelines and safeguards

Reflecting numerous concerns related to the use of AI in clinical research, regulatory agencies have initiated efforts to establish safeguards and guidelines. A 2023 Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence highlighted the essential role of oversight in the rapidly developing field of generative AI.⁶³ A discussion paper from the Food and Drug Administration underscored the need for collaboration among stakeholders, encouraging a regulatory approach that supports innovation, development, and sharing of best practices and ongoing evaluation and monitoring of AI performance.⁶⁴ In addition, the Center for Drug Evaluation and Research (CDER) recently established the Center for Drug Evaluation and Research AI Council to provide coordinated and consistent initiatives for regulatory decision-making and to better support innovation and best practices across AI-enabled medical products.^8,65 Regulation will be essential for harnessing the transformative potential of generative AI.⁶⁶ Additionally, it is critical that regulatory guidelines are iterative, reflecting the rapid pace of evolving technology.

The scientific community has also worked to enhance the evaluation of AI in health care. International consensus-based guidelines have been developed for protocol development (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence) and publication of results (Consolidated Standards of Reporting Trials–Artificial Intelligence).^67,68 These guidelines promote rigorous evaluation and transparency of methods and results, which has improved reporting practices in publications. Ongoing efforts from researchers and funders are needed to ensure continued progress beyond publications into broader clinical development outputs.⁶⁹ Moreover, the scientific community needs to differentiate between use of AI as an intervention and use of AI as a tool in the research toolkit.

Most guidance on AI focuses on its role as an intervention rather than its potential to innovate trials process. Careful evaluation of the impact of generative AI on research processes is crucial due to its significant implications for knowledge generation and population health.⁷⁰ The potential for generative AI to undermine trust in clinical research is high, demanding approaches that ensure equity and that mitigate biases.⁷¹

Looking ahead, model assessment should consider both the risk associated with a model and its use case. Determining an appropriate oversight level will be crucial for regulatory bodies and the clinical research community. Lower-risk generative AI tools, such as those used in clinical trial support or document generation, likely warrant less oversight, while those in higher-risk use cases such as data analysis or clinical trial endpoint adjudication must have much tighter scrutiny.

Fostering integrity through trust and transparency

Trust and trustworthiness are essential for integration and adoption of AI into clinical research. According to the National Institute for Standards and Technology, trustworthy AI must be valid and reliable; safe, secure, and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair, with harmful bias managed.⁷² Building trust requires open and ongoing dialogue about the capabilities and limitations of AI models, regular evaluations, and transparent feedback mechanisms to address changes in population or medical practices.⁷³ Moreover, ongoing monitoring and governance of models are critical to maintain performance and ensure sustained trust from all stakeholders.⁷⁴

One framework proposes a coordinated 3-layered approach that requires evaluating the development process (governance audits), model performance (model audits), and ongoing model output for each specific use case (application audits).⁷⁵ Independent auditing, including postrelease and impact assessment, is crucial to ensure robust performance over time and across diverse subpopulations.^66,76 Given the dynamic nature of clinical data and the evolving capabilities of AI, continuous monitoring, surveillance, and vigilance over AI systems—referred to as algorithmovigilance—will also be essential.^77,78

Establishing robust guardrails and industry-wide standards is crucial for an ethical AI environment in clinical research. Effective data management practices will also be critical to ensure data availability, quality, privacy, security, and for maintaining the rigor and reusability of research data. Although transparency levels may vary during initial design phases to protect intellectual property, critical elements such as training data and model performance must be disclosed upon final deployment to ensure accountability and trust while still respecting intellectual property rights.

As guardrails are established, they must encompass ethical standards and integrate principles such as fairness, accountability, and inclusivity.⁵ Ethical frameworks for evaluating generative AI in clinical research are not yet widely available and should be developed and disseminated.⁷⁹ Sharing knowledge and best practices about the ethical deployment of clinical research studies augmented with AI will be integral, requiring tools to ensure that AI products conform to stringent standards of integrity and ethical use.

Industry-wide ethical standards and robust guardrails would serve to protect human dignity, privacy, and rights. Adhering to these principles will ensure that generative AI reaches its full potential to revolutionize medical discovery while also safeguarding the ethical values fundamental to clinical research. Individual researchers and developers should educate themselves on AI ethics and stay updated with the latest standards. Regulatory bodies and industry groups must enforce compliance through regular audits and guideline updates. The broader community, including academia and clinical practices, should engage in forums and discussions to adapt and refine ethical standards, aligning them with technological and societal changes.

Actionable items

To optimize the integration of AI in clinical research, we suggest an intentional strategy that details the successes and failures of AI at each step of the research process combined with dissemination of findings and collaboration on improvements. We propose a thorough mapping of the clinical research process to illuminate the existing bottlenecks to discern where AI could introduce both minor and major enhancements. A framework for achieving this step was provided by Dilts and Sandler, who meticulously documented the over 700 steps required to open a phase III oncology clinical trial.⁸⁰ They identified a plethora of stages within clinical trials that are ripe for immediate AI application and enhanced efficiency, alongside those that present opportunities for more transformative, systemic changes. For example, the utility of AI in automating the translation of informed consent documents illustrates an area for swift, incremental improvement.²⁴

Beyond identifying opportunities for immediate improvements, process mapping reveals areas where AI could fundamentally redefine existing methodologies—such as leveraging AI for dynamic participant engagement through chatbots or avatars. For example, the AllianceChicago study demonstrates that AI-driven chatbots can significantly improve patient engagement in underserved communities by producing personalized messages.¹⁹ These advances necessitate not only a reimagining of the role of technology but also robust interdisciplinary collaboration to navigate the accompanying ethical, legal, and social challenges. Identifying opportunities where generative AI could be actively deployed—such as in interactive informed consent—would allow for collaboration across stakeholders. In this context, generating and showcasing AI-based solutions in an open-access manner would speed the adoption of generative AI in clinical trials and promote the development of more transformative technologies. Importantly, early demonstration of the benefits and safety of this potential use case for generative AI would build trust among important stakeholders such as trial participants and institutional review boards and demonstrate to sponsors that generative AI can be successfully integrated into the clinical trial workflow.

Successful and rapid adoption of AI technologies in clinical research hinges on the formation of an aligned and collaborative clinical research community.^81,82 Development should occur in tandem with regulatory guidance to ensure consistent evaluation standards and regulatory compliance.⁸ Implementation should mirror the ideals of learning research systems aiming for continuous learning and improvement.⁸³ This inclusive approach also ensures that diverse perspectives are considered in the development and implementation of AI solutions and will facilitate successful clinical trial workflow integration. Such a collaborative environment is essential for maintaining consistency in technological advancements, rapidly progressing toward industry-wide standards, and ultimately creating a more harmonious and efficient clinical research landscape.

Leveraging change management strategies and lessons learned from other technological advances may also help expedite the transition from research to implementation.⁵ For example, drawing lessons from e-commerce, such as prioritizing intuitive user interfaces, and from games, such as ensuring engaging and responsive experience, can further enhance the usability of AI tools. As e-commerce and gaming leverage rapid iteration based on user feedback and strong community engagement, open-access clinical trial-related AI projects might incorporate these strategies to improve and evolve continuously.

We posit that establishing open-access platforms will enhance the effective integration of AI in clinical research.⁸⁴ These platforms are instrumental for sharing training data sets, AI algorithms, and models, thereby democratizing access to advanced tools and fostering a culture of transparency and collaboration across the field. This may include making technologies publicly available for anyone to use, modify, and distribute. Developers can adopt open-source licenses, use platforms like GitHub or Hugging Face for easy access, and provide robust documentation and support to guide users. Encouraging the exchange of knowledge and insights across disciplines is vital for spurring innovation within clinical research practices.⁸⁵

This manuscript conveys key opportunities and barriers to the successful implementation of generative AI in clinical research as discussed by a multidisciplinary group of stakeholders. The proposed framework provides a means to facilitate technological advancements across the clinical research ecosystem. While the think tank attempted to capture the most crucial areas, community and patient involvement will be critical to successful generative AI implementation and future work must integrate patient-centric viewpoints through collaboration with individuals and community advisory boards.^86,87

Conclusions

Generative AI will transform clinical research. Before it becomes entrenched, it is critical to establish regulatory guidelines and guardrails and a culture of continuous monitoring, evaluation, transparency, knowledge sharing, and inclusivity. These efforts will be essential to ensure that generative AI furthers clinical research in a responsible and equitable manner.

Funding support and author disclosures

Funding support for the meeting from which this consensus document was generated was provided through registration fees from the following industry sponsors: Alnylam Pharmaceuticals, Inc (Cambridge, MA, USA); Amgen, Inc (Apex, NC, USA); AstraZeneca (Durham, NC, USA); Bayer Pharma AG (Berlin, Germany); Boehringer Ingelheim Pharma GmbH & Co KG (Ingelheim am Rhein, Germany); Bristol Myers Squibb Company (New York, NY, USA); Intellia Therapeutics, Inc (Cambridge, MA, USA); Janssen Pharmaceuticals (Beerse, Belgium); Companies of Johnson & Johnson (New Brunswick, NJ, USA); Novartis AG (Basel, Switzerland); and Novo Nordisk A/S (Bagsværd, Denmark). No government funds were used for this meeting. The think tank was funded by industry sponsors with representation at the meeting. Dr Foote is supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development under awards T32HD094671 and T32HD104576. Dr Borentain has stock or stock options with Bayer and is a Bayer employee. Dr Dreyer received travel support for attendance at a meeting by Picnic Health and has stock options from Picnic Health. Dr Fessel received payment for a lecture given at the University for Continuing Education, Kremes, Austria, and received travel support from Boehringer Ingelheim Pharma GmbH & Co KG, Department of Global Biostatistics & Data Sciences. Dr Goyal received travel support from Alnylam Pharmaceuticals for attendance at the think tank meeting and receives stock and stock options for Alnylam Pharmaceuticals as an employee. NG is also the co-founder of the patient recruitment platforms iTrials. Hanger works for the Clinical Trials Transformation Initiative, which is supported primarily by an award from the Food and Drug Administration (FDA) of the U.S. Department of Health and Human Services (HHS), with an additional 15% from nongovernmental entities (member organizations); also received travel support from the National Academies of Sciences, Engineering, Medicine Forum on Drug Discovery, Development, and Translation. Dr Hernandez has received grants or contracts from Pfizer Inc and Merck and consulting fees from Merck. Dr Lindsay has stock or stock options with GSK and Johnson & Johnson. Dr Hornik has received grants from the National Institutes of Health (award numbers: 5R01-HD106588-02, 1RL1-HD107784-03, 5R33-HL147833-05, 5T32HD104576-03, OT2OD034481, and 1U01-MD018294-01) and Burroughs Wellcome (1020016) and has received consulting fees from Lightship Inc, Cytokinetics, and UCB Pharma; and also has participated on an advisory board for The Emmes Corporation, LLC. Dr Matheny has received consulting fees from the Patient-Centered Outcomes Research Institute (PCORI) trust fund. Dr Ozer has stock with Novo Nordisk and is a Novo Nordisk employee. Dr Lindsell has received grants or contracts from the following, with all payments made to the institution: the National Institutes of Health, the Department of Defense, the U.S. Centers for Disease Control and Prevention, Novartis, AstraZeneca, Cytokinetics, Biomeme, Entegrion Inc, Endpoint Health, and bioMerieux; In addition, patents for risk stratification in sepsis and septic shock were issued to Cincinnati Children's Hospital Medical Center; has also participated in data safety monitoring boards unrelated to the current work and has participated on Clinical and Translational Science Awards (CTSA) external advisory boards; is also an ex officio member of Association for Clinical and Translational Science (ACTS) board, Clinical Research Forum (CRF) board, ACTS Advisory Committee, and the Center for Clinical and Translational Science (CCTS) Executive Committee; and has stock options with Bioscape Digital. In addition, CJL is the Editor-in-Chief of the Journal of Clinical and Translational Science. CH, JJ, KB, MA, NS, and PJE report no conflicts of interest. The first draft of the manuscript was written independently of sponsor input. Industry representatives were able to provide comments on subsequent drafts of the manuscript. The ultimate decision to include these comments rested with the primary and senior authors.