Abstract
Technologies utilizing cutting-edge methodologies, including artificial intelligence (AI), machine learning (ML) and deep learning (DL), present powerful opportunities to help evaluate, predict, and improve patient outcomes by drawing insights from real-world data (RWD) generated during medical care. They played a role during and following the Coronavirus Disease 2019 (COVID-19) pandemic by helping protect healthcare providers, prioritize care for vulnerable populations, predict disease trends, and find optimal therapies. Potential applications across therapeutic areas include diagnosis, disease management and patient journey mapping. Use of fit-for-purpose datasets for ML models is seeing growth and may potentially help additional enterprises develop AI strategies. However, biopharmaceutical companies often face specific challenges, including multi-setting data, system interoperability, data governance, and patient privacy requirements. There remains a need for evolving regulatory frameworks, operating models, and data governance to enable further developments and additional research. We explore recent literature and examine the hurdles faced by researchers in the biopharmaceutical industry to fully realize the promise of AI/ML/DL for patient-centric purposes.
Keywords: artificial intelligence, biopharmaceutical industry, Coronavirus Disease 2019, data science, deep learning, digital innovation, machine learning, patient-centricity, randomized controlled trials, real-world data
1. Introduction
The biopharmaceutical industry is increasingly realizing the potential values of artificial intelligence (AI), machine learning (ML), and deep learning (DL) to evaluate, predict, and improve patient outcomes by deriving insights from both randomized controlled trial (RCT) data and real-world data (RWD) generated from clinical or medical settings [1].
Earlier on, seminal works in AI/ML offered a historical perspective stemming from Computer Science and Information Science since the 1950s [2,3,4,5], followed by recent works on DL/AI [6,7,8]. Diverse data sources may yield useful insights through treatment pattern analysis, patient journey mapping, and longitudinal follow-ups. While classical statistical methods and tools still play an important role in healthcare analytics and regulatory pathways, data science and digital innovation are increasingly used nowadays to examine the relationships between variables. ML and DL models are increasingly used, especially for medical devices, for their ability to work with very large datasets for predictive accuracy [9,10,11,12].
Recent studies illustrate compelling applications of AI/ML/DL for diagnosis, treatment, disease management, and patient journey mapping in several non-communicable diseases, which are generally chronic diseases. As the Coronavirus Disease 2019 (COVID-19) pandemic took hold in the United States (US), there were signs that these technologies may help in infectious diseases too.
This paper explores a selection of these studies and the hurdles that researchers in industry and academia may need to overcome to fully realize the promise of AI/ML/DL for patients. Several key abbreviations, particularly those defined by the United States’ Food and Drug Administration (FDA) or European Union’s (EU) European Medicines Agency (EMA), are listed (Table 1).
Table 1.
Abbreviation | Terminology | Source | Reference |
---|---|---|---|
AI | Artificial Intelligence | FDA | [13] |
BYOD | Bring Your Own Device | EMA | [14] |
CDS | Clinical Decision Support | FDA | [15] |
CDM | Common Data Model | National Coordinator for Health Information Technology (HealthIT.gov, accessed on 6 October 2022) |
[16] |
DL | Deep Learning | FDA | [17] |
DTC | Decentralized Clinical Trial | FDA | [18] |
DTx | Digital Therapeutics | EU | [19] |
GDPR | General Data Protection Regulation | GDPR.EU | [20] |
HIPAA | The Health Insurance Portability and Accountability Act of 1996 | U.S. Department of Health and Health Services (HHS) | [21] |
ML | Machine Learning | FDA | [17] |
PCT | Pragmatic Clinical Trial | National Institute of Aging | [22] |
PHI | Protected Health Information | HHS.gov | [23] |
R&D | Research and Development | Congressional Budget Office | [24] |
RCT | Randomized Controlled Trial | National Cancer Institute | [25] |
RWD | Real-World Data | FDA | [26] |
RWE | Real-World Evidence | FDA | [26] |
SDOH | Social Determinants of Health | HHS | [27] |
2. Patient-Centricity
Patient-centered care focuses on improving an individual patient’s health outcomes, not on improving a population’s health outcomes, patient-reported outcomes, nor on measuring the performance of a healthcare institution or provider [28,29]. The term “patient-centered outcomes” was included in the US federal legislation US H.R.1865—Further Consolidated Appropriations Act, 2020 [30], and was a focus of the National Academies of Sciences, Engineering, and Medicine in their reports [31].
3. Adoptions
3.1. Disease Diagnoses
Challenges in diagnoses in terms of accuracy and reliability can lead to repeated diagnoses via multiple modalities, poor choices of therapies, and consequently high-cost burdens on the healthcare system for conditions that are difficult to diagnose and lacking in pathognomonic signs and symptoms, as well as overlapping comorbid conditions, and these negative consequences can be amplified.
Radiology, especially medical imaging, is indeed one of the fields in medicine that has had the most successful applications of AI. Over the years, it has become an essential part of medical imaging. In fact, the lead author worked in the early time of applying AI to medical imaging for several years and coauthored multiple articles, including the ones listed below. There are many publications and use-case examples of AI applications in radiology and medical imaging. Since the field is rapidly expanding and evolving, the tools and best practices to minimize biases of AI in radiology or medical imaging have been proposed [32,33,34]. For example, we obtained the numbers of PubMed-listed articles [35] by limiting publications until the end of a full year of 2021, with an understanding that 2022 is not yet a full year: using Boolean operators, and let String A alone; String R alone; Strings A and R (simply denoted as AR), where D = (“Artificial Intelligence” or “Machine Learning” or “Deep Learning”) and R = (“Radiology” or “Medical Imaging). A PubMed search of DR = D and R, yielded 4290 articles since the first article appearing in 1998 until the end of 2021. The trend based on this literature search is displayed (Figure 1).
There are a range of ways in which AI/ML/DL can support more accurate and reliable diagnosis of conditions that can severely impair patients’ quality of life. Since big data are mostly unstructured, natural language processing of texts [36], as well as medical image analysis of CAT scans, magnetic resonance images or ultrasound images [37], can be useful. AI-based diagnostic approaches could complement physicians’ efforts, creating macro efficiencies in the healthcare system and significant quality-of-life benefits for individual patients. In Section 6.2, methodological details on the applications of ML in fibromyalgia are reviewed.
3.2. Treatment Patterns
AI/ML/DL is opening the door to identify effective treatment options and better outcomes by predicting which treatment protocols are likely to succeed based on patient characteristics, comorbid conditions, and treatment rationales. Recent studies show that different approaches to cluster and subgroup analysis can support more effective treatment choices to treat difficult conditions, as illustrated by overactive bladder [38] or erectile dysfunction (ED) [39]. In particular, researchers identified natural clusters of male characteristics per country, quantified ED dynamics in these profiles and compared profiles. Clusters were mainly predicted by unhealthy behaviors, risk factors, and ED, regardless of positive health characteristics and behaviors. Subgroups of men with heightened ED risk factors were identified for precision medicine for optimal targeted therapies [40]. These examples in noncommunicable diseases (NCDs) show a range of possibilities for making more effective treatment decisions and better managing patient treatment over the course of the disease.
3.3. Disease Management
Digital health management has offered long-held hope for extending clinical resources in understanding and managing diseases by virtually connecting patients and healthcare providers through digital technology, such as mobile applications in a bring-your-own-device (BYOD) setting [9,14,41,42]. Data from personal devices can be gathered to support just-in-time adaptive interventions and health behaviors. Such digital tools with usability can help patients receive personalized support and engage with health care providers.
4. Data Volume
Approaches are promising to generate insights from large-scale and high-volume big data, such as those in the form of RWD [26]. There are a set of characteristics needed for trustworthy AI, including “accuracy, explainability and interpretability, privacy, reliability, robustness, safety, and security resilience—and that harmful biases are mitigated or controlled” [43]. However, limited data that do not well represent the populations of interest likely lead to biased models and conclusions since patient diversity might be lacking in historical trials [44,45], which could be due to various social determinants of health (SDOH) [27]. However, it is difficult to achieve without sufficiently large volume of data.
Clinical decision support (CDS) may be adopted early during the clinical evaluation stage [15,46,47]. Increasingly, AI/ML/DL are used to enhance disease understanding and the effectiveness of their therapies. At present, biopharmaceutical companies may face significant barriers in terms of accessing comprehensive and timely patient data due to the siloed nature of systems in terms of interoperability issues. Machine learning tools tend to require large datasets to generate useful results, which would be challenging to the biopharmaceutical companies, as they are mainly focused on RCT data in a much smaller volume or speed. While big data would allow for training, data scientists may apply newer techniques with fewer data points to mine and transfer them [48], despite training on limited labeled information in the data [49,50]. Models for ML can be trained with small datasets using few-shot and n-shot approaches [51,52]. Few-shot learning has the potential to help clean and label datasets, as well as generate more data. This ability to learn with limited labeled data could help re-evaluate unusable data. Few-shot approaches reduce the need to amass a large volume of the right data and to invest in the computer to train a model on those datasets. Zero-shot techniques have the ability to learn from related data or from descriptions of data, rather than designated datasets [52]. These training models generate results derived from limited data may be helpful but may still lack the generalizability and representativeness, which big data would have the advantage of. Thus, biopharmaceutical companies are tailoring their strategies to harness and maximize the values of data, especially in the form of RWD besides RCT data [53,54,55,56,57]. Even with smaller datasets becoming more useful, data sources may undergo standardization, which may be critical for those generated from disparate systems. Common data models (CDMs) may be used to solve the need for a standard format [16].
5. Patient Health Information Protection
Laws and regulations have been established over the privacy of protected health information (PHI) [23]. Data privacy protections become critical [20,21,22,23], and data-sharing practices, e.g., cross-Atlantic collaborations, must carefully regard this privacy protection [58,59,60]. Organizations may consider a risk-based approach that goes beyond simple masking techniques in order to produce a high-quality dataset that meets their specific needs for secondary use. These approaches use ML to determine the likelihood of patient re-identification, thus preserving as many critical data elements as possible to support rich insight while still ensuring compliance.
6. Use-Case Examples
Biopharmaceutical companies have multiple use-case examples found in the public domain that focused primarily in the following areas: drug discovery and development, clinical trials, drug manufacturing, and patient care.
6.1. AI Adoptions
There are several existing use-case examples on the applications of digital endpoints via crowdsourcing from biopharmaceutical study sponsors, which have been collected via crowdsourcing [61]. In addition, the FDA has showcased 90 successful examples of RWE used in medical devices [62,63]. According to the FDA, there were 18 (20%) premarket notification (510[k]) submissions; 14 (15.6%) de novo classification requests; 2 (2.2%) humanitarian device exemptions (HDE) applications; 20 (22.2%) premarket approval (PMA) original applications; 37 (41.1%) PMA panel track supplements. A set of commonly used ML algorithms, including supervised and unsupervised learning methods, has been provided [10].
According to the Deloitte’s 2022 RWE benchmark survey among 17 biopharmaceutical executives, “AI/ML workbench” has been used by 41% of the companies, while 47% plan to develop such a capability [56].
There are multiple examples of applications using AI by a number of pharma companies, focusing primarily in the following areas, including drug discovery and development, clinical trials, drug manufacturing, and patient care [64,65,66,67,68,69,70,71,72,73] (Table 2). The potentials of such innovations through AI/ML/DL can be multifold [74,75,76,77,78].
Table 2.
Example | Organization | Purpose | Project | Reference |
---|---|---|---|---|
1 | AbbVie | Compound Screening | “ChemBeads: Improving Artificial Intelligence Through Human Ingenuty.” | [64] |
2 | Amgen | Drug Discovery and Development | “AI & Data Science: Opening Up Vast New Frontiers in Drug Discovery and Development.” |
[65] |
3 | AstraZeneca | Drug Discovery and Delivery | “Data Science & Artificial Intelligence: Unlocking New Science Insights.” | [66] |
5 | GSK (with Massachusetts Institute of Technology; MIT) | Manufacturing | “GSK Manufacturing Initiative.” | [67] |
6 | Johnson & Johnson | Drug Discovery | “Can Artificial Intelligence Change How We Discover Drugs?” | [68] |
7 | Merck | Drug Discovery and Development | “Merck Announces the Launch of the Merck Digital Sciences Studio to Help Healthcare Startups Quickly Bring their Innovations to Market.” | [69] |
8 | Novartis | Disease Diagnosis | “AI-powered Diagnostic Tool to Aid in the Early Detection of Leprosy.” | [70] |
8 | Pfizer (with CytoReason) | Drug Discovery and Development | “CytoReason Announces Expanded Collaboration Deal with Pfizer to Deliver AI for Drug Discovery and Development.” | [71] |
9 | Roche | Biomarker Evaluation | “Roche Announces the Release of Its Newest Artificial Intelligence (AI) Based Digital Pathology Algorithms to Aid Pathologists in Evaluation of Breast Cancer Markers, Ki-67, ER and PR.” | [72] |
10 | Takeda (with MIT) | Human Health and Drug Development | “MIT-Takeda Program Launches: Research Projects Will Harness the Power of Artificial Intelligence to Positively Impact Human Health.” |
[73] |
The top three purposes for AI in RWD via use cases are to “enable a data-driven understanding of disease progression for populations of interest”, “analyze subpopulations to understand patient behaviors (e.g., switching patterns, adherence)”, and “segment patients based on disease characteristics and health outcomes to match them to trials”. An additional seven benefits are also summarized by Deloitte [56].
6.2. ML for Fibromyalgia and Pain
Magnetic resonance imaging has been used to distinguish the brain scans of individuals with and without fibromyalgia [79]. Characterization of individuals with fibromyalgia was based on brain futures. Hierarchical clustering was used in another study to evaluate chronic pain subgroups [80]. In addition, researchers found that ML could diagnose fibromyalgia with nearly 90% accuracy using a composition of the microbiome [81]. Nearly 20 bacterial species were identified to increase or decrease among patients with fibromyalgia. Furthermore, an ML study involving neural networks indicated the best immune biomarker for diagnosis [82]. Researchers analyzed a measure to assess alexithymia among fibromyalgia patients [83]. Moreover, time-series analysis was conducted for predictive analysis of pain among patients with painful diabetic peripheral neuropathy [84].
7. AI and COVID-19
The COVID-19 pandemic urgently demanded an accelerated pace in diagnostic, prevention, and treatment breakthroughs. However, limited data initially made it challenging for AI/ML/DL predictive algorithms to be developed and deployed. Open databases, such as the COVID-19 Open Research Dataset Challenge (CORD-19) [85], facilitated the use of text analysis to mine the literature, and consequently knowledge of the virus and its mechanisms expanded. A confluence and relationship between patient characteristics and comorbid conditions, such as NCDs, and the burden of this infectious disease helped outcome predictions and disease management [86,87,88].
We obtained several numbers of PubMed-listed articles [32]. By using search terms and limiting publications from 2019 to 2021 inclusive and Boolean operators, we focused on: String A alone; String C alone; Strings AC = A and C, where A = “Artificial Intelligence” with nearly 35,000 articles, and C = (“SARS-CoV-2” or “COVID” or “COVID-19” or “Coronavirus”) with nearly 320,000 articles. In addition, a Venn diagram was used to demonstrate the overlap of AC with over 3000 articles in three years (Figure 2).
Similarly, we expanded the literature search using: String B alone; String C alone; String BC = B and C, where B = (“Artificial Intelligence” or “Machine Learning” or “Deep Learning”) to represent data science with over 64,000 articles, and again C = (“SARS-CoV-2” or “COVID” or “COVID-19” or “Coronavirus”) with 320,000 articles. The overlap of BC also yielded over 3000 articles during the same period (Figure 3).
Due to the impact of the COVID-19 pandemic on global health, there was an explosion of publications, most of them published in pre-print servers to be disseminated in a timely fashion. For example, one of these articles showed the relationship between natural language and viral evolution [36]. Additional pandemic-specific articles cover a wide range of topics from contact tracing, detection, diagnosis, to drug repurposing (e.g., [89,90,91,92]).
8. Conclusions
Biopharma companies have placed a significant commitment in leveraging ML through the use of RWD besides RCTs [53,54,55,56]. The need to address the COVID-19 pandemic over the last several years has shown the need for advances in AI/ML/DL capabilities. There remains a need for agreed regulatory approaches, operating models, and governance, as well as data science talents who understand end-to-end R&D process and health technology assessments in order to enable a much wider spectrum of successful use-case applications.
Developing these capabilities will be a core element in future patient-centric approaches, as one of the top 10 priorities for health economics and outcomes research in 2022 to 2023 [57]. Significant efforts and extensive strategies are needed for biopharmaceutical industries to conduct such activities. As shown in the literature, AI/ML/DL can make a meaningful difference and provide data-driven approaches for stakeholders across the healthcare ecosystem. Such an intersection between data science, AI/ML/DL algorithms, and digital health innovation also presents opportunities for the biopharmaceutical industry, and more broadly, the healthcare industry, to enhance and improve patient care, although with caution on how explainable AI may limit the benefits of black-box ML/DL algorithms [9,93,94,95,96,97,98]. Finally, it is important to emphasize a holistic approach to AI [99], as in the recent AI Bill of Rights in the US [100].
Acknowledgments
The authors thank IQVIA for writing and content development support of the prior newsletter content. Additional editorial support for the current peer-reviewed manuscript was provided by Mamatha K. and Shanthakumar V., Viatris Inc.
Author Contributions
Conceptualization, K.H.Z. and J.Z.L.; methodology, K.H.Z.; software, K.H.Z. and J.Z.L.; validation, K.H.Z. and J.Z.L.; formal analysis, K.H.Z. and J.Z.L.; investigation, K.H.Z. and J.Z.L.; resources, K.H.Z. and J.Z.L.; data curation, K.H.Z. and J.Z.L.; writing—original draft preparation, K.H.Z. and J.Z.L.; writing—review and editing, K.H.Z. and J.Z.L.; visualization, K.H.Z. and J.Z.L.; supervision, K.H.Z. and J.Z.L.; project administration, K.H.Z. and J.Z.L.; funding acquisition, K.H.Z. and J.Z.L. All authors have read and agreed to the published version of the manuscript.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
K.H.Z. and L.J.Z. are shareholders of Pfizer Inc. and Viatris Inc.
Disclaimer
K.H.Z. and L.J.Z. are employees of Viatris Inc., merged between the Upjohn Division of Pfizer Inc. and Mylan Inc. The views expressed are their own and do not necessarily represent those of their employers. This article was based on an early version of a newsletter article, entitled “Enhanced patient-centricity: how the biopharmaceutical industry is optimizing patient care through AI/ML/DL” published in PharmaVoice and received a permission from its editorial office for its adaptation for peer-reviewed journal submission. However, this earlier version is unavailable in any public domain, and therefore K.H.Z. and L.J.Z. have revised critically since that prior newsletter article.
Funding Statement
Upjohn (New York, NY, USA), a Dvision of Pfizer Inc. (New York, NY, USA), merged with Mylan Inc. (Canonsburg, PA, USA) to form Viatris Inc. (Canonsburg, PA, USA). Funding number: not applicable.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Haskett C., Faircloth B., Roper S. Artificial Intelligence in Life Sciences: The Formula for Pharma Success Across the Drug Lifecycle. 2018. [(accessed on 6 October 2022)]. Available online: https://www.lek.com/insights/ei/artificial-intelligence-life-sciences-formula-pharma-success-across-drug-lifecycle.
- 2.Turing A.M.I. Computing machinery and intelligence. Mind LIX. 1950;236:433–460. doi: 10.1093/mind/LIX.236.433. [DOI] [Google Scholar]
- 3.McCarthy J., Minsky M.I., Rochester N., Shannon C.E. A proposal for the Dartmouth Summer Research Project on artificial intelligence. AI Mag. 2006;27:12. [Google Scholar]
- 4.Solomonoff R.J. An Inductive Inference Machine. Preprint from 1957 IRE Convention Record, Section on Information Theory. [(accessed on 6 October 2022)]. Available online: http://www.raysolomonoff.com/publications/An%20Inductive%20Inference%20Machine1957.pdf.
- 5.Solomonoff R.J. A formal theory of inductive inference. Part I. Inf. Control. 1964;7:1–22. doi: 10.1016/S0019-9958(64)90223-2. [DOI] [Google Scholar]
- 6.Goodfellow I., Bengio Y., Courville A. Deep Learning. MIT Press; Cambridge, MA, USA: 2016. [Google Scholar]
- 7.Sejnowski T.J. The Deep Learning Revolution. MIT Press; Cambridge, MA, USA: London, UK: 2018. [Google Scholar]
- 8.Russell S., Norvig P. Artificial Intelligence: A Modern Approach. 4th ed. Pearson; Boston, MA, USA: 2020. [Google Scholar]
- 9.Zou K.H., Salem L.A., Ray A., editors. Real-World Evidence in a Patient-Centric Digital Era. CRC Press; Boca Raton, FL, USA: 2022. [Google Scholar]
- 10.Zou K.H., Li J.Z., Imperato J., Potkar C.N., Sethi N., Edwards J., Ray A. Harnessing real-world data for regulatory use and applying innovative applications. J. Multidiscip. Healthc. 2020;13:671–679. doi: 10.2147/JMDH.S262776. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Zou K.H., Li J.Z., Salem L.A., Imperato J., Edwards J., Ray A. Harnessing real-world evidence to reduce the burden of noncommunicable disease: Health information technology and innovation to generate insights. Health Serv. Outcomes Res. Methodol. 2021;21:8–20. doi: 10.1007/s10742-020-00223-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Alemayehu D., Cappelleri J.C., Emir B., Zou K.H. Statistical Topics in Health Economics and Outcomes Research. CRC Press; Boca Raton, FL, USA: 2018. [Google Scholar]
- 13.U.S. Food & Drug Administration Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML) Based Software as a Medical Device. [(accessed on 6 October 2022)]; Available online: https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf.
- 14.European Medicines Agency Draft Guideline on Computerised Systems and Electronic Data in Clinical Trials. 2021. [(accessed on 6 October 2022)]. Available online: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/draft-guideline-computerised-systems-electronic-data-clinical-trials_en.pdf.
- 15.U.S. Food & Drug Administration Clinical Decision Support Software. [(accessed on 6 October 2022)]; Available online: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software.
- 16.HealthIT Common Data Model Harmonization. The Office of the National Coordinator for Health Information Technology (ONC) [(accessed on 6 October 2022)]; Available online: https://www.healthit.gov/topic/scientific-initiatives/pcor/common-data-model-harmonization-cdm.
- 17.U.S. Food & Drug Administration Artificial Intelligence and Machine Learning in Software as a Medical Device. [(accessed on 6 October 2022)];2021 Available online: https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device.
- 18.U.S. Food & Drug Administration Advancing Oncology Decentralized Trials: Modernizing Evidence Generation. [(accessed on 6 October 2022)]; Available online: https://www.fda.gov/about-fda/oncology-center-excellence/advancing-oncology-decentralized-trials.
- 19.European Union Digital Therapeutics (DTx) [(accessed on 6 October 2022)]. Available online: https://edps.europa.eu/press-publications/publications/techsonar/digital-therapeutics-dtx_en.
- 20.GDPR Complete Guide to GDPR Compliance. [(accessed on 6 October 2022)]. Available online: https://gdpr.eu.
- 21.HHS Health Information Privacy. [(accessed on 6 October 2022)]; Available online: https://www.hhs.gov/hipaa/index.html.
- 22.National Institute on Aging Pragmatic Clinical Trials: Testing Treatments in the Real World. [(accessed on 6 October 2022)];2017 Available online: https://www.nia.nih.gov/research/blog/2017/06/pragmatic-clinical-trials-testing-treatments-real-world.
- 23.HHS What Is PHI? [(accessed on 6 October 2022)]; Available online: https://www.hhs.gov/answers/hipaa/what-is-phi/index.html.
- 24.Congressional Budget Office Research and Development in the Pharmaceutical Industry. [(accessed on 6 October 2022)];2021 Available online: https://www.cbo.gov/publication/57126.
- 25.National Cancer Institute Randomized Clinical Trial. [(accessed on 6 October 2022)]; Available online: https://www.cancer.gov/publications/dictionaries/cancer-terms/def/randomized-clinical-trial.
- 26.U.S. Food & Drug Administration Real-World Evidence. [(accessed on 6 October 2022)]; Available online: https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence.
- 27.U.S. Department of Health and Human Services Social Determinants of Health. [(accessed on 6 October 2022)]; Available online: https://health.gov/healthypeople/priority-areas/social-determinants-health.
- 28.Yeoman G., Furlong P., Seres M., Binder H., Chung H., Garzya V., Jones R.R. Defining patient centricity with patients for patients and caregivers: A collaborative endeavour. BMJ Innov. 2017;3:76–83. doi: 10.1136/bmjinnov-2016-000157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Cappelleri J.C., Zou K.H., Bushmakin A.G., Alvir J.M.J., Alemayehu D., Symonds T. Patient-Reported Outcomes. CRC Press; Boca Raton, FL, USA: 2014. [Google Scholar]
- 30.Congress. US H.R.1865—Further Consolidated Appropriations Act, 2020. [(accessed on 6 October 2022)];2020 Available online: https://www.congress.gov/bill/116th-congress/house-bill/1865/text.
- 31.National Academies Building Data Capacity for Patient-Centered Outcomes Research: An Agenda for 2021 to 2030. 2021–2022. [(accessed on 6 October 2022)]. Available online: https://www.nationalacademies.org/our-work/building-data-capacity-for-patient-centered-outcomes-research-an-agenda-for-2021-to-2030.
- 32.Mongan J., Moy L., Kahn C.E., Jr. Checklist for artificial intelligence in medical imaging (CLAIM): A guide for authors and reviewers. Radiol. Artif. Intell. 2020;25:e200029. doi: 10.1148/ryai.2020200029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Radiological Society of North America Special Report Lays Out Best Practices for Handling Bias in Radiology AI: Regulatory Challenges, Translational Gaps Hinder Machine Learning Implementation. 2022. [(accessed on 6 October 2022)]. Available online: https://www.rsna.org/news/2022/august/Handling-AI-Bias.
- 34.Rouzrokh P., Khosravi B., Faghani S., Moassefi M., Vera Garcia D.V., Singh Y., Zhang K., Conte G.M., Erickson B.J. Mitigating Bias in Radiology Machine Learning: 1. Data Handling. Radiol. Artif. Intell. 2022;4:e210290. doi: 10.1148/ryai.210290. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.National Library of Medicine National Center for Biotechnology Information. [(accessed on 6 October 2022)];2022 Available online: https://pubmed.ncbi.nlm.nih.gov.
- 36.Hie B., Zhong E.D., Berger B., Bryson B. Learning the language of viral evolution and escape. Science. 2021;371:284–288. doi: 10.1126/science.abd7331. [DOI] [PubMed] [Google Scholar]
- 37.Laino M.E., Ammirabile A., Lofino L., Mannelli L., Fiz F., Francone M., Chiti A., Saba L., Orlandi M.A., Savevski V. Artificial intelligence applied to pancreatic imaging: A narrative review. Healthcare. 2022;10:1511. doi: 10.3390/healthcare10081511. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Rahman S.N., Monaghan T.F., Weiss J.P. Development and validation of a machine learning algorithm for predicting response to anticholinergic medications for overactive bladder syndrome. Obstet. Gynecol. 2020;135:483. doi: 10.1097/AOG.0000000000003687. [DOI] [PubMed] [Google Scholar]
- 39.Goldstein I., Goren A., Liebert R., Tang W.Y., Hassan T.A. National Health and Wellness Survey exploratory cluster analysis of males 40–70 years old focused on erectile dysfunction and associated risk factors across the USA, Italy, Brazil and China. Int. J. Clin. Pract. 2019;73:1–15. doi: 10.1111/ijcp.13376. [DOI] [PubMed] [Google Scholar]
- 40.Panch T. Using AI to Amplify Care for Patients with Chronic Disease. AJMC. 2019. [(accessed on 6 October 2022)]. Available online: https://www.ajmc.com/contributor/trishan-panch/2019/01/using-ai-to-amplify-care-for-patients-with-chronic-disease.
- 41.Demanuele C., Lokker C., Jhaveri K., Georgiev P., Sezgin E., Geoghegan C., Zou K.H., Izmailova E., McCarthy M. Considerations for conducting Bring Your Own “Device” (BYOD) Clinical Studies. Digit. Biomark. 2022;6:47–60. doi: 10.1159/000525080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Alzahrani A.S., Gay V., Alturki R., AlGhamdi M.J. Towards understanding the usability attributes of AI-enabled eHealth mobile applications. J. Healthc. Eng. 2021;2021:5313027. doi: 10.1155/2021/5313027. [DOI] [PMC free article] [PubMed] [Google Scholar] [Retracted]
- 43.Schwartz R., Vassilev A., AGreene K.K., Perine L., Burt A., Hall P. Towards a Standard for Identifying and Managing Bias in Artificial Intelligence. National Institute of Standards and Technology; Gaithersburg, MD, USA: 2022. [Google Scholar]
- 44.U.S. Food & Drug Administration Clinical Trial Diversity. [(accessed on 6 October 2022)];2022 Available online: https://www.fda.gov/consumers/minority-health-and-health-equity/clinical-trial-diversity.
- 45.U.S. Food & Drug Administration Diversity Plans to Improve Enrollment of Participants from Underrepresented Racial and Ethnic Populations in Clinical Trials: Guidance for Industry; Draft Guidance for Industry. [(accessed on 6 October 2022)];2022 Available online: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/diversity-plans-improve-enrollment-participants-underrepresented-racial-and-ethnic-populations.
- 46.Vasey B., Nagendran M., Campbell B., Clifton D.A., Collins G.S., Denaxas S., Denniston A.K., Faes L., Geerts B., Ibrahim M., et al. DECIDE-AI expert group. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat. Med. 2022;28:924–933. doi: 10.1038/s41591-022-01772-9. [DOI] [PubMed] [Google Scholar]
- 47.Crossnohere N.L., Elsaid M., Paskett J., Bose-Brill S., Bridges J.F.P. Guidelines for Artificial Intelligence in Medicine: Literature Review and Content Analysis of Frameworks. J. Med. Internet Res. 2022;24:e36823. doi: 10.2196/36823. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Vabalas A., Gowen E., Poliakoff E., Casson A.J. Machine learning algorithm validation with a limited sample size. PLoS ONE. 2019;14:e0224365. doi: 10.1371/journal.pone.0224365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Lawton G. Using Small Data Sets for Machine Learning Models Sees Growth. TechTarget 2019. [(accessed on 6 October 2022)]. Available online: https://searchenterpriseai.techtarget.com/feature/Using-small-data-sets-for-machine-learning-models-sees-growth.
- 50.Feng R., Zheng X., Gao T., Chen J., Wang W., Chen D.Z., Wu J. Interactive few-shot learning: Limited supervision, better medical image segmentation. IEEE Trans. Med. Imaging. 2021;40:2575–2588. doi: 10.1109/TMI.2021.3060551. [DOI] [PubMed] [Google Scholar]
- 51.Naghizadeh A., Metaxas D.N., Liu D. Greedy auto-augmentation for n-shot learning using deep neural networks. Neural Netw. 2021;135:68–77. doi: 10.1016/j.neunet.2020.11.015. [DOI] [PubMed] [Google Scholar]
- 52.Dutta T., Biswas S. Generalized Zero-Shot Cross-Modal Retrieval. IEEE Trans. Image Process. 2019;28:5953–5962. doi: 10.1109/TIP.2019.2923287. [DOI] [PubMed] [Google Scholar]
- 53.Deloitte Getting Real with Real-World Evidence (RWE) 2017 RWE Benchmark Survey. 2017. [(accessed on 6 October 2022)]. Available online: https://www2.deloitte.com/us/en/pages/life-sciences-and-health-care/articles/real-world-evidence-benchmarking-survey.html.
- 54.Davis B., Morgan J., Shah S. The Future of Real-World Evidence Biopharma Companies Focus on End-to-End, AI-Driven, Internally Developed Solutions. 2018. [(accessed on 6 October 2022)]. Available online: https://www2.deloitte.com/us/en/insights/industry/life-sciences/2018-real-world-evidence-benchmarking.html?id=gx:2sm:3tw:4RWE_LSHC18::6Life_Sciences_and_Healthcare:20180711095200:Global&linkId=54041116.
- 55.Morgan J., Feghali K., Shah S., Miranda W. RWE Focus Is Shifting to R&D, Early Investments Begin to Pay Off. 2020. [(accessed on 6 October 2022)]. Available online: https://www2.deloitte.com/us/en/insights/industry/health-care/real-world-evidence-study.html?id=us:2sm:3li:4di_gl:5eng:6di.
- 56.Morgan J., Feghali K., Chang C., Miranda W. Real-World Evidence’s Evolution into a True End-to-End Capability. [(accessed on 6 October 2022)]. Available online: https://www2.deloitte.com/us/en/insights/industry/health-care/real-world-evidence-study.html?id=us:2em:3na:4diUS175115:5awa:6di:MMDDYY:author&pkid=1008576.
- 57.The Professional Society for Health Economics and Outcomes Research. ISPOR 2022–2023 top 10 HEOR Trends Report. [(accessed on 6 October 2022)]. Available online: https://www.ispor.org/heor-resources/about-heor/top-10-heor-trends.
- 58.European Commission. European Health Data Space. [(accessed on 6 October 2022)]. Available online: https://health.ec.europa.eu/ehealth-digital-health-and-care/european-health-data-space_en.
- 59.The White House FACT SHEET: United States and European Commission Announce Trans-Atlantic Data Privacy Framework. [(accessed on 6 October 2022)];2022 Available online: https://www.whitehouse.gov/briefing-room/statements-releases/2022/03/25/fact-sheet-united-states-and-european-commission-announce-trans-atlantic-data-privacy-framework.
- 60.European Data Protection Board Statement 01/2022 on the Announcement of an Agreement in Principle on a New Trans-Atlantic Data Privacy Framework. 2022. [(accessed on 6 October 2022)]. Available online: https://edpb.europa.eu/our-work-tools/our-documents/statements/statement-012022-announcement-agreement-principle-new-trans_en.
- 61.Digital Medicine Society (DiMe) DiMe’s Library of Digital Endpoints. 2022. [(accessed on 6 October 2022)]. Available online: https://www.dimesociety.org/communication-education/library-of-digital-endpoints.
- 62.FDA Voices Leveraging Real World Evidence in Regulatory Submissions of Medical Devices. [(accessed on 6 October 2022)];2021 Available online: https://www.fda.gov/news-events/fda-voices/leveraging-real-world-evidence-regulatory-submissions-medical-devices.
- 63.U.S. Food and Drug Administration Examples of Real-World Evidence (RWE) Used in Medical Device Regulatory Decisions: Selected Examples with File Summaries, Details on Real-World Data Source, Populations, and Descriptions of Use. Center for Devices and Radiological Health. Selected Examples with File Summaries, Details on Real-World Data Source, Populations, and Descriptions of Use. [(accessed on 6 October 2022)];2021 Available online: https://www.fda.gov/media/146258/download.
- 64.AbbVie ChemBeads: Improving Artificial Intelligence through Human Ingenuity. 2019. [(accessed on 6 October 2022)]. Available online: https://stories.abbvie.com/stories/chembeads-improving-artificial-intelligence-through-human-ingenuity.htm.
- 65.Amgen AI & Data Science: Opening Up Vast New Frontiers in Drug Discovery and Development. [(accessed on 6 October 2022)]. Available online: https://www.amgen.com/science/research-and-development-strategy/ai-and-data-science.
- 66.AstraZeneca Data Science & Artificial Intelligence: Unlocking New Science Insights. [(accessed on 6 October 2022)]. Available online: https://www.astrazeneca.com/r-d/data-science-and-ai.html.
- 67.MIT Media Lab GSK Manufacturing Initiative. [(accessed on 6 October 2022)]. Available online: https://www.media.mit.edu/projects/gsk-manufacturing-initiative/overview.
- 68.Johnson & Johnson Can Artificial Intelligence Change How We Discover Drugs? 2018. [(accessed on 6 October 2022)]. Available online: https://www.jnj.com/latest-news/how-artificial-intelligence-is-helping-Janssen-discover-new-drugs.
- 69.Merck Merck Announces the Launch of the Merck Digital Sciences Studio to Help Healthcare Startups Quickly Bring Their Innovations to Market. 2022. [(accessed on 6 October 2022)]. Available online: https://www.merck.com/news/merck-announces-the-launch-of-the-merck-digital-sciences-studio-to-help-healthcare-startups-quickly-bring-their-innovations-to-market.
- 70.Novartis AI-Powered Diagnostic Tool to Aid in the Early Detection of Leprosy. 2020. [(accessed on 6 October 2022)]. Available online: https://www.novartisfoundation.org/news/ai-powered-diagnostic-tool-aid-early-detection-leprosy.
- 71.Pfizer CytoReason Announces Expanded Collaboration Deal with Pfizer to Deliver AI for Drug Discovery and Development. 2020. [(accessed on 6 October 2022)]. Available online: https://www.pfizer.com/news/press-release/press-release-detail/cytoreason-announces-expanded-collaboration-deal-pfizer.
- 72.Roche Roche Announces the Release of its Newest Artificial Intelligence (AI) Based Digital Pathology Algorithms to Aid Pathologists in Evaluation of Breast Cancer Markers, Ki-67, ER and PR. [(accessed on 6 October 2022)]. Available online: https://diagnostics.roche.com/global/en/news-listing/2021/roche-announces-release-of-its-newest-ai-based-digital-pathology-algorithms-to-aid-pathologists-in-evaluation-breast-cancer-markers-ki67-er-pr.html.
- 73.Massachusetts Institute of Technology MIT-Takeda Program Launches; Research Projects Will Harness the Power of Artificial Intelligence to Positively Impact Human Health. [(accessed on 6 October 2022)]. Available online: https://news.mit.edu/2020/mit-takeda-program-launches-research-ai-and-human-health-0618.
- 74.Kaitin K.I. Artificial intelligence and patient-centric approaches to advance pharmaceutical Innovation. Clin. Ther. 2019;41:1406–1407. doi: 10.1016/j.clinthera.2019.06.014. [DOI] [PubMed] [Google Scholar]
- 75.Bhhatarai B., Walters W.P., Hop C.E.C.A., Lanza G., Ekins S. Opportunities and challenges using artificial intelligence in ADME/Tox. Nat. Mater. 2019;18:418–422. doi: 10.1038/s41563-019-0332-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Mak K.K., Pichika M.R. Artificial intelligence in drug development: Present status and future prospects. Drug. Discov. Today. 2019;24:773–780. doi: 10.1016/j.drudis.2018.11.014. [DOI] [PubMed] [Google Scholar]
- 77.Liu Z., Roberts R.A., Lal-Nag M., Chen X., Huang R., Tong W. AI-based language models powering drug discovery and development. Drug. Discov. Today. 2021;26:2593–2607. doi: 10.1016/j.drudis.2021.06.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Paul D., Sanap G., Shenoy S., Kalyane D., Kalia K., Tekade R.K. Artificial intelligence in drug discovery and development. Drug. Discov. Today. 2021;26:80–93. doi: 10.1016/j.drudis.2020.10.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79.Woo C.W., Pujol J., Deus J., Harrison B.J., Monfort J., Wager T.D. Towards a neurophysiological signature for fibromyalgia. Pain. 2017;158:34–47. doi: 10.1097/j.pain.0000000000000707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Alter B.J., Anderson N.P., Gillman A.G., Yin Q., Jeong J.H., Wasan A.D. Hierarchical clustering by patient-reported pain distribution alone identifies distinct chronic pain subgroups differing by pain intensity, quality, and clinical outcomes. PLoS ONE. 2021;16:e0254862. doi: 10.1371/journal.pone.0254862. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Minerbi A., Gonzalez E., Brereton N., Fitzcharles M.A., Chevalier S., Shir Y. Altered serum bile acid profile in fibromyalgia is associated with specific gut microbiome changes and symptom severity. Pain. 2022 doi: 10.1097/j.pain.0000000000002694. [DOI] [PubMed] [Google Scholar]
- 82.Andrés-Rodríguez L., Borràs X., Feliu-Soler A., Pérez-Aranda A., Rozadilla-Sacanell A., Arranz B., Montero-Marin J., García-Campayo J., Angarita-Osorio N., Maes M., et al. Machine learning to understand the immune-inflammatory pathways in fibromyalgia. Int. J. Mol. Sci. 2019;20:4231. doi: 10.3390/ijms20174231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Orrù G., Gemignani A., Ciacchini R., Bazzichi L., Conversano C. Machine learning increases diagnosticity in psychometric evaluation of alexithymia in fibromyalgia. Front. Med. 2020;6:319. doi: 10.3389/fmed.2019.00319. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Alexander J., Jr., Edwards R.A., Brodsky M., Manca L., Grugni R., Savoldelli A., Bonfanti G., Emir B., Whalen E., Watt S., et al. Using time series analysis approaches for improved prediction of pain outcomes in subgroups of patients with painful diabetic peripheral neuropathy. PLoS ONE. 2018;13:e0207120. doi: 10.1371/journal.pone.0212959. Erratum in PLoS ONE 2019, 14, e0212959. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.AI2 Allen Institute for AI CORD-19: COVID-19 Open Research Dataset. 2022. [(accessed on 6 October 2022)]. Available online: https://github.com/allenai/cord19.
- 86.Hassan T.A., Saenz J.E., Li J.Z., Ducinskiene D., Imeprato J., Zou K.H. A Confluence of Acute and Chronic Diseases: Risk Factors Among Covid-19 Patients. Significance 2020. [(accessed on 6 October 2022)]. Available online: https://www.significancemagazine.com/science/671-a-confluence-of-acute-and-chronic-diseases-risk-factors-among-covid-19-patients.
- 87.Zou K.H., Li J.Z., Hassan T.A., Imeprato J., Saenz J.E., Ducinskiene D. The Role of Data Science and Risk Assessments During the COVID-19 Pandemic. CIO Applications. [(accessed on 6 October 2022)]. Available online: https://www.cioapplications.com/cxoinsights/the-roles-of-data-science-and-risk-assessments-during-the-covid19-pandemic-nid-5981.html.
- 88.Hassan T.A., Sáenz J.E., Ducinskiene D., Cook J.P., Imperato J.S., Zou K.H. New Strategies to Improve Patient Adherence to Medications for Noncommunicable Diseases During and After the COVID-19 Era Identified via a Literature Review. J. Multidiscip. Healthc. 2021;14:2453–2465. doi: 10.2147/JMDH.S313626. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89.Ahmad K., Alam F., Qadir J., Qolomany B., Khan I., Khan T., Suleman M., Said N., Hassan S.Z., Gul A., et al. Global user-level perception of COVID-19 contact tracing applications: Data-driven approach using natural language processing. JMIR Form. Res. 2022;6:e36238. doi: 10.2196/36238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 90.Babukarthik R.G., Adiga V.A.K., Sambasivam G., Chandramohan D., Amudhavel J. Prediction of COVID-19 using genetic deep Learning Convolutional Neural Network (GDCNN) IEEE Access. 2020;8:177647–177666. doi: 10.1109/ACCESS.2020.3025164. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 91.Yildirim E., Cicioğlu M., Çalhan A. Real-time internet of medical things framework for early detection of Covid-19. Neural Comput. Appl. 2022:1–14. doi: 10.1007/s00521-022-07582-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 92.Mohanty S., Rashid M.H.A., Mridul M., Mohanty C., Swayamsiddha S. Application of artificial intelligence in COVID-19 drug repurposing. Diabetes Metab. Syndr. 2020;14:1027–1031. doi: 10.1016/j.dsx.2020.06.068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93.Hosny A., Aerts H.J.W.L. Artificial intelligence for global health. Science. 2019;366:955–956. doi: 10.1126/science.aay5189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 94.Davenport T., Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc. J. 2019;6:94–98. doi: 10.7861/futurehosp.6-2-94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 95.Yu K.H., Beam A.L., Kohane I.S. Artificial intelligence in healthcare. Nat. Biomed. Eng. 2018;2:719–731. doi: 10.1038/s41551-018-0305-z. [DOI] [PubMed] [Google Scholar]
- 96.Rajpurkar P., Chen E., Banerjee O., Topol E.J. AI in health and medicine. Nat. Med. 2022;28:31–38. doi: 10.1038/s41591-021-01614-0. [DOI] [PubMed] [Google Scholar]
- 97.Jassar S., Adams S.J., Zarzeczny A., Burbridge B.E. The future of artificial intelligence in medicine: Medical-legal considerations for health leaders. Healthc. Manag. Forum. 2022;35:185–189. doi: 10.1177/08404704221082069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 98.Babicsara B., Evgenouand G., Cohen I.G. Beware explanations from AI in health care: The benefits of explainable artificial intelligence are not what they appear. Science. 2021;373:284–286. doi: 10.1126/science.abg1834. [DOI] [PubMed] [Google Scholar]
- 99.Ng M.Y., Kapur S., Blizinsky K.D., Hernandez-Boussard T. The AI life cycle: A holistic approach to creating ethical AI for health decisions. Nat. Med. 2022 doi: 10.1038/s41591-022-01993-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100.The White House The White House. [(accessed on 6 October 2022)]; Available online: https://www.whitehouse.gov/ostp/ai-bill-of-rights.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable.