FDA Fosters Innovative Approaches in Research, Resources, and Collaboration

We were pleased to read the December 2020 editorial in this journal [1], “Research, reuse, repeat.” The editorial extolls the benefits of making tools (methods, data, and code) available to others who can replicate the results and build on previously developed work. Here, we highlight three lines of activities in the FDA Center for Devices and Radiological Health (CDRH) that embody the spirit portrayed in the editorial. These activities include research, resources, and collaborations that enable assessment of the safety and efficacy of FDA-regulated products, including products enabled by Artificial Intelligence and Machine Learning (AI/ML).

Firstly, the FDA performs scientific research. Along with other research activities across the FDA, the Office of Science and Engineering Laboratories (OSEL) is the research arm of CDRH, and is focused on approximately 20 program areas covering a wide range of medical device technologies. OSEL scientists are charged with conducting regulatory research to accelerate product development and bring products to patients as rapidly as possible. Results of these efforts are disseminated in multiple ways including peer-reviewed publications and by incorporating research results into regulatory resources.

Secondly, the FDA shares resources. CDRH actively seeks to disseminate information and tools to support the application of regulatory science through publications, guidance documents, public summaries of authorized products, and open tools and datasets. The knowledge base of CDRH is its corpus of guidance documents. Guidance documents provide current thinking on policy and regulatory issues. They include technical and conceptual frameworks, as well as methods to guide the robust development and reproducible assessment of regulated products. For medical imaging software as a medical device (SaMD)[2], two recent guidance documents discuss a regulatory framework specifically for evaluating computer aided detection (CADe) algorithms in radiology: one pertains to information needed for a premarket notification (also known as a 510(k) submission) [3], and the other provides additional information about how to assess the device clinical performance in the hands of the clinicians[4]. Another important FDA information resource is the collection of Medical Device Databases, which includes decision summaries for certain types of authorized devices with information on the methods and results relied upon for the regulatory decision. Some examples for SaMD in radiology are related to computer assisted diagnosis (CADx) software for lesions suspicious for cancer[5], [6], computer assisted detection and diagnosis software (CADe + CADx)[6], [7], and computer aided triage and notification software (CADt)[6], [8]. More recently, FDA rendered its first authorization for an H&E digital pathology slide scanner[6], [9] and an AI-based SaMD that uses that scanner’s images to help pathologists detect prostate cancer[6], [10]. To proactively support emerging technologies, OSEL has recently compiled results from its research activities into a Catalog of Regulatory Science Tools to Help Assess New Medical Devices. This catalog supplements guidance documents and recognized standards by providing a list of newly developed methods, phantoms, and computational models and simulations. For example, to address reader variability in Multi-Reader, Multi-Case imaging studies, iMRMC is a publicly available statistical software package that supports the assessment of clinical performance of imaging devices and related AI/ML. iMRMC-related dataset repositories hold supplementary materials from published articles, allowing others to easily understand the data and code with examples.

Thirdly, the FDA collaborates with stakeholders. The Medical Device Development Tool (MDDT) program is a formal CDRH program that encourages stakeholders to propose and pursue FDA qualification for specific tools, which is a voluntary process. Such tools facilitate device development or evaluation, reduce early risk in product development, and provide a means for collecting the necessary information for a regulatory submission. Examples include patient-reported outcome instruments, physical and computational models and simulations of devices and biology, biomarkers to monitor effectiveness of therapies, and AI/ML validation datasets. For example, the high-throughput truthing (HTT) project has submitted a proposal for a validation dataset for the evaluation of tumor infiltrating lymphocytes in digital histopathology images of breast cancer biopsies[11]. The HTT project has publicly shared pilot study data and MDDT submission materials. In addition to the data, the public repository includes scripts to explore and analyze the data. As the HTT project progresses, anyone can adapt the approach to validate other algorithms and biomarkers, whether or not the HTT project successfully qualifies the data as an MDDT. The HTT project has benefitted from a diverse group of collaborators, including several organizations that provide expertise during project development and feedback on deliverables. Similarly, CDRH builds and leverages external expertise through collaborations. CDRH participates in collaborative communities, believing they can help inform and generate regulatory science solutions of public health importance, by bringing together diverse stakeholders to develop tools, methods, and data. This is of particular importance as AI/ML penetrates many technologies. For example, members of the Pathology Innovation Collaborative Community, including chair J. Lennerz, provided a practical tool to navigate FDA guidance documents related to AI/ML[12, Fig. 1]. Another example of collaborative efforts is the Medical Device Innovation Consortium (MDIC). MDIC convenes three collaborative communities and has initiated a digital health working group to “complement FDA’s efforts to develop an innovative regulatory pathway for software.”

Collectively these activities demonstrate FDA’s commitment to communicate and conduct collaborative regulatory science: the science of developing tools, standards, and approaches to reproducibly assess the safety, efficacy, quality, and performance of all FDA-regulated products. FDA is actively engaging researchers, industry, patients, and other stakeholders to identify the needs of industry and the public, and to tackle emerging challenges in the pre-competitive space. You should feel invited to reach out to the FDA or join them in a collaborative community.