Advances in the Use of Real‐World Evidence for Medical Devices: An Update From the National Evaluation System for Health Technology

The National Evaluation System for health Technology (NEST), a multistakeholder partnership with a mission to accelerate the development and translation of new and safe health technologies leveraging real‐world evidence (RWE), was established in 2016. Recent advances in the availability of real‐world data (RWD), defined as data generated at the point of care or in the activities of daily life, have increased the potential to generate robust clinical data or real‐world evidence. This article describes NEST's progress.

Since 1976, the US Food and Drug Administration (FDA) Center for Devices and Radiological Health (CDRH) has been responsible for assuring the safety and effectiveness of medical devices in the United States through a risk‐based framework. Medical devices have made significant contributions to the improvement of patient health and include a diverse array of technologies from high‐risk, life‐saving implants, such as cardiac defibrillators, heart valves, and coronary stents, to moderate‐risk devices, such as knee joint replacements.

Characteristics of medical devices have made the implementation of randomized controlled trials challenging. These include iterative and rapid changes in device design, the need to account for the role of operator expertise in clinical outcomes, and challenges in implementing blinding and using placebos. 1

Recent advances in the availability of RWD, defined as data generated at the point of care or in the activities of daily life, have increased the potential to generate robust RWE, using observational or randomized study designs. RWD includes a variety of sources of data, such as electronic health records, claims, billing data, pharmacy data, wearables, and mobile technology. As early as 2012, CDRH identified the growing opportunity to leverage RWD sources in RWE studies to improve premarket review and postmarket surveillance. In fulfilling its 2016–2017 strategic priorities, CDRH increased the use of RWE by 193% between 2015 and 2016, the majority of instances using well‐established registries, such as those in the National Cardiovascular Data Registry.2

The establishment of NEST Coordinating Center3 in 2016 marked a further step in formally coordinating stakeholders to improve evidence generation for medical devices to inform decisions across the total product life cycle, including marketing authorization, postmarket surveillance, payer coverage and reimbursement, clinical practice, and patient choice. Funded through the 2017 Medical Device User Fee Agreement (MDUFA), NEST is an initiative of the Medical Device Innovation Consortium (MDIC) and is advised by a multistakeholder Governing Committee that includes patients, clinicians, regulators, industry, health systems, clinical researchers, and public and private payers. NEST has established formal partnerships with health providers, health payers, and professional registries, that collect, curate, and analyze RWD from electronic health records, claims, pharmacies, and other sources, including registries. The founding NEST network collaborators include over 195 hospitals and 3,942 outpatient clinics across the United States (Table 1). These institutions have made large financial and human resource investments into the collection, curation, and organization of their data to assure it is research grade with financial support from federal, private, and nongovernmental sources, such as the Patient‐Centered Outcomes Research Institute (PCORI).4 These data are organized into several standardized common data models (including domains such as demographics, diagnoses, procedures, and laboratory tests).

Table 1.

NEST founding network collaborators

Network collaborator	Hospitals (n), clinics (n), and patient records (in millions)	US statesa
Duke University Health System	Hospitals: 3 Clinics/outpatient facilities: 80+ Patient records: 42 M	North Carolina
HealthCore/Anthem	Hospitals: > 90% of US hospitals Clinics: > 80% of US clinics Patient records: 70 M	All US states
Lahey Hospital & Medical Center	Hospitals: 6 Clinics: 75 Patient records: 3.3 M	Massachusetts
Mayo Clinic	Hospitals: 23 Clinics: 159 Patient records: 9.2 M	Arizona, Florida, Iowa, Minnesota, and Wisconsin
MDEpiNet	15 Coordinated Registry networks Patient records: 15 M	All US states
Mercy Health	Hospitals: 43 Clinics: 1,099 Patient records: 9 M	Arkansas, Kansas, Missouri, and Oklahoma
Star Clinical Research Network (formerly Mid‐south clinical data research network)	Hospitals: 65 Clinics: 549 +  Patient records: 14 M	Arizona, Florida, North Carolina, South Carolina, and Tennessee
PEDSnet	Hospitals: 9 Clinics: > 150 Patient records: 6.5 M	Alaska, Colorado, Delaware, Illinois, Kentucky, New Jersey, Pennsylvania, Massachusetts, Missouri, New Hampshire, Ohio, Washington, and Wyoming
OneFlorida Clinical Research Consortium	Hospitals: 22 Clinics: 1,200 +  Patient records: 15.1 M	Florida and Texas
Vanderbilt University Medical Center	Hospitals: 5 Clinics: 130 +  Patient records: 2.8 M	Tennessee
New York Clinical Data Research Network	Hospitals: 20 Clinics: 9 Patient records: 300 M	New York
Yale New Haven Health System	Hospitals: 5 Clinics: 130 +  Patient records: 2 M	Connecticut and Rhode Island

NEST, National Evaluation System for health Technology.

^aThis information pertains to the location of the network or organization's facilities and not to the states in which the patients reside. Many hospitals and clinics treat patients from across the United States and from other countries.

In order to demonstrate proof‐of‐concept for the generation of robust RWE, NEST is funding several rounds of test cases. The first set of test cases, selected after a thorough review process from topics proposed by medical device companies, launched in the Fall of 2018 and will seek to explore the feasibility of generating RWE in preparation for regulatory submissions, such as label expansions, or to meet postmarket surveillance requirements. The test cases include a range of medical devices spanning disease areas from cardiology (mechanical heart valve, catheter ablation devices), to vascular (endovascular stents), orthopedic (total knee replacements, lumbar interbody spinal systems, craniomaxillofacial distractors), surgical (microwave ablation devices), and dermatological (wound care technologies) areas. The technologies include both lower‐risk devices designated through the 510(k) pathway and higher‐risk devices requiring a Premarket Approval. Seven of the studies will use retrospective data already collected by the health systems, whereas one will require prospective data collection. The feasibility stage will assess the availability of critical variables, including device exposure, procedure of interest, covariates relating to patient, procedural and provider characteristics, and relevant clinical and safety outcomes that are needed to characterize a cohort of patients exposed to the device. In the absence of widespread adoption of the Unique Device Identifier, brand‐specific devices can be identified by using additional sources of data, such as manufacturer‐owned data, health system inventory management systems, or registries. Studies are planning to report within 12−18 months of launch and if the exploratory stages of the test cases are successful, a full clinical study protocol will be developed with an appropriate analysis plan. A second round of test cases is planned in 2019 and will include topics generated by a broader range of stakeholders, including the FDA, patient foundations, and health systems.

NEST's activities in 2019 will have an additional focus on the use of RWD and RWE for active surveillance. The timely and accurate detection of safety signals for medical devices is a high priority for the FDA.5 Because it is not possible to identify all risks posed by medical products both prior to after they receive marketing authorization by the FDA, timely, effective, and efficient postmarket surveillance is critical. As has been noted by the FDA, postapproval studies for devices remain hard to recruit for and complete. In addition, the current passive surveillance system for identifying safety signals for medical devices is based on an individual identifying that a problem has occurred that may be associated with a device and takes the time to report it. It is generally agreed that the combination of under‐reporting, incomplete reporting, and lack of denominator is not conducive to the robust and timely detection of safety signals.6 The potential for RWD and RWE to accurately detect safety signals has been discussed for several years.7 The access to large datasets owned by health systems collaborating together and coordinated registry networks is opening up new possibilities for active surveillance of medical devices. Several studies published in the last few years have described tools to conduct active surveillance. For example, Resnic et al.8 describe a registry‐based prospective active surveillance safety study of vascular closure devices after percutaneous coronary intervention in the National Cardiovascular Data Registry CathPCI registry, using the Data Extraction and Longitudinal Trend Analysis (DELTA) method.

Through additional funding from the FDA, active surveillance of medical devices leveraging RWD in health systems will be a key area of focus for NEST in 2019. A multistakeholder Task Force convened in early 2019 with representation from patients, clinicians, the FDA, industry, health systems, payers, privacy, and methodology experts. NEST's active surveillance work will focus on demonstrating the feasibility of deploying active surveillance tools within the NEST Data Network, resolving methodological issues, and exploring scalability of active surveillance to include a large number of devices.

Although there are significant opportunities for increasing the quality and quantity of evidence for medical devices using RWD and RWE, concerns about the study validity when using RWD and RWE appropriately focus on two areas: the quality of the source data and the appropriateness of the analysis methods.9 Data quality issues include lack of standards for data collected at the point of care, the availability of data that reflect the comprehensive and longitudinal care of the patient over time, the processes for checking the internal validity of data collected, and the appropriateness of policies to safeguard privacy and security of patient‐identified data. Methodological challenges include issues around the aggregation and analysis of data originating from a variety of disparate sources. To address these challenges, NEST has established expert committees in data quality and methods that include representatives from industry, academia, and government. They will develop a data quality framework that NEST network collaborators will be required to conform to and provide methodological reviews of test case protocols, where they have no conflicts of interest.

NEST is in the early stages of establishing a network of organizations to support increasing numbers of high‐quality RWE studies. The test cases will provide important lessons for the medical device ecosystem on the practical and scientific challenges of conducting RWE medical device studies. Work remains to be done to set the data quality framework and ensure the use of appropriate methodological approaches that will meet FDA evidentiary standards. Although the current set of NEST test cases use observational designs, randomized studies at the point of care that leverages electronic health data to identify, and follow‐up patients are possible and an objective of NEST.10

The establishment of NEST marks an important milestone on the road to delivering on the promises of RWD and RWE, and the learning health system, with the goal of improving patients' timely access to safe, effective, and innovative medical technologies. Although challenges remain, if successful, NEST has the opportunity to impact regulatory, clinical, and coverage decision making in the United States and improve the health and the quality of life of patients.

Funding

Funding for this publication was made possible, in part, by the US Food and Drug Administration through Grant 1 U01 FD 006292‐01.

Conflict of Interest

R.L.F. and J.S. declared no competing interests for this work.

Disclaimer

R.L.F. is an employee of the Medical Device Innovation Consortium (MDIC), which is a 501(c)3 public‐private partnership whose membership is open to industry, government agencies, nonprofit organizations, patient organizations, and professional societies. MDIC is funded through a combination of membership fees and federal and nonfederal grants. Views expressed in written materials or publications do not necessarily reflect the official policies of the Department of Health and Human Services, nor does any mention of trade names, commercial practices, or organization imply endorsement by the US Government.