Review times and adverse events for cardiovascular devices

Abstract

Shorter regulatory review times for high-risk cardiovascular devices correlate with the likelihood of reports of adverse events.

The regulatory review process of new high-risk medical devices remains controversial. This is in part because of the time and expense involved in rigorous pre-market assessment. In the United States, the Food and Drug Administration (FDA) requires manufacturers to demonstrate reasonable assurance of safety and effectiveness prior to market approval. Regulators then weigh the potential life-enhancing or life-saving benefits of the device against the negative consequences of approving an unsafe or malfunctioning product¹.

Because gathering and reviewing clinical data can take months or even years, the FDA has been criticized for delays in approving ultimately useful high-risk devices — for instance, the Sapien Transcatheter Heart Valve (Fig. 1) was sold in Europe at least four years before it was available to patients in the United States^2,3. Also, device industry leaders⁴ and advocacy groups⁵ have sought to accelerate the FDA review of high-risk devices⁶. However, studies of FDA-approved prescription drugs have linked shortened FDA review periods to increased rates of serious adverse events⁷, and have suggested that setting artificial deadlines for FDA reviews is associated with subsequent safety-related label revisions⁸. For cardiovascular devices, evidence from international studies suggests that the longer regulatory approval process of the FDA yields meaningful patient benefit and safety, and results in the commercialization of more innovative devices⁹.

Figure 1 |.

Schematic of an implanted Sapien transcatheter aortic valve. Image from Edwards Lifesciences.

The competing goals of thorough assessment of product safety and shorter review times (and thus earlier availability of the device to patients) are particularly apparent in the regulatory process for cardiovascular devices, as they are typically life-saving yet represent both the largest fraction of reported adverse events for high-risk devices¹⁰ and the largest share of the FDA’s highest-risk recalls¹¹. To assess whether review time and product novelty among high-risk cardiovascular devices are related to the adverse events reported for these products, we considered all 106 FDA-approved high-risk cardiovascular devices over the decade beginning 1 January 2000 (Table 1), and matched each device to its known adverse-event reports for four years following approval (Table 2). Review time was defined as elapsed time between pre-market-approval submission and FDA approval. We have followed the FDA’s definition of ‘cardiovascular device’: any device used to diagnose and treat heart disease and related health problems. Details of device cohort selection, a full list of products, data-matching methods and statistical analysis can be found in the Supplementary Information.

Table 1 |.

Summary statistics for 106 high-risk cardiovascular devices approved by the FDA within 10 years from 1 January 2000.

Variable	Quantity
Any adverse event reported (%)	45
Any adverse event reported within same product code (%)	89
Any adverse event with death reported (%)	17
Any adverse event with serious injury reported (%)	25
Any adverse event with death or serious injury reported (%)	25
Product eligible for priority review (%)	12
First application in product code (%)	18
Second application in product code (%)	15
Third application in product code (%)	12
Reviewed by an advisory panel (%)	23
Review time (months; mean ± s.d.)	14.63 ± 9.77
Total adverse events resulting in death or serious injury, average (min, max)	18.15 (0, 474)
Conditional on any adverse events*
Total adverse events, average (min, max)	181.69 (4, 2,706)
High-report devices (%)	25

^*For the 48 devices with adverse-event reports.

Table 2 |.

Summary statistics for the 8,721 adverse events, reported within the four years following FDA approval, associated with the same device cohort as in Table 1.

Variable	Total (percentage)
Involving malfunction	1,026 (11.8)
Involving death	243 (2.8)
Involving serious injury	1,681 (19.3)
Involving death or serious injury	1,924 (22.1)
Involving other minor incidents	5,771 (66.2)

We have found a negative and statistically significant relationship between the review times of high-risk cardiovascular devices and the likelihood of observing adverse-event reports (Table 3). This relationship holds when considering reports within the first year or within the first four years following a device’s approval. As per sample averages, an additional month spent obtaining regulatory approval was associated with a 10.7% decrease in the probability of observing any subsequent adverse events (odds ratio (OR) = 0.893; 95% confidence interval (95% CI) = 0.835–0.955; p value = 0.001) and an 8.8% decrease in the probability of observing reports of serious injury or death (OR = 0.912; 95% CI = 0.839–0.992; p value = 0.03).

Table 3 |.

Relationship between adverse-event reports, review time and product novelty for the same device cohort as in Table 1.

	Any adverse event			Any adverse event with death or serious injury			Any high-report device
	OR	95% CI	p value	OR	95% CI	p value	OR	95% CI	p value
Review time (months)	0.893	(0.835, 0.955)	0.001	0.912	(0.839, 0.992)	0.030	0.800	(0.649, 0.985)	0.035
Priority reviewed	2.015	(0.389, 10.448)	0.404	2.879	(0.690, 12.011)	0.150	4.590	(0.837, 25.181)	0.079
First application in product code	0.569	(0.115, 2.828)	0.491	1.273	(0.318, 5.095)	0.730	1.260	(0.146, 10.888)	0.834
Second application in product code	0.919	(0.198, 4.275)	0.914	0.619	(0.141, 2.713)	0.520	0.532	(0.061, 4.645)	0.568
Third application in product code	0.630	(0.095, 4.180)	0.632	0.622	(0.110, 3.532)	0.590	2.810	(0.342, 23.099)	0.336
Reviewed by panel	0.955	(0.240, 3.808)	0.948	2.241	(0.707, 7.101)	0.170	4.826	(0.909, 25.617)	0.065
Application year	0.428	(0.306, 0.599)	<0.001	0.914	(0.743, 1.123)	0.390	0.872	(0.614, 1.238)	0.443

^*CI, confidence interval; OR, odds ratio, calculated on the basis of logistic-regression modelling.

The duration of regulatory review times is also predictive of the relative frequency of adverse-event reports: on average, a 1-month increase was associated with a 20% decrease in the probability of being a ‘high-report device’ (OR = 0.800; 95% CI = 0.649–0.985; p = 0.035), defined as a device with over 160 adverse-event reports in the 4 years following approval (corresponding to the top quartile of adverse events). Statistical controls for advisory-panel review and priority review were not associated with high-report devices at reasonable levels of statistical significance, but showed a weak association (p < 0.10); however, this relationship is expected to reflect statistical selection, as riskier devices are more likely to lead to review by expert panels and to be identified as regulatory priorities. Notably, the results presented are robust to the inclusion and exclusion of these controls, suggesting that the relationships observed are statistically meaningful whether or not these other risk factors, inherent in riskier devices, are considered. We did not find that first-in-class or other early entrants in a product code were associated with a higher probability of adverse-event reporting. All regression results presented in Table 3 were highly similar when reproduced in sensitivity analyses using linear probability and negative binomial models (Supplementary Information). Sensitivity analyses confirmed that the results were similar when the top 5% of devices with the most adverse events or longest review times were excluded.

Therefore, we found that devices that experienced longer periods of regulatory review were less likely to be involved in both overall safety reports and the most severe post-market safety reports, even when controlling for other factors expected to be correlated with product risk. Notably, we did not find that first-in-class products or early entrants were associated with a higher probability of adverse-event reporting. Controlling for review times, new devices and older products did not differ in the probability of having adverse events reported.

The data highlight the trade-offs inherent in initiatives that impact the duration of regulatory review for high-risk medical devices. Faster review could bring to market new, potentially life-saving devices more quickly. However, although our study design does not confirm a causal relationship between review timing and post-approval performance, shorter review times are more frequently associated with safety risks experienced by patients receiving new products. This tension currently underlies debates about medical-device regulation in the European Union, which permits the marketing of devices if they perform ‘as intended’ and are likely to be safe, without consistently requiring levels of clinical testing similar to those in the United States. Many high-risk medical devices are, therefore, available to European patients sooner than to patients in the United States, although such devices may also experience a higher prevalence of safety-related label changes and other issues¹². As a result, physicians, policymakers and technology assessment experts in the European Union have called for a centralized system for the evaluation of high-risk devices, as well as for stronger and more transparent clinical-data requirements that incorporate expert medical advice^13,14.

Notably, an adverse-event report neither necessarily indicates that a product is unsafe, nor does it imply a causal relationship between a device and an adverse event. Manufacturers and user facilities are required to report all serious adverse events, but reports in the database may also reflect deaths or injuries unrelated to product safety. Additionally, the frequency with which a device is used may affect the patterns observed. Our fixed-effects analysis addresses potential bias across different categories of products but cannot rule out differential use probabilities among devices within the same product code. However, we have no reason to believe that ‘reportability’ of adverse events would vary across products of the same type. Moreover, as designed, our analysis shows that longer periods of review within a given product code are associated with a lower probability of adverse-event reports. Since this association was observed within product codes, our findings are not impacted by the fact that certain types of device may be more likely to be given to sicker patients. An important area for further research will be to better understand the true risks of the products for which more adverse events are reported: are such reports more likely to arise for inherently riskier devices, or to trigger reporting itself?

The time that is spent pursuing regulatory approval for a product may reflect a number of actions, including communication between manufacturers and the FDA and requests for additional data. Also, more product-safety data may be collected over the period of regulatory review, which could augment the submitted data. Moreover, a protracted period of regulatory review may also allow surveillance over a longer period of time before an approval decision is reached. Furthermore, device registries may allow for faster reactions to safety concerns after an initial safety incident is reported. Unfortunately, not all contributors to aggregate review time are publicly available, so it remains unclear whether these aspects have an impact on adverse-event reporting. As such, additional transparency into the activities completed during the regulatory approval process for high-risk devices would provide important insights into the components of regulatory oversight that are associated with subsequent adverse events. A more detailed understanding of the nature and frequency of activities occurring over the course of product review (such as additional completed tests, additional data submissions and the active use of data from registries) will be important for understanding the mechanisms underlying the association between aggregate review times and adverse-event reports.

Only about 45% of all new high-risk cardiovascular devices in our study period had adverse events reported in the 4 years immediately following approval. This result was lower than expected, and suggests that adverse events may be systematically under-reported to the FDA. Given the high rate of success of our algorithm in linking reports to products (Supplementary Information), any unobserved adverse events are almost certainly a result of missing reports. Future policies should strive to reduce barriers to the reporting of adverse events, such as the 2014 implementation of electronic submission for mandatory reports (Federal Register, citation 79 FR 8832).

Despite the limitations, a key finding is that observable features of the medical-device review process, such as aggregate time spent under FDA review, can help predict the likelihood that adverse events are subsequently reported. A 2009 study found that the FDA’s Center for Devices and Radiological Health “does not use adverse event reports in a systematic manner to detect and address safety concerns about medical devices”¹⁵. Yet our results suggest that doing so could inform post-market surveillance efforts. For example, the predictors that we found, if verified in other device fields and by other investigators, could be of use to regulators and policymakers in assessing which sorts of devices are optimal candidates for enhanced oversight, such as prospective trials and registry initiation. In the European Union, the lack of transparency to physicians, patients, and even regulators of the market-registration process for new devices, makes it much more difficult to identify products with elevated need for post-approval observation or to use post-approval experiences to enhance the quality of future device reviews.

Paradoxically, in light of regulatory evidence from the European Union, policymakers in the United States continue to advocate for the expedited regulatory review of more devices, which would increasingly shift part of the safety assessment of many new devices towards the post-market period. If pre-market trials become shorter¹⁶ and post-market review becomes an increasingly important component of device surveillance, a system to promote more comprehensive reporting of adverse events, as well as a greater understanding of the risks associated with such products, will be crucial for the FDA’s mandate to ensure product safety.