Automated surveillance to detect post-procedure safety signals of approved medical devices

Abstract

Context

Assuring the safety of medical devices challenges current surveillance approaches which rely heavily on voluntary reporting of adverse events. Automated surveillance of clinical registries may provide early warnings in the post-market evaluation of medical device safety.

Objective

To determine whether automated safety surveillance of clinical registries using a computerized tool can provide early warnings regarding the safety of new cardiovascular devices.

Design

Prospective propensity matched cohort analysis of seven newly introduced cardiovascular devices, utilizing data from patients undergoing percutaneous coronary intervention (PCI) in Massachusetts between April 2003 and October 2007.

Setting and Patients

All adults undergoing PCI in Massachusetts licensed hospitals utilizing clinical data captured in the Massachusetts implementation of the National Cardiovascular Data Repository CathPCI Registry.

Main Outcome Measure

The presence of any safety alert, triggered if the cumulative observed risk for a given device exceeded the upper 95% confidence interval (CI) of comparator control device. Predefined sensitivity analyses assessed robustness of alerts when triggered.

Results

We evaluated 74,427 consecutive interventional coronary procedures. Three of 21 safety analyses triggered sustained alerts in two implantable devices. Patients receiving Taxus Express2^® drug eluting stents (DES) experienced a 1.28 fold (2.87% versus 2.25%, absolute risk increase of 0.62%, CI: 0.25-0.99%) increased risk of post-procedural myocardial infarction and a 1.21 fold increase major adverse cardiac events relative to alternative DES (4.24% vs. 3.50%, absolute increase of 0.74%, CI: 0.29-1.19%). Patients receiving the Angio-Seal STS^® vascular closure device (VCD) experienced a 1.51 fold increased risk (1.09% vs. 0.72%, absolute increased risk 0.37%, CI: 0.03-0.71%) increased risk of major vascular complications compared with alternative VCD. Sensitivity analyses confirmed increased risk following use of Taxus Express2^® but not for the Angio-Seal STS.^®.

Conclusion

Automated prospective surveillance of clinical registries is feasible, and can identify low frequency safety signals for new medical devices.

Background

Monitoring the safety of approved medical products of vital public health importance, given that such medical products are often used in far greater numbers and in diverse patient populations than studied in pre-market evaluations and clinical trials ^1-6. Within the broad range of medical products, implantable medical devices represent high-risk and uniquely challenging products to monitor because there is little consensus regarding the most appropriate methods to account for the complex interactions among devices, medications, patients and implanting physicians. In addition, the lack of unique medical device identifiers challenges the effective use of administrative claims data and electronic health records as a primary data-source to evaluate manufacturer-specific device safety ⁷.

In recent years, detailed clinical registries have been established at the state, regional and national levels for many high-risk implantable medical devices to support clinical research and quality improvement efforts ^8-11 which may provide unique opportunities to prospectively monitor the safety of implanted medical devices ¹². The Massachusetts statewide coronary intervention registry ¹³ was established in 2003 to monitor the quality of care of hospitals and physicians in the state. This registry is a mandatory clinical outcomes registry based on the American College of Cardiology National Cardiovascular Data Repository (NCDR) CathPCI dataset and includes manufacturer–specific device information for every adult patient undergoing angioplasty procedures in non-federal hospitals since April, 2003¹⁴. This mandatory registry represents a high quality data source for safety surveillance, as it is comprehensively audited and adjudicated for major adverse events and risk factors ^{13, 15}.

Using the Massachusetts statewide coronary intervention registry, we examined in-hospital safety signals for recently introduced interventional cardiovascular devices using an automated safety surveillance system to assess the feasibility of such an approach to prospective medical device safety surveillance.

Methods

Study Setting and Data Sources

The Massachusetts angioplasty registry collects detailed clinical data and inpatient outcome information for all adults (aged 18 or above) who undergo coronary intervention at all non-federal Massachusetts' in-patient facilities. All registry records between April 1, 2003 and September 30, 2007 were included in the analysis. Detailed clinical information obtained during the hospital admission was collected prospectively by trained data managers using variables defined in the NCDR CathPCI Registry dataset¹⁶, and was subject to detailed review and audit procedures both at the hospital level, and at the State level where all major adverse events are reviewed by a panel of trained volunteer physicians and nurse data managers.

Automated Prospective Safety Surveillance System

A computerized automated safety surveillance tool, the Data Extraction and Longitudinal Trend Analysis system, DELTA, was developed and validated on outcomes and clinical trial databases, and shown to efficiently identify very low frequency events utilizing an array of Bayesian and frequentist inference methods^17-19. The system supports multiple simultaneous device-specific analyses, tracking the accumulating experience of multiple devices, while monitoring multiple independent datasets simultaneously ^{12, 20}. Tools within the system allow for joining multiple related datasets and establishing independent prospective analyses using numerous analytic options including propensity matching, risk adjusted cumulative outcomes analysis, sequential methods and survival methods. The system can be configured to trigger alerts at flexible levels of deviation from expected outcomes, and to signal the analyst through e-mail notification when an alert is generated. DELTA was implemented at a central data repository to monitor the accumulating Massachusetts interventional cardiology registry for device specific safety signals, and to trigger safety alerts when specific statistical thresholds were achieved for any monitored device.

Exposures

Four classes of high-risk interventional cardiovascular devices, recently approved and introduced into clinical practice during the study period, were selected for safety monitoring. These included drug-eluting coronary stents, small vessel bare metal coronary stents, vascular closure devices, and embolic protection devices. Potential devices were selected among all high risk devices if they met the sample size required to achieve 80% power to detect a 50% increase in adverse event rates using a Type I error rate of 0.05. For example, assuming average composite adverse event rate of 2.0%, a sample size of 3,826 patient exposures would be required to attain an 80% power to detect a 50% increase in event rate (to 3.0%). Sample size requirements varied from 853 for vascular closure device exposures to 3,536 for drug-eluting stent (DES) exposures. Based on the evolution of the NCDR dataset specification over the study period, manufacturer-specific device information was available for DES throughout the study period. Vascular closure devices, bare metal stents and embolic protection devices had similar detailed information available beginning in January, 2005. The unit of inference was the procedure with subjects receiving multiple studied devices included in each device-outcome analysis.

Outcomes

Each medical device was evaluated for acute adverse outcomes specific to the device group as selected by the investigator team based on clinical relevance to the device class, and incorporating the recommendations of collaborators at FDA's Center for Devices and Radiologic Health. All adverse events and clinical risk factors were defined in accordance with the NCDR CathPCI dataset definitions ¹⁶. For each stent and embolic protection device, adverse events included in-hospital post-procedure myocardial infarction (MI), in-hospital death, and a composite endpoint of major adverse cardiac events (MACE) including emergent revascularization, death and MI. For the vascular closure devices studied, adverse events included in-hospital minor vascular complications (including access site bleeding, hematoma>5cm, pseudoaneursym and AV fistula), major vascular complications (including retroperitoneal hemorrhage, vessel dissection or occlusion or need for urgent vascular procedure) and any vascular complication.

Propensity Score Matching

For each exposure of interest, a propensity score matched concurrent control population was developed based on published risk factors for the outcome of interest, as well as factors which were considered by domain experts to potentially influence the selection of one device versus another in its group (see Appendix A). Propensity scores were developed from a non-parsimonious hierarchical logistic regression analysis developed with the device of interest (exposure) used as the dependent variable, adjusting for baseline covariates of the factors listed in the appendix, as well as between-hospital differences in device utilization. Initial matches were selected from the population of patients exposed to an alternative device within the same group as the exposure of interest (i.e., alternative drug-eluting stents). The cohorts were matched within 6 months of device implantation date and within a fixed propensity score caliper of 0.05 using a greedy matching algorithm²¹. The relative imbalance between the exposed and unexposed groups were assessed using the absolute standardized difference (percentile) in covariates means and proportions, with values greater than 10% considered severely imbalanced²². The propensity matching was considered insufficient to examine overall safety profile of the device if less than 50% of total exposures of a device were successfully matched to control cases (typically due to high utilization of the exposure of interest). In this circumstance, there was poor balance of the clinical features of patients receiving the device of interest and alternative (control) devices. In these situations, the potential control population was expanded through use of less restrictive device exposure parameters (i.e., all drug eluting and bare metal coronary stents).

Surveillance Methods

Adverse event rates were calculated quarterly for the propensity score matched unexposed and exposed cohorts. Safety alerts were triggered if the confidence intervals around the difference between two independent proportions (as measured by the Wilson method) did not cross zero, which indicates a statistically significant difference between the exposed and unexposed groups. The confidence intervals were established by using a 95% confidence interval corrected for multiple comparisons through the use of the O'Brien-Fleming alpha-spending method²³. The chi-square test was used for comparisons of categorical data, and the 2-tailed Student t-test was used to compare continuous variables. All prospective surveillance statistical analyses were performed within the DELTA safety monitoring system (Coping Systems, Andover, MA)^{18, 20, 23}. Population summary statistics were calculated using STATA version 8.0 (Stata Corp LP, College Station, TX). All statistical tests were 2-sided, with a p-value less than 0.05 considered statistically significant for all comparisons.

In order to further investigate a potential safety signal and explore potential sub-populations affected, a series of pre-specified sensitivity analyses were performed through the DELTA system if three or more safety signal alerts were generated for a device-outcome pair during the analysis. These sensitivity analyses included: periodic anlaysis, subpopulation analyses, and alternative risk modeling methods. The periodic analysis used periodic, rather than cumulative, safety signal evaluation to explore consistency of elevated rates and temporal trends in outcomes. To explore whether potential imbalance of specific risk factors between exposed and matched populations might be related to an alert, univariate comparisons of matched and unmatched populations were performed. Finally, relative device safety was assessed using logistic regression based risk adjustment using historical non-exposed patients. The multiple logistic regression model was developed using backward stepwise selection to identify predictors of specified complications and final models incorporated those covariates with consistent associations of p-values≤0.20. The model was developed and calibrated using control cases in the 12 months prior to the study period for the particular device, and applied prospectively to the entire cohort of patients exposed to the device of interest. This method supported inclusion of the entire cohort exposed to the device of interest, rather than only the subset with an adequate match to a concurrent control population.

The study protocol was approved by the hospital's institutional review board, and the FDA Research Involving Human Subjects.

Results

Patient and provider de-identified data for 74,427 consecutive coronary interventional procedures performed from April 1, 2003 to September 30, 2007 in non-federal Massachusetts hospitals were evaluated. Seven devices met the sample size requirements for automated safety monitoring, including two drug-eluting coronary stent systems (Boston Scientific Taxus Express2^® and Cordis Cypher^® stents), one bare metal stent (Guidant/Abbott Mini-Vision), one embolic protection device (Boston Scientific FilterWire^®) and three vascular closure devices (St. Jude Medical Angio-Seal STS^®, Abbott Vascular Proglide^® and Abbott Vascular StarClose^®). Table 1 summarizes the seven devices, along with the 21 safety analyses performed and the matched concurrent control populations chosen for each analysis. The proportion of exposures successfully matched using the propensity matching algorithm ranged from 51% for the Cypher® DES to >99% for the StarClose^® and Perclose Proglide^® vascular closure devices. The FilterWire^® embolic protection device and the Angio-Seal STS^® device required expansion of control patient populations due to very high utilization rates (and therefore limited concurrent controls in same device group) of the devices of interest.

Table 1. Medical devices analyzed, outcomes captured, control populations and final alert status for each analysis.

Device Monitored	Mini-Vision BMS (Guidant/Abbott)	Taxus-Express DES (Boston Scientific) a	Cypher DES (Cordis) b	FilterWire EPD (Boston Scientific) c	Angio-Seal STS VCD (St. Jude Medical) d	StarClose VCD (Abbott)	Perclose Proglide VCD (Abbott)
Surveillance Dates	Jan 2005 - Dec 2007	April 2004 - Dec 2007	April 2004 - Dec 2007	Nov 2005 - Dec 2007	Jan 2005 - Dec 2007	Apr 2006 - Dec 2007	Jan 2005 - Dec 2007
Total Device Exposures	3 253	18 277	28 955	892	10 801	2 291	2 171
Number of Quarterly Analyses	11	15	15	11	11	7	11
Average Quarterly Exposure	295	1 218	1930	81	980	327	197
Standard Deviation of Mean Average Quarterly Exposure	158.9	463.1	544.3	22.2	267.1	41.1	62.7
Comparator Group	Alternative BMS with diameter <2.75mm	Alternative DES (predominately Cypher)	Alternative DES (predominately Taxus)	PCI procedures on SVG without EPD use	Alternative VCD or mechanical hemostasis	Alternative VCD or mechanical hemostasis	Alternative VCD or mechanical hemostasis
Cases Matched	1 959	14 893	14 743	599	8 015	2 277	2 160
Match Proportion	55.6%	81.5%	50.9%	67.1%	74.2%	99.4%	99.5%
Outcomes Monitored	In hospital death, post-procedure MI, MACE	In hospital death, post-procedure MI, MACE	In hospital death, post-procedure MI, MACE	In hospital death, post-procedure MI, MACE	Major, Minor and Any Vascular Complication	Major, Minor and Any Vascular Complication	Major, Minor and Any Vascular Complication
Safety Alerts Triggered	none	Post-procedure MI, MACE	none	none	Major Vascular Complication	none	none
Time to First Safety Alert		15 months (6 764 cases)			12 months (3 287 cases)

Abbreviations: BMS, bare metal stent; DES, Drug Eluting Stent; SVG, saphenous vein graft

^aA small number of alternative DES used as part of clinical trials included in match

^bDevice approved April, 2003; Insufficent comparators until April 2004

^cInsufficient comparator EPD devices available in dataset

^dImproved outcomes over time associated with changes in anticoagulation strategy

Of the 21 safety analyses performed, three (14%) generated a repeated or sustained safety signal involving two implanted devices, prompting detailed sensitivity analysis per study protocol (Table 1). The safety alerts included an increased risk of post-procedural MI as well as an increased risk of major adverse cardiac events (MACE) following implantation of Taxus Express2^® DES. In addition, an increased rate of major vascular complications following implantation of the Angio-Seal STS^® vascular closure device was observed. All other safety analyses resulted in outcomes within (or superior to) the 95% confidence interval established by the propensity matched control population.

Taxus Express2^® drug-eluting stent analysis

Figure 1 illustrates the cumulative safety analysis for the Taxus Express2^® DES. A total of 18,277 patient-procedures involved implantation of one or more Taxus Express2^® DES, of which 14,893 (81.5%) were successfully matched to DES control cases (predominantly the Cordis Cypher^® DES during the study period). Though proportions of use differed significantly among the institutions, both the Taxus Express2^® DES and alternative DES were used in all 22 hospitals included in the analysis. As shown in Figure 1a, by the end of the study period (October 2007) the rate of post-procedural MI was 27.6% higher for Taxus Express2^® DES as compared with alternative DES (2.87% versus 2.25%, absolute risk increase of 0.62%, 95% CI 0.25-0.99%). The surveillance system first alerted in quarter 5 of the analysis, and then demonstrated sustained alerts for increased risk with the Taxus Express2^® DES beginning in July 2005. Similarly, the rates of MACE were increased by 21.1%, driven by the increased post-procedural MI difference, for the Taxus Express2® DES relative to the MACE rate for the propensity matched control population (4.24% vs. 3.50%, absolute increase of 0.74%, CI: 0.29-1.19%) (Figure 1c), and a sustained safety alert for MACE was triggered beginning in July 2007. No increased risk of death was observed among the exposure cohorts (Figure 1b).

Figure 1. Summary safety analysis of the Taxus Express2^® Drug Eluting Stent.

Each panel represents the longitudinal propensity matched analysis of the cumulative incidence of peri-procedural myocardial infarction (panel A), in-hospital death (panel B) and major adverse cardiac event (a combination of death, myocardial infarction or urgent revascularization – panel C) following implantation of at least one Taxus Express2^® drug eluting stent. Circles indicate the cumulative observed event rates for patients receiving Taxus Express2^® DES with circle size proportional to number of Taxus Express2^® stents used in the State during the calendar quarter. The light blue area represents the 95% confidence interval for the propensity matched control population (receiving alternative DES) after correction using the O'Brien-Fleming method for multiple comparisons. The solid blue line within the confidence interval represents the mean event rate of the comparator group. The original uncorrected an upper and lower 95% confidence interval is represented by the dashed black line. The left vertical axis indicates the cumulative adverse event rate. Green circles indicate cumulative event rates within propensity matched expectations, while red circles indicate higher than expected event rate (safety alerts) indicating that the observed event rate exceeded the upper 95% confidence interval boundary for the propensity matched control group. The purple line indicates the cumulative sample size.

Baseline disparities between the Taxus and non-Taxus patients were virtually eliminated for nearly all covariates after the propensity match was applied, with the standardized difference measure less than 10% for all covariates (see Table 2). Significant differences remained, however, in age, exposure to glycoprotein IIb/IIIA antagonists, mean final stent diameter and maximum lesion length (as a surrogate for stent length – see Table 2), however the findings regarding safety of the stent were unchanged after controlling for these factors in multivariate analysis.

Table 2. Distribution of clinical covariates within cases (Taxus Express2^® treated patients) and controls (other DES treated patients), both unselected population, the propensity matched groups, and the unmatched cases receiving Taxus Express2^®.

	No. (%)
	Total Study Population			After Propensity Match			Unmatched Exposures
	Taxus Express	Alternate DES		Taxus Express	Alternate DES		Taxus Express
Covariate	(N=18 277)	(N=28 327)	Std Diff a	(N=14 893)	(N=14 893)	Std Diff a	(N=3 384)	Std Diff a b
Age, mean (SD),y	64.6 (12.2)	64.8 (12.6)	1.60%	64.6 (12.2)	64.3 (12.3)	2.40%	64.4 (12.2)	1.60%
Female Gender	31.0%	30.1%	2.00%	30.9%	30.1%	1.70%	31.7%	1.70%
History of Diabetes	30.6%	30.2%	0.90%	30.1%	30.3%	0.40%	33.0%	6.20%
History of MI	27.8%	29.3%	3.30%	28.6%	29.6%	1.80%	24.2%	10.40%
Current Smoker	21.6%	20.4%	2.90%	21.4%	20.9%	1.20%	22.9%	3.60%
History of Renal Insufficiency	5.26%	6.25%	4.30%	5.23%	5.46%	1.00%	5.36%	0.60%
History of PAD	13.5%	13.7%	0.60%	13.7%	13.8%	0.30%	13.0%	2.00%
Ejection Fraction <30%	41.9%	43.8%	3.80%	41.2%	41.6%	0.80%	44.6%	6.90%
Emergent Procedure	16.2%	15.1%	3.00%	16.1%	15.5%	1.60%	16.7%	1.60%
MI on presentation	36.3%	35.3%	2.10%	36.4%	36.3%	0.20%	35.9%	1.00%
Left Main Disease >50%	5.91%	6.27%	1.50%	6.20%	6.15%	0.20%	4.63%	6.90%
Vein Graft Lesion	5.08%	6.20%	4.90%	5.28%	5.67%	1.70%	4.20%	5.10%
Glycoprotein IIbIIIa Antagonist	29.3%	34.5%	11.10%	28.4%	27.4%	2.20%	33.3%	10.60%
Final Stent Diameter, mean (SD), mm	3.15 (0.52)	3.22 (0.49)	13.90%	3.16 (0.53)	3.23 (0.54)	0.70%	3.11 (0.55)	9.30%
Max Lesion Length, mean (SD), mm	17.8 (9.9)	17.1 (9.7)	7.10%	17.6 (9.7)	18.4 (10.3)	8.00%	18.2 (10.6)	5.90%

Abbreviations: MI, acute myocardial infarction, PAD, Peripheral arterial disease, DES, drug eluting stent. Max lesion length refers to the longest lesion treated during procedure.

^aStd Diff = Absolute standardized difference (percentile) in covariate means and proportions, before and after matching.

^bIndicates comparison between unmatched cases receiving Taxus Express2^® stent and cases receiving Taxus Express2^® stents included in the propensity matched analysis

Pre-defined sensitivity analyses were automatically performed to explore potential explanations of the positive safety signals. Rolling quarter (period based) analysis of the Taxus Express2^® DES demonstrated consistent post-procedural MI and MACE rates at or above the safety alerting threshold throughout the study, thereby confirming a temporally consistent increased hazard for the use of the Taxus Express2^® stent. Additionally, a multiple logistic regression predictive model for the risk of post-procedure MI based on all non-Taxus DES used in 2003-2004 (the time period immediately preceding the study period) was applied prospectively to the entire cohort of patients receiving Taxus Express2^® DES. We observed no significant difference from the alerting behavior observed in the original propensity match compared to the analysis using all patients receiving the stent.

Angio-Seal STS^® vascular closure device

A total of 8,015 Angio-Seal^® STS vascular closure device cases were successfully matched from a total population of 10,801 patients receiving Angio-Seal STS® (74.2%). Though frequency of use differed, both the Angio-Seal STS^® device and alternative vascular closure strategies were used at all 22 institutions included in the analysis. Those exposed to Angio-Seal STS^® were found to have a consistently higher than expected rate of major vascular complications, though a lower rate of minor complications compared with alternative vascular management strategies using the propensity matched concurrent control method (Figure 2). By the end of the observation period, the matched subset of Angio-Seal STS^® cases experienced a 51.3% increased risk of risk of major vascular complications compared with the concurrent control population (1.09% vs. 0.72%, absolute increased risk 0.37%, CI: 0.03-0.71%). Baseline disparities between the Angio-Seal STS^® and alternative closure treated patients were reduced for most covariates after the propensity match was applied. Some differences remained between the matched cohorts in the proportion of patients presenting with symptoms of congestive heart failure, and low ejection fraction, however the standardized difference demonstrated adequate balance of all covariates after the match (Table 3),.

Figure 2. Summary safety analysis of the Angio-Seal^® STS vascular closure device.

Each panel represents the longitudinal propensity matched analysis of the cumulative incidence of in-hospital major vascular complication (panel A), minor vascular complication (panel B) and any vascular complication (panel C) following implantation of at least one Angio-Seal^® STS vascular closure device (VCD). Circles indicate the cumulative observed event rates for patients receiving the Angio-Seal STS VCD with circle size proportional to number of Angio-Seal^® STS VCD used in the State during the calendar quarter. The light blue area represents the 95% confidence interval for the propensity matched control population (receiving alternative closure methods), after correction using the O'Brien-Fleming method for multiple comparisons. The solid blue line within the confidence interval represents the mean event rate of the comparator group. The original uncorrected an upper and lower 95% confidence interval is represented by the dashed black line. The left vertical axis indicates the cumulative adverse event rate. Green circles indicate cumulative event rates within propensity matched expectations, while red circles indicate higher than expected event rate (safety alerts) indicating that the observed event rate exceeded the upper 95% confidence interval boundary for the propensity matched control group. The purple line indicates the cumulative sample size.

Table 3.

Distribution of clinical covariates within cases (patients receiving Angio-Seal STS^®) and controls (patients receiving other VCD), both unselected population, the propensity matched groups, and the unmatched cases receiving Angio-Seal STS^®.

	No. (%)
	Total Study Population			After Propensity Match			Unmatched Exposures
	Angio-Seal STS	Alternate VCD		Angio-Seal STS	Alternate DES		Angio-Seal STS
Covariate	(N=10 801)	(N=12 365)	Std Diff a	(N=8 015)	(N=8 015)	Std Diff a	(N=2 786)	Std Diff a b
Age, mean (SD),y	63.2 (12.3)	64.2 (12.7)	8.00%	63.4 (12.4)	64.1 (12.6)	5.60%	62.6 (12.4)	6.50%
Female Gender	27.5%	28.8%	2.90%	28.6%	30.0%	3.10%	24.6%	9.10%
Body Mass Index, mean (SD)	29.5 (6.3)	29.4 (6.3)	1.60%	29.5 (6.5)	29.4 (6.1)	1.60%	29.5 (6.2)	0.00%
Hypertension	75.6%	73.9%	3.90%	75.7%	76.3%	1.40%	75.4%	0.70%
History of Diabetes	29.8%	28.1%	3.70%	30.5%	31.2%	1.50%	27.9%	5.70%
History of Renal Insufficiency	4.35%	4.06%	1.40%	4.98%	5.12%	0.60%	2.54%	12.90%
History of PAD	8.3%	10.2%	6.60%	11.3%	11.3%	0.00%	0.01%	50.40%
AMI on presentation	38.6%	39.5%	1.80%	39.2%	39.3%	0.20%	37.0%	4.50%
CHF on presentation	10.6%	8.7%	6.40%	10.3%	12.3%	6.30%	11.4%	3.50%
Ejection Fraction <30%	39.2%	45.0%	11.80%	39.9%	43.5%	7.30%	37.1%	5.70%
Emergent Procedure	18.3%	21.1%	7.00%	18.9%	18.9%	0.00%	16.7%	5.70%
Glycoprotein IIbIIIa Antagonist	42.2%	49.3%	14.30%	43.2%	44.0%	1.60%	39.4%	7.70%
Bivalirudin use	30.5%	29.8%	1.50%	28.5%	28.8%	0.70%	36.1%	16.30%
Right heart catheterization	9.82%	9.40%	1.40%	11.6%	11.2%	1.30%	4.70%	25.40%

Abbreviations: MI, acute myocardial infarction, PAD, Peripheral arterial disease, DES, drug eluting stent. Max lesion length refers to the longest lesion treated during procedure.

^aStd Diff = Absolute standardized difference (percentile) in covariate means and proportions, before and after matching.

^bIndicates comparison between unmatched cases receiving Taxus Express2^® stent and cases receiving Taxus Express2^® stents included in the propensity matched analysis

The pre-specified sensitivity analysis exploring temporal changes in outcomes for the Angio-Seal STS^® demonstrated significant heterogeneity in the outcomes observed; with an early period of increased risk of major vascular complications (March, 2005 through December, 2005) followed by consecutive periods of acceptable risk (January, 2006 through December 2006). A detailed exploration of differences in clinical and demographic covariates was performed and demonstrated a significant increase in the use of the direct thrombin inhibitor, bivalirudin, as well as a reduction in the use of Angio-Seal STS^® in patients with concomitant venous access between the early and later time periods. The propensity matched rates of major vascular complications (and overall vascular complications) for patients treated with bivalirudin demonstrated that Angio-Seal STS^® patients had a 38% reduced rate of major vascular complications compared to those patients treated with alternative vascular management (0.49% vs. 0.79%, absolute reduction of 0.30%, CI: 0.14-0.46%). Finally, a second propensity analysis, including only patients who received alternative implantable vascular closure devices, confirmed the principle findings of these analyses. However, only 48% of patients receiving Angio-Seal STS^® could be matched to this control group due the high overall utilization of Angio-Seal STS^® within the registry population.

An independent multiple logistic regression predictive model for the risk of major vascular complications based on all non-Angio-Seal STS^® VCD used in 2004 (the time period immediately preceding the propensity matched analysis period) was developed and applied prospectively to the entire cohort of patients receiving Angio-Seal STS^® VCD. We observed no significant difference between the alerting behavior using the logistic regression prediction method versus the original propensity match thereby supporting the findings of the primary propensity analysis.

Discussion

This study demonstrates the feasibility of automated safety surveillance of implantable devices when applied to a clinical outcomes registry through the use of computerized adverse event surveillance. The methodologies incorporated into the surveillance system were able to distinguish low frequency medical device safety risks in which were not highlighted in pre-market approval studies. In this study, 14% of the monitored device-outcome pairs triggered a sustained potential safety signal necessitating detailed sensitivity analysis. We found that patients receiving the Taxus Express2^® DES experienced a significantly higher rate of post-procedural MI as compared with patients receiving alternative DES. This finding was sustained after temporal trend analysis and alternative risk modeling approaches, though the impact of smaller stent diameters and shorter total lesion length in Taxus DES recipients could not be fully excluded as confounders of the results observed.

Our findings regarding the increased risk of peri-procedural MI with the Taxus Express2^® stent are supported by trends reported in prospective randomized clinical trials, including the Taxus V trial, in which patients receiving multiple Taxus Express2^® stents experienced significantly increased risk of MI within 30 days of the procedure as compared to patients randomly assigned to receive bare metal stents (8.3% vs.3.3%, p=0.047)²⁴. A potential mechanism for this increased risk has been proposed by Popma and colleagues in a detailed angiographic review of the ENDEAVOR-IV trial, which demonstrated an increased frequency of side-branch compromise, associated with peri-procedural MI, when the Taxus Express2^® stent was used as compared with the Endeavor DES²⁵.

The use of Angio-Seal STS^® was associated with a significantly increased risk of major vascular complications in the early period of experience with the device. However, this risk reversed over time in association with changes in practice in anti-thrombotic therapy. For the Angio-Seal STS^® device, we conclude that case selection, changes in medical therapy, and potential learning curve effects likely explain a significant proportion of the increased risk observed in the early period of use of this device.

There is little comparative data on the safety of specific vascular closure devices, although Angio-Seal STS^® has not previously been shown to have an increased risk of major vascular complications relative to other vascular closure devices in two meta-analyses ^{26, 27} and a large comparative safety study²⁸. A learning curve effect in the use of the Angio-Seal^® device has been described ²⁹, as has been the significant reduction in major vascular complications through the use of bivalirudin during PCI procedures which was observed during the period of safety surveillance in this analysis ^{6, 30, 31}.

There was no evidence for increased risks of the analyzed outcomes in the use of the other cardiovascular devices studied including the Cypher^® DES, FilterWire^® embolic protection device, StarClose^® or Perclose Proglide^® vascular closure devices.

This study demonstrates the feasibility of automated surveillance of clinical device registries, and provides a potential framework for temporally efficient comparative safety analysis over broad populations of “real-world” patients. Prospective computerized monitoring, such as demonstrated here, can support the simultaneous monitoring of many device-outcome pairs, thereby permitting the efficient utilization of valuable human resources to explore specific risks identified through the automated safety screening algorithms and alerts. While there are a limited number of data sources with similar features to the Massachusetts angioplasty registry, such detailed clinical registries are becoming more widespread. In addition, alternative clinical data repositories, such as pooled data from increasingly available electronic health record systems as well as medical condition-specific clinical outcomes registries may prove to be valuable resources for additional exploration of automated safety surveillance approaches. Such automated prospective medical device safety surveillance can aid public health officials who rely on passive surveillance tools which lack “denominator” data (i.e. comprehensive exposure information) and therefore provide accurate comparative assessments of safety risk. In addition, federally mandated post-approval studies are often of limited scope and duration, have limited control populations, and often lack statistical power to detect very low frequency safety signals^32-34. Automated safety surveillance may also complement plans for the recently announced Sentinel Initiative³⁵, an active surveillance program, being implemented by the Food and Drug Administration to utilize existing electronic healthcare information sources in order to efficiently generate, strengthen and/or confirm safety signals for medical products⁷. In this context, registries will achieve optimal utility when linked with longitudinal data sources. This is particularly true for implantable devices where outcomes of interest often extend beyond the hospital stay.

It is important to note that potential signals generated in automated surveillance systems must be interpreted with caution and that system safety alerts are intended to generate hypotheses for more in-depth exploration. All potential signals identified through such methods require further evaluation, including sensitivity analyses and more formal epidemiologic studies (which may include medical record validation of outcomes as appropriate). Also, while simple alert boundaries based on statistically significant increased risk were used in this analysis, alternative alert boundary conditions potentially incorporating the severity of the adverse outcome being studied may help inform the choice of the magnitude of the signal to be identified as well as the threshold for significance that would merit additional exploration.

There are several additional limitations of the present analysis which may impact the generalizability of the results. The Massachusetts angioplasty registry is an audited and adjudicated dataset which provides a high quality data source, but may not be representative of other post-market device clinical registries. In an effort to reduce bias in our estimates, our case matching strategy lead, in some instances, to the exclusion of unique, yet high risk, subsets of exposed patients. In this case, because matching controls could not be identified, we were unable to make any comparative statements about safety in these patient populations. Also, potential for residual confounding (from known and unknown factors, including those influencing patient selection) may remain despite propensity-based adjustment methods as utilized here.

In conclusion, automated safety surveillance of medical devices is feasible using automated monitoring tools applied to detailed clinical registries and can efficiently help identify emerging potential post-market safety risks. Automated medical product surveillance can complement existing public health strategies, providing an additional mechanism to assess the comparative safety of approved medical products and improve the quality of healthcare delivered.