Table 1.

Alerting algorithms evaluated in a sequential matched cohort simulation study

Type	Group no.	Specific algorithm	Parameter values
Naïve, fixed nominal Type I error levels	1	Alert when the exact P-value for the cumulative risk ratio < p and the observed cumulative risk ratioa (RR) > θb	p = {0.000001, 0.00001, 0.0001, 0.001, 0.01, 0.05, 0.10, 0.20, 0.30, 0.40}
Group-sequential-Monitoring methods based on cumulative α-spending	2	Alert when the exact P-value, based on cumulative data, < alpha for that monitoring period as defined by the Pocock-like spending function based on cumulative alpha of α and assuming 20 equally-spaced monitoring periods, and the observed cumulative RR > θ	α = {0.000001, 0.00001, 0.0001, 0.001, 0.01, 0.05, 0.10, 0.20, 0.30, 0.40}
	3	Alert when the exact P-value, based on cumulative data, < alpha for that monitoring period as defined by the O’Brien-Fleming-like spending function based on cumulative alpha of α and assuming 20 equally-spaced monitoring periods, and the observed cumulative RR > θ	α = {0.000001, 0.00001, 0.0001, 0.001, 0.01, 0.05, 0.10, 0.20, 0.30, 0.40}
Sequential-probability-ratio test	4	Alert when the test statistic for the maximum sequential-probability-ratio test for binomial data exceeds the critical value based on α and using the appropriate matching ratio and the, observed cumulative RR > θ	α = {0.001, 0.01, 0.05}
Statistical-process-control rulesc	5	Alert when the exact P-value for the period-specific RR < p and the observed cumulative RR > θ	p = {0.000032, 0.000088, 0.00233, 0.00577, 0.001349, 0.00298, 0.00621, 0.012224, 0.02275, 0.040059}
	6	Alert when the exact P-value for X out of Y consecutive period-specific RRs < 0.02275 and the observed cumulative RR > θ	X,Y = {(5,5), (4,5), (4,4), (3,5), (3,4), (3,3), (2,5), (2,4), (2,3), (2,2)}
	7	Alert when the exact P-value for X consecutive period-specific RRs < 0.1587 and the observed cumulative RR > θ	X = {11, 10, 9, 8, 7, 6, 5, 4, 3, 2}
	8	Alert when the exact P-value for X consecutive period-specific RRs < 1.0 and the observed cumulative RR > θ	X = {12, 11, 10, 9, 8, 7, 6, 5, 4, 3}
Disproportionality measures	9	Alert when X consecutive observed cumulative RRs exceed θ	X = {11, 10, 9, 8, 7, 6, 5, 4, 3, 2}
	10	Alert when X consecutive observed period-specific RRs exceed θ	X = {11, 10, 9, 8, 7, 6, 5, 4, 3, 2}

^aThe risk ratio is the observed risk ratio in the simulated data

^bθ is a pre-defined signaling threshold equal to either 1, 1.25, 1.5, or 2, depending on the scenario

^cThe observed risk ratio corresponding to the p-values must have been indicative of harm