Recommendation	Rationale
Choice of a disproportionality statistic for signal detection should be primarily based on ease of implementation, interpretation and optimisation of resources	Several disproportionality statistics are currently used in data mining spontaneous report databases. All these can achieve similar overall performance by choice of appropriate signal detection algorithm. Thus, choice should be based on criteria other than signal detection performance. Factors that might be considered include the computing requirements to run the system, the ease of maintaining and adapting the system and whether the operation of the system can be easily communicated to non-statisticians [35]
Consideration should be given to the choice of signal detection algorithm used with disproportionality statistics because these can have important effects on quantitative signal detection performance	In contrast to the choice of disproportionality statistic, the choice of signal detection algorithm to define a SDR can provide very different levels of quantitative signal detection performance in terms of sensitivity, precision and time to signal. Hence, these criteria must be carefully selected on the basis of empirical evidence [35]
For moderate to large spontaneous report databases, the relative performance of a quantitative signal detection algorithm in one database can be predicted from research in other databases	In the PROTECT study, signal detection algorithms with good signaling properties (in terms of sensitivity and positive predictive value) compared to other signal detection algorithms in one spontaneous report database also had relatively good signaling properties in other spontaneous report databases. The databases were both regulatory and company based and ranged in size from about 500,000 to 5,000,000 reports. Hence, relative performance in moderately large databases can be reliably inferred from evaluations in other settings [35]
Absolute performance of the selected quantitative signal detection algorithm must be validated in the target spontaneous report database	Although the relative performance of signal detection algorithms is similar in different spontaneous report databases, the absolute performance characteristics may vary substantially. Hence, it is advisable to test the chosen disproportionality statistic with a range of signal detection algorithms within the target database [35]
Consideration should be given to the effect of reduced positive predictive value with time on the market	There appears to be a reduction in precision with time and hence it may be more productive to put additional effort into the evaluation of signals from newer products. This finding has been validated excluding ADRs identified prior to authorisation from the reference database but further work is ongoing to characterise this effect [35]
Consideration should be given to carrying out comparisons of quantitative signal detection methods across spontaneous report databases matching at the drug-event combination level rather than averaging over all drug-event combinations	It is possible that some ADRs may be more easily found in some databases. This was not investigated in PROTECT
It would be useful to conduct research to establish empirically the best method for quantitative signal detection in combination products	Combination products and single substances are often treated as unrelated in signal detection systems; a question remains whether combining data from these products will provide more or less accurate detection of signals
Consideration should be given to establishing a framework for selecting the best quantitative signal detection algorithm to suit the organisational goals and resource available within a pharmacovigilance group	Our research has shown a predictable trade-off between sensitivity and precision as far as purely quantitative signal detection algorithms are concerned. However, the means of striking the correct balance between sensitivity and the concomitant burden of false positives for a given organisation requires careful consideration