Statistics in pills: meta-analysis of rare events

Abstract

Meta-analysis of rare events requires special considerations regarding which statistical method to use. This is because standard meta-analytical models are not well suited for the task, especially when some of the identified studies have reported zero events in one or more treatment groups.

An overview of methods for rare events meta-analysis

• Risk difference methods have poor statistical performance when event rates are low.

• A way to overcome problems associated with zero events is to employ a ‘continuity’ correction (CC), that is, to add a fixed number to the ${\displaystyle 2x2}$  tables in studies with zero events.

• The inverse-variance method with a 0.5 CC has been used extensively, but it can lead to biased results.

• The inverse-variance method with a ‘treatment-arm’ CC offers improved performance compared with 0.5 CC. It has been criticised, however, because the arbitrary choice of the CC may have an impact on the estimates.

• Peto odds ratio (fixed-effects only) performs well as long as three conditions are met: the probability of an event is low (<1%); treatment groups have approximately equal number of patients within each study; treatment effects are small.

• The Mantel-Haenszel (MH) method can be used to pool odds ratios (ORs), risk ratios or risk differences. A MH meta-analysis of OR/risk ratio does not require a CC, and it is less biased than Peto in case when the three conditions mentioned above do not hold. A random-effects MH is provided in RevMan, but this requires a CC.

• Simple logistic regression performs similarly with the MH method with no CC.

• In a Bayesian meta-analysis of rare events, the choice of prior distributions is very important. ‘Uninformative’ priors may dominate meta-analytical results. Especially the choice of prior distribution for heterogeneity in a random-effects meta-analysis may have a strong impact on model estimates. Using informative prior distributions bypasses this issue. A Bayesian approach using available informative prior distributions for heterogeneity is a good way to account for random-effects in meta-analysis.

• Beta-binomial with correlated responses can include studies with zero events in one or both treatment arms, without CC. It has been shown to perform well in a range of settings, when studies are balanced.

• The use of arcsine difference as an effect measure tackles all problems associated with rare events in meta-analysis. It is, however, very difficult to interpret in clinical terms.

General considerations

• The use of of CC, and especially the usual 0.5, should be avoided (with a possible exception when visualising data).

• Results may be very sensitive to the choice of method used to analyse data. The sparser the data the larger the impact of the choice of method.

• Relative effects (eg, odds/risk ratio) should be presented along with absolute event rates, to put results into context.

• Meta-analysts should avoid using arbitrary thresholds in p values (such as the usual 0.05) to label findings as statistically significant or not, when presenting results.

• Meta-analysts should predefine an analysis plan a priori, in a protocol. This is necessary to avoid selective use of methods.

• Meta-analysts should assess the robustness of results in sensitivity analyses, using a range of alternative methods.

• When different methods lead to results with different clinical implications, results should be interpreted with caution. In such cases, results should be considered as hypothesis generating¹.