Dear Editor,
The results of a meta-analysis can be influenced by factors such as study design, quality and methodologies. Sensitivity analysis is a pivotal step in meta-analysis to assess the robustness of the results to these factors. It provides insights into the robustness and reliability of the findings. Researchers can ensure that specific studies or biases do not unduly influence their conclusions by systematically exploring how different assumptions and methodological choices impact the results. Sensitivity analysis enhances the transparency and credibility of meta-analyses, ultimately contributing to more reliable and generalisable evidence in healthcare research.[1]
The researcher is reassured that the underlying factor(s) had little to no influence or impact on the primary conclusions if, following a sensitivity analysis, the results align with those from the primary analysis and lead to similar conclusions regarding intervention.[2] A sensitivity analysis is deemed necessary based on certain characteristics if the eligibility criteria are in question, like a wide range of age, heterogeneity in the type of intervention, dose or route of administration, variable study design and outcome of the studies. If there is heterogeneity in the data type used for analysis (data from clustered, randomised controlled trials or crossover trials, variable data like continuous or ordinal for measuring the same outcome, variable time-to-event data), a sensitivity analysis should be called for. When there is confusion between the analysis methods like the use of fixed versus random effect model for heterogeneity assessment, use of odds ratio, risk ratio or risk difference in dichotomous outcomes, and use of mean difference, standardised mean difference for continuous outcomes, a sensitivity analysis is mandatory.[3] Studies with a high risk of bias or having more weight also impact the robustness of the analysis. These studies can be excluded during sensitivity analysis, and the forest plot must be reinterpreted. If the results are the same, the results are robust [Figure 1a–d]. However, if the results are grossly different, the results of that analysis need to be interpreted with caution.
Figure 1.
(a) Weight of a study as 64%, which is expected to influence the meta-analysis result. (b) A study with 64% weight was removed (weight now 0%), and the results did not change, suggesting a robust result. (c) Use of fixed effect model for meta-analysis. (d) Using a random effect model for the same data has not changed the results, which suggests robust results. CI = confidence interval, MH = Mantel-Haenszel
In a paper by Thabane et al.,[4] the authors reported that the point prevalent use of sensitivity analyses was about 26.7% (36/135), which is exceedingly low. While sensitivity analysis assesses the robustness of the conclusions made, a meta-regression is a statistical method used in meta-analysis to explain the heterogeneity in study results due to variability in the type of studies or patients included.
Sensitivity analysis is often confused with a subgroup analysis. Subgroup analyses are performed in a meta-analysis to assess the effect of an intervention specific to a subgroup of treated patients. These groups are usually identified by age, gender, race and the American Society of Anesthesiologists’ physical status. In a meta-analysis investigating a regional anaesthesia technique for abdominal surgeries, the subgroup could be surgeries specific like caesarean section or hernioplasty. It is recommended that forest plots be provided to interpret the results of a particular intervention in the subgroups, unlike sensitivity analysis.[5]
Controlling the type I error rate is crucial when examining the impact of an intervention across several subgroups. There is a considerable risk of inflating the total type I error rate when statistical tests are performed on many subgroups to identify a subgroup with a significant therapeutic effect. Usually, the sample size will be small, and the subgroups will sometimes not be appropriately defined. Subgroup analysis could be confirmatory or exploratory. In a confirmatory subgroup analysis, the subgroups are formally defined in the methodology. The preliminary evidence generated in an exploratory subgroup analysis is used for hypothesis testing in future studies.[6]
To conclude, if indicated, it is crucial to plan a sensitivity and subgroup analysis before starting the meta-analysis, and the details need to be reported.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
REFERENCES
- 1.Mathur MB, VanderWeele TJ. Robust metrics and sensitivity analyses for meta-analyses of heterogeneous effects. Epidemiology. 2020;31:356–8. doi: 10.1097/EDE.0000000000001180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Patsopoulos NA, Evangelou E, Ioannidis JP. Sensitivity of between-study heterogeneity in meta-analysis: Proposed metrics and empirical evaluation. Int J Epidemiol. 2008;37:1148–57. doi: 10.1093/ije/dyn065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Available from: https://training.cochrane.org/handbook/current/chapter-10 . [Last accessed on 2024 May 30] [Google Scholar]
- 4.Thabane L, Mbuagbaw L, Zhang S, Samaan Z, Marcucci M, Ye C, et al. A tutorial on sensitivity analyses in clinical trials: The what, why, when and how. BMC Med Res Methodol. 2013;13:92. doi: 10.1186/1471-2288-13-92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Wang X, Piantadosi S, Le-Rademacher J, Mandrekar SJ. Statistical considerations for subgroup analyses. J Thorac Oncol. 2021;16:375–80. doi: 10.1016/j.jtho.2020.12.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Borenstein M, Higgins JP. Meta-analysis and subgroups. Prev Sci. 2013;14:134–43. doi: 10.1007/s11121-013-0377-7. [DOI] [PubMed] [Google Scholar]