Section of protocol (reference) | Method |
Timing of outcome assessment (Mathews 2015, p 4) | We planned to classify primary and secondary outcomes using three time periods: short‐term outcomes (assessed immediately after the intervention and up to 12 months after the intervention); medium‐term outcomes (assessed between one and three years after the intervention); and long‐term outcomes (assessed more than three years after the intervention). However, this method was not used because there were no included studies that assessed outcomes beyond three to six months. |
Measures of treatment effect (Mathews 2015, p 7) |
Dichotomous data Where necessary, we planned to report dichotomous data with raw counts and rates for intervention and control groups. We would have summarised study effects using risk ratios and corresponding 95% confidence intervals. However, this method was not used because none of the studies included outcomes with dichotomous data. |
Mean difference For continuous data where the same scale was used to measure similar outcomes, we planned to summarise study effects as mean differences and 95% confidence intervals. However, this method was not used because we found studies used different measures with different scales. | |
Unit of analysis issues
(Mathews 2015, p 7) |
Cluster‐randomised trials We planned to use an estimate of the intracluster correlation coefficient (ICC) from an included study that adequately accounted for a clustered design and reported an ICC. However, this method was not used because clustering was not addressed in the original trials. We planned to conduct sensitivity analyses to assess the adjustments by ICC. However, this method was not used because there were too few included studies. |
Studies with multiple treatment groups In trials with multiple intervention groups and control groups, or both (i.e. multi‐arm studies), we planned to determine which intervention groups were most relevant to the review according to the intervention type and outcomes. Where appropriate, we would have combined all relevant intervention groups into a single intervention group and all control groups into a single control group, to enable a single pairwise comparison (Higgins 2022d, Section 23.3.4). For continuous data, we planned to combine sample sizes, means, and standard deviations using the formula detailed in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2022c, Section 6.5.2.10, Table 6.5a). For dichotomous data, we planned to collate numbers of participants in each of the intervention groups who did and did not experience the outcome. However, these methods were not used because we found no multi‐arm studies. | |
Assessment of reporting biases
(Mathews 2015, p 8) |
Assessing non‐reporting bias and small‐study effects (i.e. publication bias) We planned to assess publication bias in the case of sufficient studies. The Cochrane Handbook for Systematic Reviews of Interventions recommends that tests for funnel plot asymmetry should be used (i) only when there are at least 10 studies included in a meta‐analysis (as fewer studies mean that the test will be underpowered), and (ii) when studies vary in size (as similar‐sized studies will likely have similar standard errors of effect estimates) (Page 2022, Section 13.3.5.4). However, these methods were not used because fewer than 10 studies could be included in our meta‐analyses. |
Data synthesis (Mathews 2015, p 9) |
Programme typology We planned to statistically investigate possible components of effective training interventions in subgroup analyses in an attempt to link specific intervention components to effectiveness. However, we were unable to test these proposals in subgroup analyses because there were too few studies. Instead, we have provided a narrative summary of the characteristics of included studies and present details in the Characteristics of included studies tables. |
Subgroup analyses and investigation of heterogeneity (Mathews 2015, p 9) | Subgroup analyses involve dividing data into subsets for comparison. In this review, we planned to answer questions about intervention types. If there were at least 10 studies (Deeks 2022, Section 10.11.5.1), we would have undertaken the following subgroup analyses to identify if effects were different by subgroup:
We planned to assess differences between subgroups by informally comparing the magnitude of effects via initial inspection of the confidence intervals. If these do not overlap, it may indicate a statistically significant difference in training effects between the subgroups. This must then be followed by a formal statistical approach; for example, examining variability in effect estimates via comparison of I² statistics or examining interaction effects using analysis of variance (ANOVA), as described by Deeks 2022 (Section 10.11.3.1), or both. However, we did not perform these subgroup analyses because there were too few studies reporting these data. |
Sensitivity analysis (Mathews 2015, p 9) | We planned to perform sensitivity analyses to test the robustness of decisions made in the review (Deeks 2022), providing there were sufficient data (i.e. 10 or more studies). We planned to:
However, we did not perform these analyses due to insufficient numbers of included studies. |