Skip to main content
. 2015 Apr 7;2015(4):CD003406. doi: 10.1002/14651858.CD003406.pub4
Analysis Method
Measures of treatment effect Dichotomous data
For binary outcomes, for example, 'problem behaviour present' or 'not present', we will calculate a standard estimation of the odds ratio (OR) with 95% confidence intervals (CI). If ORs are not given or cannot be calculated based on 2 x 2 tables, we will include studies reporting risk ratios (RR), but ORs and RRs will not be combined in a meta‐analysis
Continuous data
When different measures are used to assess the same outcome, we will combine the data using the standardised mean difference (SMD) and calculated 95% CIs. However, we will only include normally distributed data in the meta‐analysis. To assess the distribution of the data, we will calculate the observed mean minus the lowest possible value (or the highest possible value minus the observed mean) divided by the standard deviation; a value less than two is evidence of skew. If the data are skewed, we will contact the authors to request individual participant data or data summaries so that we may transform the data using a log scale. If we are unable to to retrieve this information, we will present a narrative description of the results.
Unit of analysis If clustering is not taken into account, we will perform approximately correct analyses according to the methods described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011), providing we can obtain or estimate the intraclass correlation coefficient (ICC)
Dealing with missing data First, we will try to contact the authors to request any missing data (e.g. missing participants, summary data). Failing that, we will follow the recommendations in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011)
We will manage missing data by performing an intention‐to‐treat analysis. When dichotomous data is missing, we will assume that participants who dropped out had a less favourable outcome (e.g. problem behaviour present) and impute the data accordingly. For continuous missing data, we will use 'the last observation carried forward (LOCF)'.
We will assess the impact of these decisions on the robustness of the results by conducting sensitivity analyses. Thus, for dicohotomous data, we will conduct a sensitivity analysis by imputing data assuming that those missing experienced the more favourable outcome (e.g. problem behaviour not present) (Gamble 2005). For continuous missing data, we will conduct a sensitivity analysis comparing outcomes based on observed data and the LOCF data.
When possible, we will obtain any missing summary data (e.g. missing standard deviations) using the methods described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).
We will describe the methods used to impute the data in the 'Characteristics of included studies' tables. If we are unable to impute the data (e.g. we have insufficient information as regards the numbers missing from each group), we will analyse the available data only and explain in the main text the reasons why we were not able to impute the missing data. When we are unable to impute the results, we will present a narrative description of the results of these studies.
Assessment of heterogeneity We will assess statistical heterogeneity by visual inspection of the forest plots, to examine the extent to which confidence intervals overlap around the estimate for each included study on the forest plots. If confidence intervals have generally limited overlap, this could indicate the presence of statistical heterogeneity.
We will use the Chi² test to formally test for heterogeneity, with a low P value (i.e. less than 0.10) indicating possible heterogeneity (Higgins 2011). We will also use the I² to determine the proportion of variation in point estimates that is due to heterogeneity rather than sampling error or chance (Higgins 2003). We will consider I² values in the range of 50% to 90% to represent substantial statistical heterogeneity.
Due to the potential unreliability of the Chi² test, we will also present the magnitude of the heterogeneity and compare it with the distribution suggested by Turner 2012 to confirm whether there is substantial heterogeneity in the included studies in the meta‐analysis.
We will discuss in full any unexpected variability that may arise.
Subgroup analysis and investigation of heterogeneity We will conduct subgroup analyses for studies that include data on adults with intellectual disabilities and those that include children and for studies of participants with different severity of intellectual disabilities (e.g. mild and moderate versus severe intellectual disabilities)
Data synthesis Given the considerable variability between studies due to the different interventions and populations taking part, we will undertake a meta‐analysis using a random‐effects model as it allows for between‐study variability
Assessment of reporting bias If we have a sufficient number of included studies (10), we will draw funnel plots to identify asymmetry due to publication bias and other small‐study effects. We will apply Egger's regression intercept (Egger 1997) and Begg's rank correlation (Begg 1994) to assess funnel plot asymmetry. However, both tests may be subject to low power to detect bias if there are only a small number of included studies
Sensitivity analysis If a meta‐analysis is performed, we will conduct a sensitivity analysis excluding studies of low quality (e.g. studies where there was a bias in the randomisation process or where there was no concealment of group allocation or other bias)