Issue |
Method |
Data management |
Data extraction
Review authors will independently conduct data extraction using a specially developed data extraction form. Where the essential statistics are not presented or further information is required, study authors will be contacted. Relevant information will be included in the description of studies. |
Data collection |
When more than two treatment arms were included in the same trial, all arms will be described.
The following data will be collected for all trial arms:
1) Descriptive data, including participant demographics (age, gender, ethnicity, familial gang involvement, previous criminal record);
2) Intervention characteristics (including delivery, duration, setting, within‐intervention variability, and programme staff demographics);
3) Other services received; and
4) Outcome measures listed above.
The following data will be collected for all studies:
1) Programme differentiation, such as contact or crossover between groups, modifications of procedures, use of intervention curricula or protocols, and actual frequency and duration of administered and received interventions (Dane 1998, Montgomery 2005, MRC 2000); and
2) Context. |
Methodological quality |
Both reviewers will independently assign each included study to a quality category described in the Cochrane Handbook (Higgins 2005). Study authors will be contacted if further information could resolve initial disagreements about quality categories and if a consensus cannot be reached, the Review Group Coordinator of the CDPLPG will be consulted. Criteria to determine quality categories:
A) indicated adequate concealment of the allocation (for example, by telephone randomisation, or use of consecutively numbered, sealed, opaque envelopes);
(B) indicated uncertainty about whether the allocation was adequately concealed (for example, where the method of concealment is not known);
(C) indicated that the allocation was definitely not adequately concealed (for example, open random number lists or quasi‐randomisation such as alternate days, odd/even date of birth, or hospital number)
In studies classified as 'B' (unclear) and 'C' (inadequate) the pre‐treatment assessment and the allocation of participants will be described in the Description of Studies to identify differences between intervention and control groups that may have existed at baseline.
Existing scales for measuring the quality of controlled trials have not been properly developed, are not well‐validated and are known to give differing (even opposing) ratings of trial quality in systematic reviews (Moher 1995). At present, evidence indicates that, "scales should generally not be used to identify trials of apparent low quality or high quality in a given systematic review. Rather, the relevant methodological aspects should be identified a priori and assessed individually" (Juni 2001).
The following components would have been described in narrative form in the Description of studies:
1) Allocation bias (Was group assignment related to outcomes or the interventions received? Attention would have been given to the possible impact of allocation methods on the magnitude and direction of results);
2) Performance bias (Were there systematic differences in care given to the treatment and control groups other than the intervention in question; could the services provided have been influenced by something other than the interventions being compared?);
3) Detection bias (Were outcomes influenced by anything other than the constructs of interest, including biased assessment or the influence of exposure on detection?);
4) Report bias (Were the outcomes, measures and analyses selected a priori and reported completely? Were participants biased in their recall or response?);
5) Attrition bias (Could deviations from protocol, including missing data and dropout, have influenced the results?) (Delgado 2004, Juni 2001); and
6) Outcome validity (Were the outcome measures objective, validated for the population, reported directly by the user or obtained through official records, etc.?). |
Multiple measures |
When a single study provides multiple measures of the same outcome, we will report all measures. For example, if a study includes two measures of quality of life (either measures completed by the same respondent or measures completed by different respondents), we will report both of them. If measures of an outcome are combined for meta‐analysis, we will conduct multiple meta‐analyses if multiple studies report multiple measures that can be combined in this way. If we conduct meta‐analyses in which only one effect estimate can be used from each study, we will select one measure if it is more valid or reliable than the others. For example, if a single respondent completes both a validated scale assessing multiple domains of quality of life and an unvalidated visual analogue scale, we will select the validated scale. If a study includes several equally valid measures and only one effect estimate can be used for meta‐analysis, we will calculate the average effect for this purpose (e.g. the average SMD or RR weighted by variance). |
Multiple arms |
If two or more eligible intervention groups are compared to an eligible control, thus requiring that the reviewers choose a single intervention group for comparison or inclusion in a meta‐analysis, the most intense service or the service that best follows the goals of personal assistance (e.g., services that give users more control) will be included in the meta‐analysis. If a single eligible intervention group is compared to multiple eligible control groups, 'no‐treatment' controls will be chosen over other groups for comparison and inclusion in meta‐analyses. For studies that do not have no‐treatment condition, the most common intervention in clinical practice will be chosen to maximise the external validity of the results. |
Data synthesis
(Outcome data) |
Meta‐analyses may be conducted to combine comparable outcome measures across studies. All overall effects will be calculated using inverse variance methods. Random‐effects models will be used because studies may include somewhat different treatments or populations. |
Continuous data |
Mean differences, standardised mean differences (SMDs) and 95% CIs will be calculated for comparisons of continuous outcome measures. |
Dichotomous data |
Within studies, relative risks (RRs) and 95% confidence intervals (CIs) will be calculated for comparisons of dichotomous outcome measures. Dichotomous outcome measures may be combined by calculating an overall RR and 95% CI. |
Continuous outcomes |
Continuous outcome measures may be combined when means and standard deviations or complete significance testing statistics are available, unless statistical tests assuming normality would be inappropriate. For example, for scales beginning with a finite number (such as 0), effect estimates will not be combined unless a mean is greater than its standard deviation (otherwise the mean would be very unlikely to be an appropriate measure of the centre of the distribution). If continuous outcomes are measured identically across studies, an overall weighted mean difference (WMD) and 95% CI may be calculated. If the same continuous outcome is measured differently across studies, an overall standardised mean difference (SMD) and 95% CI may be calculated (Higgins 2005). SMDs will be calculated using Hedges g. |
Types of analyses |
Studies in which participants are analysed as members of the groups to which they were originally assigned (intention‐to‐treat analysis), studies that include only those participants who were willing or able to provide data (available‐case analysis), and studies that analyse participants who adhered to the study's design (per‐protocol analysis; Higgins 2005) will be analysed separately. Studies in which the reasons for excluding participants from analyses can not be determined from relevant reports or through contact with the authors will be considered with per‐protocol analyses. |
Homogeneity |
The consistency of results will be assessed using the I‐squared statistic (Higgins 2002; Higgins 2003). If there is evidence of heterogeneity (Q‐statistic p less than or equal to 0.1 coupled with an I2 value of 25% or greater), the authors will consider sources according to pre‐specified subgroup analyses and sensitivity analyses (below) but will not report an overall estimate of effect size. If heterogeneity remains within these subgroups, the review will report the results on a trial‐by‐trial basis, in a narrative summary. |
Subgroup analyses |
Large numbers of subgroups may lead to misleading conclusions and are best kept to a minimum (Counsell 1994; Oxman 1992; Yusuf 1991). If possible, this review will include separate effect estimates for the following subgroups:
1) Organisation of services
2) Place of residence
3) Acquisition of impairment
4) Amount of assistance |
Assessment of bias |
Sensitivity analyses will investigate the influence of lower quality studies (i.e., those rated C and D on allocation concealment) on the results of the review. To investigate the possibility of bias, including publication bias, funnel plots will be drawn (Deeks 2005; Egger 1997; Sterne 2001). In the event of asymmetry, the reviewers will seek input from methodologists, including the Cochrane and Campbell Collaboration Methods Groups, on appropriate analyses. |
Graphs |
When meta‐analyses are performed, data will be entered into RevMan in such a way that the area to the left of the line of no effect indicates a favourable outcome for personal assistance. |