| Study |
Bias |
| Randomisation process |
Deviations from intended interventions |
Missing outcome data |
Measurement of the outcome |
Selection of the reported results |
Overall |
| Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
| Kilbourne 2012 |
Low risk of bias |
The study used a random allocation sequence. The description of the job roles and tasks suggests that allocation was concealed prior to consent. There were no substantial differences between the arms at baseline. |
Low risk of bias |
We assumed that an ethical consent to a psychosocial intervention would require participants to be aware of which are they were assigned to. We also assumed it would require intervention practitioners to be aware of allocation. We did not mark risk of bias based on these factors. No deviation from the intervention was noted. Fidelity score of 79%. The authors do not state if the analysis is on an intent to treat basis, but the reviewers have assumed this to be the case. |
Low risk of bias |
Data were available for nearly all participants randomised: 65/68 randomised (96%). |
Low risk of bias |
SF12 was used to capture quality of life: we consider this to be an appropriate measure. It is unlikely measurement could vary between arms. It was not reported whether researchers were masked to allocation. This is a self‐report measure, and, as mentioned above, it will not have been possible to mask the participants; we did not mark down on the basis of self‐assessors not being masked. |
Some concerns |
This study did not have a pre‐published analysis plan. This was a feasibility study and its authors mention that testing analysis options as one of the aims of the study. We considered this to still increase the risk of bias in the result as it possible the published result was selected from multiple eligible analyses of the data. |
Some concerns |
Results not produced in line with a pre‐specified analysis plan. Therefore there is a risk of published results having been selected by the authors. |
| Kilbourne 2013 |
Low risk of bias |
The study used a random allocation sequence. The description of the job roles and tasks suggests that allocation was concealed prior to consent. There were no substantial differences between the arms at baseline. |
Low risk of bias |
We assumed that an ethical consent to a psychosocial intervention would require participants to be aware of which are they were assigned to. We also assumed it would require intervention practitioners to be aware of allocation. We did not mark risk of bias based on these factors. No deviation from the intervention was noted. Fidelity score of 68%. The authors do not state if the analysis is on an intent to treat basis, but the reviewers have assumed this to be the case. |
Some concerns |
High drop out rates were recorded at months 12 and 24 (71 and 75 of 118 randomised participants, respectively), leading to missing outcome data. Drop outs were similar across both arms, minimising the impact of this missingness. The study did not report if missingness was different for different outcomes. The study did not report drop outs at month 6, and the authors could not provide this information. We did not mark down further for this missing drop out data, as we have conservatively utilised the number of participants from the 12 month time point. |
Low risk of bias |
SF12 was used to capture quality of life: we consider this to be an appropriate measure. It is unlikely measurement could vary between arms. Researchers were not masked to allocation. However, this is a self‐report measure, and we considered the participants to be the primary data collector in this instance. As mentioned above, it will not have been possible to mask the participants, but we did not consider this to increase the risk of bias. |
Low risk of bias |
Results were published inline with a pre‐specified analysis plan. It was unclear to whether these were published before unmasked outcome data was available. We did not consider that results had been selected from multiple eligible analyses of the data. |
Some concerns |
High drop out rates across both arms have led to missing data. |
| van der Voort 2015 |
High risk of bias |
Allocation was not concealed from potential participants, recruiting researchers or intervention practitioners prior to consent. There were substantial differences at baseline across arms, which the study authors suggest was due to the lack of concealment. This study was cluster‐randomised: individual participants were not identified before clusters were allocated, and the clinician responsible for identifying eligible participants was aware of allocation. We consider this to create a high risk of bias. |
Some concerns |
Analysis was on an intent‐to‐treat basis. We assumed that an ethical consent to a psychosocial intervention would require participants to be aware of which are they were assigned to. We also assumed it would require intervention practitioners to be aware of allocation. We did not mark risk of bias based on these factors. The study authors note several deviations from the intended intervention in the intervention arm; some elements of the intervention were taken only taken up by 55% of the intervention arm participants. It is unclear whether these were due to the research context. We considered these factors to increase risk of bias. |
High risk of bias |
Data were not available for all randomised participants, and drop out varied across arms. 26 of 71 participants randomised to the intervention dropped out. 10 participants dropped out of the control arm; inconsistent reporting makes it difficult to ascertain whether 80 or 82 participants were allocated to this arm. A large number of the drop outs are due to 2 clusters withdrawing from the intervention arm. This suggests that missing data may be due to difficulties implementing the intervention and therefore missingness may be related to true value, leading to a high risk of bias. |
Low risk of bias |
WHOQOL‐BREF was used to capture quality of life: we consider this to be an appropriate measure. It is unlikely measurement could vary between arms. Researchers were masked to allocation. However, this is a self‐report measure, and, as mentioned above, it will not have been possible to mask the participants; we did not consider this to increase risk of bias. |
High risk of bias |
There is a pre‐published analysis plan. It is unclear whether this was published before unmasked results were available, however, the lack of masking of the majority of the study team suggests it was not. In the protocol paper it is unclear which outcome was intended to be the primary outcome of the study and some outcomes are missing from the published results (brief symptom inventory and clinical global impression for bipolar disorder). We judged this to mean results may have been selected from multiple eligible analyses, increasing risk of bias. |
High risk of bias |
Allocation not concealed prior to recruitment. Baseline data imbalances. There were deviations from the intended intervention. There were high drop out rates. The published results were inconsistent with pre‐specified analysis plan. |
