| Study |
Bias |
| Randomisation process |
Deviations from intended interventions |
Missing outcome data |
Measurement of the outcome |
Selection of the reported results |
Overall |
| Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
Authors' judgement |
Support for judgement |
| Haight 2005 |
Some concerns |
Method used to generate random sequence and to conceal allocation are not described: "Ten mothers were randomly assigned to the treatment group, and ten to the comparison group"
No significant differences reported between groups: "T‐tests and chi‐square analyses revealed that with one exception intervention and comparison groups did not differ significantly on the characteristics described above. Mothers in the intervention group were more likely to have been placed in foster care themselves as children" |
Low risk of bias |
Participants and people delivering the interventions would have likely been aware of their group allocation due to differences in the interventions
No deviation from intended intervention specified, unlikely that the outcome would be impacted by lack of blinding
Assume data available for all participants as participants did the intervention, visit and assessment (clinical interview) all on the same day in succession |
Low risk of bias |
Data available for all participants |
Low risk of bias |
Direct observation rated using an unvalidated coding strategy
Same measurement and assessment time points used for all participants
Outcome assessors were blinded to treatment condition: "...raters were blind as to the group assignment of mothers" |
Some concerns |
No published protocol
Reported results correspond to intended outcome measurements
No analysis plan described |
Some concerns |
Overall some concerns due to no information on the randomization process and insufficient information on the analysis intentions |
| Liu 2021 |
Some concerns |
Method used to generate random sequence and to conceal allocation not described, only reported as "conditional randomization"
No significant differences reported between groups: "Except for biological sex, the control and intervention groups did not show significant differences in other demographics and study variables." Although there was a difference in sample size, "The uneven sample sizes in the active control and intervention groups were a deliberate design decision to include maximum access to active intervention in these vulnerable populations" |
Low risk of bias |
Participants and people delivering the interventions would have likely been aware of their group allocation due to differences in the interventions, including filming, role playing and video coaching
No deviation from intended intervention specified, unlikely that the outcome would be impacted by lack of blinding
Modified ITT excluding participants with missing outcome data |
Some concerns |
34% dropout (from total sample), greater dropout in FIND (38%) than control (26%)
No evidence that the results was not biased by the missing data
Reasons for dropout not reported |
Some concerns |
Validated outcome measure used, however “all measures were reported by caregivers, which might induce reporter biases.”
Same measurement and assessment time points used for all participants, however “during the study procedure, contact with the families was maintained through weekly meetings with FIND coaches (for intervention group families) or weekly check‐in phone calls/texts (for control group families).” P. 4. So there appear to be some differences in the weekly contact (meeting vs check‐in / text). Also, we don’t know if there were different types of assessors (psychologists, GP’s, RA’s).
Measurement tool is self‐report, so participants are assessors and cannot be blinded
The outcome assessment could be potentially influenced by the knowledge of the intervention received. It is unlikely that the reporting was influenced by the treatment received |
Low risk of bias |
No published protocol, however outcomes in methods section align with those in the results
All reported results correspond to intended outcome measurements. Note, while only the subscale that showed significant change was reported in the paper, on request, the authors supplied all subscale data
Seems likely that all reported results correspond to intended analyses, even though the analysis for this outcome is not reported in the analysis plan |
Some concerns |
Overall some concerns due to no information on the randomization process, missing outcome data (large dropout) and potential bias in the measurement of outcome (self‐report) |
| Rosenblum 2017 |
Some concerns |
Method used to generate random sequence is adequate, however unclear how they concealed allocation, as no details provided on who called the participant informing them of their allocation, and how that individual received allocation information: "Women were randomized using a computer‐generated algorithm (urn randomization stratified by baseline trauma exposure and depression to ensure balanced groups to either the multifamily group [treatment condition] or mailing group [control condition])."
No significant differences reported between groups: "Women in both conditions were balanced on demographic, mental health and trauma variables" |
Low risk of bias |
Participants and people delivering the interventions would have likely been aware of their group allocation due to differences in the interventions
No deviation from intended intervention specified, unlikely that the outcome would be impacted by lack of blinding
Modified ITT excluding participants with missing outcome data |
Some concerns |
36‐37% dropout (from total sample)
No evidence that the results was not biased by the missing data, however no analysis method or sensitivity analysis to correct for bias
Reasons for dropout are not reported, however dropout is equal between groups |
Some concerns |
Validated outcome measure used
Same measurement and assessment time points used for all participants
Measurement tool is self‐report, so participants are assessors and cannot be blinded
The outcome assessment could be potentially influenced by the knowledge of the intervention received. It is unlikely that the reporting was influenced by the treatment received |
Some concerns |
No published protocol, however outcomes in methods section align with those in the results
Insufficient detail on analysis intentions, there is likely more than one way in which the outcome measurement could have been analysed |
Some concerns |
Overall some concerns due to no information about allocation concealment, missing outcome data (large dropout), potential bias in the measurement of outcome (self‐report), and insufficient information on the analysis intentions |
| Steele 2019 |
Some concerns |
Method used to generate random sequence not described: "Families were randomized in two blocks of 8, stratified by child age less than versus greater than 18 months of age, with 57% of children being under 18 months of age, and the remainder (43%) over 18 months but under 3 years of age"
Methods used to conceal allocation not fully described, assignment envelopes were used without all the appropriate safeguards, no indication if they were non‐opaque or sequentially numbered: "For randomization, research staff opened sealed envelopes that allocated participants in a 1:1 ratio to treatment (GABI or STEP control) groups, stratified by child age (less than vs. greater than 18 months)"
No significant differences between groups. |
Low risk of bias |
Participants and people delivering the interventions would have likely been aware of their group allocation due to differences in the interventions (i.e., those allocated to the intervention condition received additional care strategies and more contact time)
No deviation from intended intervention specified, unlikely that the outcome would be impacted by lack of blinding
Modified ITT excluding participants with missing outcome data |
High risk of bias |
63‐68% dropout (from total sample)
No evidence that the results was not biased by the missing data
Some reasons for dropout reported (unstable housing), not specific to each condition, so assume missing mechanism is similar between conditions
"Failed intakes and attrition was due mainly to housing instability with families moving and becoming un‐contactable", CONSORT states for both groups, "attended no more than a few sessions, moved, were not contactable"
Likely that missingness in the outcome depends on its true value as missing data from participants with unstable living conditions who may be more likely to score worse on this outcome |
Low risk of bias |
Validated outcome measure used
Same measurement and assessment time points used for all participants
Outcome assessors were blinded to treatment condition: "Assessors (and coders) were masked to treatment group and time" |
Low risk of bias |
No published protocol, however outcomes in methods section align with those in the results
All reported results correspond to intended outcome measurements, however data only reported at post‐treatment, baseline data not reported.
As interaction for this outcome was not significant, relevant adjusted means difference SE of difference and 95%CIs not reported: "Where an interaction of treatment group by ACE group is found to be significant, the relevant adjusted means difference for the ACE subgroups at end of treatment, standard error of difference, and 95% confidence intervals are also provided in the tables" |
High risk of bias |
Overall high risk of bias due to no information on the randomization process and missing outcome data (large dropout) |
