Table 2.
PASS analysis of 13 studies in the review
| Reference in Supplementary File | Study method | Practicability (1–3) | Applicability (1–3) | Sensitivity (1–3) | Specificity (1–3) | Justification |
|---|---|---|---|---|---|---|
| Reference in Supplementary File | Study method | Practicability (1–3) | Applicability (1–3) | Sensitivity (1–3) | Specificity (1–3) | Justification |
| Ref no. 47 | Exp | 1 | 1 | 1 | 1 | P: Difficult to recruit participants; full scale trial would be difficult to achieve. A: Difficult to generalize to population of interest in settings that could deliver the intervention. Se: Small sample limits detection of effect. Sp: Small sample limits ability to rule out no effect. |
| Ref no.68 | Exp | 2 | 1 | 1 | 1 | P: Uncertain whether adequately powered scale trial could be conducted within acceptable resource constraints. A: Sample likely to be unrepresentative; difficult to translate findings to real-world settings. Se: Small sample and weak outcome measurement limits detection of effect. Sp: Small sample and potential confounding limit ability to rule out no effect. |
| Ref no.71 | Exp | 2 | 2 | 2 | 2 | P: Adequate power to detect an effect on the intervention package but not to detect effect of individual BCTs. A: Uncertain how well could be applied in practice. Se: Weak outcome measurement relative to what is needed for clinically meaningful health outcomes; difficult to extract individual BCT effectiveness. Sp: Sample too small to rule out small effects of individual BCTs or specific BCT combinations. |
| Ref no.1 | Exp | 2 | 2 | 1 | 1 | P: Identified a specific BCT but challenging to undertake a study of sufficient size to obtain reliable effect size estimates at sufficiently long follow-up. A: Unclear whether study population is generalizable to populations of interest. Se: Small sample size limits ability to detect BCT effect. Sp: Small sample size limits ability to be confident about no effect. |
| Ref no.34 | Exp | 1 | 1 | 1 | 1 | P: Setting up a sufficiently well-controlled study with a large enough sample with adequate outcome measures is extremely challenging. A: The setting, methods, and sample make generalizability to the wider population problematic. Se: Weak outcome assessment and comparing two potentially active BCTs against each other limits ability to detect effects. Sp: Use of multiple outcomes and potential for bias in measurement make confident ascription of effects to BCTs problematic. |
| Ref no.56 | Exp | 2 | 2 | 2 | 1 | P: Study was able to detect intervention effect, but design did not include studying effect of specific BCTs. A: The study methods provide reasonable confidence of generalizability to population of interest but need for consent and self-selection bias may have influenced findings. Se: As an evaluation of an intervention package, identifying contribution of specific BCTs could not be done with confidence, but possible mediating effect of nicotine replacement use provided some indication. Sp: As an evaluation of an intervention package, identifying lack of contribution of specific BCTs was not possible. |
| Ref no.16 | Exp | 2 | 1 | 1 | 1 | P: Identifying specific BCTs using this methodology would be highly challenging as would securing a sufficient sample size. A: High risk of self-selection bias Se: Small sample size limits ability to detect BCT effects. Sp: Small sample size and potential measurement bias limits ability to rule out ineffective BCTs |
| Ref no.75 | Meta-an | 2 | 2 | 1 | 1 | P: Limited number of studies available for analysis; large variability on multiple features among the included studies; poor recording of BCTs in intervention and control conditions in studies included. A: Significant risk of self-selection bias in studies included; many studies were more than 10 years old. Se: Able to detect an overall effect of intervention packages but not the effect of specific BCTs. Sp: Unable to rule out effect of specific BCTs |
| Ref no.76 | Meta-reg | 3 | 3 | 2 | 2 | P: Sufficient studies to draw useful conclusions; somewhat limited by study descriptions. A: Some risk of self-selection bias but otherwise reason to believe findings would be generalizable. Se: Able to detect higher effect sizes associated with inclusion of specific BCTs; not able to detect BCT interactions; not able to rule out confounding by other study features or other BCTs. Sp: Unable to draw confident conclusions about ineffective BCTs. |
| Ref no.97 | Meta-reg | 3 | 3 | 2 | 2 | P: Sufficient studies to draw useful conclusions; somewhat limited by study descriptions. A: Some risk of self-selection bias but otherwise reason to believe findings would be generalizable. Se: Able to detect higher effect sizes associated with inclusion of specific BCTs; not able to detect BCT interactions; not able to rule out confounding by other study features or other BCTs. Sp: Unable to draw confident conclusions about ineffective BCTs. |
| Ref no.105 | Desc | 1 | 2 | 1 | 1 | P: Limited number of studies with significant effects on which to draw; large variability in study methods and outcome measures. A: Self-selection bias of studies may limit generalizability; some interventions may not translate to real- world settings. Se: Few effective studies and low quality limits ability to identify effective BCTs. Sp: Few effective studies and low quality limits ability to identify ineffective BCTs. |
| Ref no.135 | Desc | 1 | 2 | 1 | 1 | P: Most of the included studies were small, lacked a control group, and had a short follow-up. A: Unclear how well BCTs could be applied in current settings as more than 50% of studies were more than 10 years old. Se: Limited ability to detect effective BCTs given the number of studies. Sp: Specific BCTs in effective interventions may not be contributing to the effect. |
| Ref no.115 | Desc | 1 | 2 | 1 | 1 | P: Able to be conducted within a reasonable time frame and reasonable cost; severely limited by the number and quality of studies available for review. A: Unclear how well BCTs could be applied in current settings as more than 50% of studies were more than 10 years old. Se: Limited ability to detect effective BCTs given the number of studies. Sp: Specific BCTs in effective interventions may not be contributing to the effect. |
Exp Experimental study; Obs Comparative observational study; Meta-an Meta-analysis; Meta-reg Meta-regression; Desc Description of intervention content of effective interventions in RCTs.
BCTs behavior change techniques; P Practicability; A Applicability; Se Sensitivity; Sp Specificity.