1. Visual analysis |
4-step, 6-variable procedure (based on Parsonson & Baer, 1978) |
Acceptable (no specific guidelines or procedures offered) |
Change in level
Minimal score overlap
Change in trend
Adequate length (≥ 3)
Stable data (Franklin et al., 1996; Parsonson & Baer, 1992)
|
N/A |
Not acceptable (“use statistical analyses or describe effect sizes” p. 389) |
N/A |
2. Statistical analysis procedures |
Estimating effect sizes: nonparametric and parametric approaches, multilevel modeling, and regression (recommended) |
Preferred when the number of data points warrants statistical procedures (no specific guidelines or procedures offered) |
Rely on the guidelines presented by Wilkinson and the Task Force on Statistical Inference of the APA Board of Scientific Affairs (1999) |
Type not specified – report value of the effect size, type of summary statistic, and number of people providing the effect size information |
Specific statistical methods are not specified, only their presence or absence is of interest in completing the scale |
Aggregated or disaggregated approach
Model used in analyses
Details of procedures (e.g., autocorrelation approach, random effect levels)
|
3. Demonstrating an effect |
Documented consistency of level, trend, and variability within each phase
Documented immediacy of the effect, the proportion of overlap, the consistency of the data across phases
Identify for whom the intervention is and is not effective, if available
Examine external factors and anomalies
|
ABAB - stable baseline established during first A period, data must show improvement during the first B period, reversal or leveling of improvement during the second A period, and resumed improvement in the second B period (no other guidelines offered) |
0.05 alpha levels
Nonsignificant or negative outcomes noted
Type of effect size, type of data on which effect size is based, effect size statistic
Clinical/educational significance (e.g., social comparison)
Follow-up of original study participants and multiple intervals with same outcome measures
|
N/A |
N/A |
N/A |
4. Replication |
Minimum of 5 studies
The studies must be conducted by at least 3 different research teams at 3 different geographical locations
The combined number of experiments (i.e., single-case design examples) across the studies totals at least 20
|
3 replications of ≥ 3 subjects each
Replications conducted by ≥ 2 independent research groups
|
Same intervention (treatment protocol and duration)
Same target problem and sample
Independent evaluation
|
N/A |
Replication occurs across subjects, therapists, or settings |
N/A |