Table 5.

Analysis Standards and Guidelines

	What Works Clearinghouse	APA Division 12 Task Force on Psychological Interventions	APA Division 16 Task Force on Evidence-Based Interventions in School Psychology	National Reading Panel	The Single-Case Experimental Design Scale (Tate et al., 2008)	Ecological Momentary Assessment (Stone & Shiffman, 2002)
1. Visual analysis	4-step, 6-variable procedure (based on Parsonson & Baer, 1978)	Acceptable (no specific guidelines or procedures offered)	Change in level Minimal score overlap Change in trend Adequate length (≥ 3) Stable data (Franklin et al., 1996; Parsonson & Baer, 1992)	N/A	Not acceptable (“use statistical analyses or describe effect sizes” p. 389)	N/A
2. Statistical analysis procedures	Estimating effect sizes: nonparametric and parametric approaches, multilevel modeling, and regression (recommended)	Preferred when the number of data points warrants statistical procedures (no specific guidelines or procedures offered)	Rely on the guidelines presented by Wilkinson and the Task Force on Statistical Inference of the APA Board of Scientific Affairs (1999)	Type not specified – report value of the effect size, type of summary statistic, and number of people providing the effect size information	Specific statistical methods are not specified, only their presence or absence is of interest in completing the scale	Aggregated or disaggregated approach Model used in analyses Details of procedures (e.g., autocorrelation approach, random effect levels)
3. Demonstrating an effect	Documented consistency of level, trend, and variability within each phase Documented immediacy of the effect, the proportion of overlap, the consistency of the data across phases Identify for whom the intervention is and is not effective, if available Examine external factors and anomalies	ABAB - stable baseline established during first A period, data must show improvement during the first B period, reversal or leveling of improvement during the second A period, and resumed improvement in the second B period (no other guidelines offered)	0.05 alpha levels Nonsignificant or negative outcomes noted Type of effect size, type of data on which effect size is based, effect size statistic Clinical/educational significance (e.g., social comparison) Follow-up of original study participants and multiple intervals with same outcome measures	N/A	N/A	N/A
4. Replication	Minimum of 5 studies The studies must be conducted by at least 3 different research teams at 3 different geographical locations The combined number of experiments (i.e., single-case design examples) across the studies totals at least 20	3 replications of ≥ 3 subjects each Replications conducted by ≥ 2 independent research groups	Same intervention (treatment protocol and duration) Same target problem and sample Independent evaluation	N/A	Replication occurs across subjects, therapists, or settings	N/A