Table 1.
SPR standards for efficacy
| Number | Standards |
|---|---|
| 1. | A statement of efficacy should be of the form that “Intervention X is efficacious for producing Y outcomes for Z population at time T in setting S.” |
| Intervention description | |
| 2.a. | The intervention must be described at a level that would allow others to implement/replicate it. |
| 2.b. | A clear theory of causal mechanisms should be stated. |
| 2.c. | A clear statement of “for whom” and “under what conditions” the intervention is expected to be effective should be stated. |
| 2.d. | The core components of the intervention and the theory relating these components to the outcomes must be identified and described. |
| 2.e. | The anticipated timing of effects on theoretical mediators and ultimate outcomes must be described. |
| 2.f. | It is necessary to characterize the research evidence supporting the potential that the intervention will affect outcomes that have practical significance in terms of public health impact. |
| Measures and their properties | |
| 3.a. | The statement of efficacy can only be about the outcomes that are measured and reported [Reporting Standard]. |
| 3.b. | The quality and quantity of implementation must be measured and reported. |
| 3.b.i | Precursors to actual implementation must be measured and reported. |
| 3.b.ii | The integrity and level of implementation/delivery of the core components of the intervention must be measured and reported. |
| 3.b.iii | The acceptance, compliance, adherence, and/or involvement of the target audience in the intervention activities must be measured and reported. |
| 3.b.iv | Level of exposure should be measured, where appropriate, in both the treatment and control conditions. |
| D | Document factors related to the quality and quantity of implementation. |
| 3.c. | Clear cost information must be reported [Reporting Standard]. |
| D | Report cost-effectiveness information. |
| D | Collect data on outcomes that have clear public health impact. |
| D | Measure potential side effects or iatrogenic effects. |
| 3.d. | There must be at least one long-term follow-up at an appropriate interval beyond the end of the intervention or, for ongoing interventions, beyond the implementation of the intervention. |
| 3.e. | Measures must be psychometrically sound. |
| 3.e.i | Construct validity—Valid measures of the targeted behavior must be used, following standard definitions within the appropriate related literature. |
| 3.e.ii | Reliability—Internal consistency (alpha), test–retest reliability, and/or reliability across raters must be reported. |
| D | Use of multiple measures and/or sources. |
| 3.e.iii | Where “demand characteristics” are plausible, there must be at least one form of data (measure) that is collected by people different from the people who are applying or delivering the intervention. This is desirable even for standardized achievement tests. |
| Theory testing | |
| 4. | The causal theory of the intervention should be tested. |
| Valid causal inference | |
| 5.a. | The design must have at least one control condition that does not receive the tested intervention. |
| 5.b. | Assignment to conditions needs to minimize bias in the estimate of the relative effects of the intervention and control condition, especially due to systematic selection, and allow for a legitimate statistical statement of confidence in the results. |
| 5.b.i | For generating statistically unbiased estimates of the effects of most kinds of preventive interventions, well-implemented random assignment is best. |
| 5.b.ii | Publications should specify exactly how the randomization was done and provide evidence of group equivalence [Reporting Standard]. |
| 5.b.iii | Well-conducted regression discontinuity designs are second only to random assignment studies in their ability to generate unbiased causal estimates. |
| 5.b.iv | For some kinds of large-scale interventions where randomization is not practical or possible, comparison time series designs can provide unbiased estimates of intervention effects. |
| 5.b.v | Nonrandomized matched control designs rarely produce credible results. They should be used only under the certain conditions specified in text. |
| 5.c. | The extent and patterns of missing data must be addressed and reported. |
| Statistical analysis | |
| 6.a. | Statistical analysis must be based on the design and should aim to produce a statistically unbiased estimate of the relative effects of the intervention and a legitimate statistical statement of confidence in the results. |
| 6.b. | In testing main effects, the analysis must assess the treatment effect at the level at which randomization took place. |
| 6.c. | In testing main effects, the analysis must include all cases assigned to treatment and control conditions. |
| 6.d. | Pretest differences must be measured and statistically adjusted, if necessary. |
| 6.e. | When multiple outcomes are analyzed, the researcher must provide a clear rationale for the treatment of multiple outcomes. |
| Efficacy claims | |
| 7.a. | Results must be reported for every targeted outcome that has been measured in the efficacy study, regardless of whether they are positive, nonsignificant, or negative [Reporting Standard]. |
| 7.b. | Efficacy can be claimed only for constructs with a consistent pattern of nonchance findings in the desired direction. |
| 7.c. | For an efficacy claim, there must be no serious negative (iatrogenic) effects on important outcomes. |
| Reporting | |
| 8. | Research reports should include the elements identified in the 2010 CONSORT guideline or a relevant extension of these guidelines. |
Note: Desirable standards are shown in italics and denoted as “D” in the Number column. Bold indicates that the standard has been taken with little or no modification from Flay et al. (2005)