Figure 2.

Hypothetical AMP-based measurement results are illustrated for the ideal scenario that a true-positive effect is observed in the test category X/O (dark bar), whereas the absence of false-positive effects is confirmed (bright bars) as part of some generic research design. The figure illustrates the AMP-4 experimental design (Walleczek, in preparation). The form of the standard, true-experiment is visualized in (A), where the true-intervention is marked X, and the accompanying standard control is marked O. The formal structure of the sham-experiment is visualized in (B), where the sham-intervention is marked X_S, and the accompanying control is marked O_S. For an explanation of the purpose of the sham-experiment, see Section “Sham-Experiment: Counterfactual Meta-Experimentation.” For demonstration purposes only, examples of hypothetical (artifact-free) findings are shown in (C) for the true-experimental categories X/O and O/O (A) as well as for the sham-experimental categories X_S/O_S and O_S/O_S (B). In this hypothetical scenario, the true-positive finding represents an effect increase by X (dark bar) which is not challenged by systematic methodological error or SME, either in the form of class-A error or in the form of class-B error (see Section “Sham-Experiment: Counterfactual Meta-Experimentation”), as is indicated by the null results for test categories O/O, X_S/O_S, and O_S/O_S in (C). For an explanation of the SNC-experiment, see Section “Three Additional Control Test Categories.” The dashed line in (C) represents the null-effect line. For details, see main text.