Figure 1.

Diagrammatic representations of potential explanations for weak estimated effects attributed to theory in meta-analyses. The diagram provides a visualization of four hypothetical scenarios where a comparison is made between a set of theory-based studies vs. a set that did not use theory. Each box represents one study. The number of green stars and the darkness of the fill represents stronger estimated treatment effects. In each hypothetical scenario, the overall effect in the theory study arm may be attenuated by either confounding, imprecise measurement, conflation, or status pooling. In the example confounding scenario, studies that use theory also have a higher quality experimental design; the effects in Study 3 may be underestimated as an artifact of poor design features. The theory depth scenario exemplifies the situation where theory is invoked in a study, but it was not used in the intervention design, so studies that used theory throughout the intervention development are combined with those that only invoke theory in the research report. In the conflation scenario, studies based on models are treated as being theory-based, so theories are pooled with other types of scientific representations. In the pooling of status scenario, studies using theories that are poor fits to the context or have been demonstrated to be inferior for the particular application and proposed theories are pooled with accepted theories that have been successful and severely tested in the domain of application.