Table 1.

Issues of efficacy vs effectiveness related to implementation of interventions into the multilevel context of routine practice and policy^*

Multilevel intervention considerations	Efficacy	Effectiveness
Personnel	Carefully selected, trained, and supervised in their behavior as interventionists Little discretion permitted in their deviation from the experimental protocol	Usually not as dedicated to the intervention (one of many responsibilities) Level of training, supervision, and protocol-adherence varies
Financing and time allocation	Research grant–supported intervention provides for greater and more dependable resource allocation/dedication in time and funding	Grant support rarely covers dedicated time/effort in nonacademic practice settings Significant competing demands for time and attention
Diversity of patients	Focus on carefully considered inclusion and exclusion criteria Restricts exposure to those most likely to benefit Exclusion criteria and attrition in highly controlled trials skews distribution of patient characteristics to a more unrepresentative sample from which to infer applicability of intervention elsewhere	Applied to patients with greater diversity and heterogeneity Higher external validity but with greater variability in effect Subgroup analyses are important for evaluating differential effects but are usually omitted because subgroups were not randomized Subgroup analyses enable better judgments about relevance and applicability of findings for different types of patients in different settings Advance consideration in sampling and stratification needed to ensure adequate subgroup sample sizes
Diversity/mix of providers	May focus on a very small number of providers (even n = 1) Provider qualifications may be specific to setting (ie, skill-mix unique to large tertiary care academic medical center) May represent willing colleagues with established relationships	Greater diversity of provider training, experience, and skill Higher external validity but with greater variability in effect May require adding training and other provider behavior change components Should include provider-level measurements
Diversity of practices or organizations	Commonly one or more selected academic medical centers (rarely if ever randomly drawn) May include principal investigator’s institution, potentially conferring unusual degree of influence/control	If retain focus on academic centers, may draw from diverse geographic regions and locations (eg, urban/rural) May require additional training and other organizational behavior change components Likely to require site investigators and provider behavior change components relevant to local context Should include practice and/or organizational level measurements Use of one or more PBRN increases external validity
Diversity of community/area	Tends to reflect large urban areas	Still tend to reflect larger urban areas but may stratify by region, location, or other area characteristics (eg, health-care resources, sociodemographic mix)
Unintended consequences of study procedures	Informed consent and testing procedures limit generalizability to settings/applications where these procedures would not be linked to the intervention	Consent and testing procedures commonly still in place May influence sample representativeness at multiple levels (ie, inability to assess effectiveness in sites without an IRB if conducting research)

^*IRB = institutional review board; PBRN = practice-based research network.