Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Nov 1.
Published in final edited form as: Am Psychol. 2020 Nov;75(8):1052–1066. doi: 10.1037/amp0000721

Towards the Data-Driven Dissemination of Findings from Psychological Science

Jonathan Purtle 1, Jacob S Marzalik 2, Raquel W Halfond 2, Lynn F Bufka 2, Bethany A Teachman 3, Gregory A Aarons 4
PMCID: PMC8182894  NIHMSID: NIHMS1697965  PMID: 33252944

Abstract

The public health impact of psychological science is maximized when it is disseminated clearly and compellingly to audiences who can act on it. Dissemination research can generate knowledge to help achieve this, but dissemination is understudied in the field of implementation science. As a consequence, the designs of dissemination strategies are typically driven by anecdote, not evidence, and are thus often ineffective. We address this issue by synthesizing key theory and findings from consumer psychology and detailing a novel research approach for “data-driven dissemination.” The approach has three parts: 1) formative audience research, which characterizes an audience’s awareness about, adoption of, and attitudes towards an intervention, as well as preferences for receiving information about it, 2) audience segmentation research, which identifies meaningful sub-groups within an audience to inform the tailoring of dissemination strategies, and 3) dissemination effectiveness research, which determines the strategies that are most effective. This approach is then illustrated using the dissemination of the American Psychological Association’s (2017) Clinical Practice Guideline for the Treatment of Post-Traumatic Stress Disorder (PTSD) in Adults as a case study. Data are presented from a 2018–2019 survey of licensed APA-member psychologists who treat adults with PTSD (n= 407, response rate= 29.8%). We present survey findings on awareness about, attitudes towards, and adoption of the guideline and find significant differences across these domains between psychologists who do and do not regularly use clinical practice guidelines. We conclude by discussing future directions to advance dissemination research and practice.

Keywords: Dissemination research methods, Clinical practice guidelines, Implementation science

The public health impact of psychological science is optimized when communicated in meaningful and persuasive ways to the audiences who can act on it (McHugh & Barlow, 2010). Dissemination research is central to effective communication. The National Institutes of Health (NIH) define dissemination research as “the scientific study of targeted distribution of information and intervention materials to a specific public health or clinical practice audience,” and define implementation research as “the scientific study of the use of strategies to adopt and integrate evidence-based health interventions into clinical and community settings” (NIH, 2019). In other words, dissemination focuses on the spread of the knowledge via strategic communication whereas implementation focuses on the use of knowledge via targeted behavior change. Furthermore, as Lomas (1993) describes, dissemination is also different from diffusion; dissemination involves active and strategic efforts to spread information whereas diffusion relates to the passive and “haphazard” spread of information (p. 263).

Despite the fact that dissemination is practically and conceptually distinct from implementation, dissemination has been understudied in the field of dissemination and implementation (D&I) research (“d&I” more accurately reflects the state of the science). For example, Proctor and colleagues’ (2011) widely cited taxonomy of implementation outcomes does not discuss dissemination outcomes. Lists of strategies to promote evidence-based practice in health care—such as the Effective Practice and Organization of Care taxonomy and Expert Recommendations for Implementing Change (Powell et al., 2015) compendium—do not differentiate between dissemination and implementation strategies.

Recently, however, dissemination has received increased attention in the field. Leeman and colleagues (2017) classified dissemination strategies as a distinct type of activity to translate research into practice—defined as strategies that influence outcomes of awareness about, attitudes towards, and intention to adopt an evidence-based intervention through the development and distribution of messages and materials about an evidence-based intervention. An Agency for Healthcare Research and Quality systematic review identified three ways that such strategies could be enhanced: 1) tailoring materials to resonate with different target audiences; 2) using narratives and case studies to make materials more engaging; and 3) using framing strategies to emphasize specific aspects of an intervention and its benefits (McCormack et al., 2013). Brownson and colleagues (2018) and Purtle and colleagues (2020) further outline practical strategies to disseminate information about evidence-based interventions.

While these strategies offer conceptual guidance to inform dissemination activities, the practice of dissemination requires making concrete decisions about the content of dissemination materials and how they are distributed. These decisions relate to questions such as:

  • What information should be included in dissemination materials?

  • Who should deliver dissemination materials?

  • What should be emphasized in a narrative that illustrates the benefits of an intervention?

  • How should materials be tailored for different target audiences?

These are empirical questions, but the current D&I literature provides little guidance on how to answer them. As a consequence, the designs of dissemination strategies are typically based on anecdote instead of evidence and are thus often ineffective at reaching practice audiences or meaningfully changing their awareness about, attitudes towards, or intentions to adopt an intervention or guideline (Grimshaw et al., 2004; McCormack et al., 2013). This is in stark contrast to the data-driven processes through which findings from consumer psychology research are applied to persuade consumers to buy products (Haugtvedt, Herr, & Kardes, 2018), and direct to consumer marketing efforts that aim to cultivate demand for evidence-based mental health treatments among patients (Becker, 2015).

Within the field of implementation science there are a multitude of frameworks, which can be classified as determinant (i.e., what predicts implementation outcomes), process (i.e., describing or guiding phases of implementation), or evaluation (i.e., specifying criteria to evaluate implementation) frameworks (Nilsen, 2015). In this paper, the Exploration, Preparation, Implementation, Sustainment (EPIS) framework is used to guide the discussion (Aarons, Hurlburt, & Horwitz, 2011). Temporally, effective dissemination typically precedes implementation and reaches an audience during the “exploration phase” in which new practices are being considered (Aarons et al., 2011).

This article begins to address a dissemination knowledge gap by detailing a research approach for “data-driven dissemination” (i.e., using empirical data, not dogma or anecdote, to inform dissemination decisions). To situate the approach within the broader context of relevant psychological science, we first selectively review theories and findings about persuasion from the field of consumer psychology. Then, after the approach is detailed, it is illustrated using the dissemination of the American Psychological Association’s (APA, 2017) Clinical Practice Guideline for the Treatment of Post-Traumatic Stress Disorder (PTSD) in Adults as a case study. Empirical data from a formative audience dissemination research survey are presented and examples of how these data can inform future audience segmentation and dissemination effectiveness research are discussed. Finally, priorities are highlighted for dissemination research and practice within the broader field of implementation science. The overarching goal of the article is to provide guidance about how to conceptualize and carry out dissemination research, not how to execute dissemination practice. General overviews of strategies for dissemination practice are provided elsewhere (e.g., Brownson et al., 2018; McCormack et al., 2013; Purtle et al., 2020). The article aims to spur and support the generation of knowledge that can be applied to increase the success of active dissemination strategies.

Persuasion: Applying Theory and Findings from Consumer Psychology to Dissemination Research

Like dissemination research, consumer psychology research is conducted with the goal of generating knowledge to inform the development of persuasive campaigns. The key distinction, however, is that dissemination efforts seek to “sell” evidence to practice audiences while marketing and advertising generally seek to sell products to consumers. Despite this fundamental difference, consumer psychology and the broader body of psychological science related to persuasion (e.g., Cialdini, 1993; O’keefe, 2008) hold insights relevant to dissemination research. In this section, we provide a selective overview of theory and findings that are highly applicable. The overview was informed by reviewing broad, authoritative sources—such as the APA Handbook of Consumer Psychology (Haugtvedt et al., 2018), journals of APA Division 23 (The Society for Consumer Psychology) such as Consumer Psychology Review, and Annual Review of Psychology papers on the topic (Loken, 2006; Tybout & Artz, 1994)—with a focus on persuasion via asynchronous communication (e.g., marketing and advertising). This focus reflects NIH’s definition of dissemination research (stated above). Theories and findings most relevant were then explored further through targeted literature searches.

Elaboration Likelihood Model

The Elaboration Likelihood Model (ELM) is a general framework developed to explain the processes through which persuasive communications influence attitudes (Petty & Cacioppo, 1986, 1996; Petty, Haugtvedt, & Smith, 1995). ELM is not specific to consumer psychology but has been instrumental to shaping advertising and marketing research. In short, ELM posits that persuasive messages are processed through two routes: a central route that is cognitive and analytic, and a peripheral route that is more affective and reliant on heuristics (Petty & Cacioppo, 1986). If a person is motivated to process a message (e.g., it is perceived as relevant) and has the ability to process it (e.g., they have the prerequisite knowledge for comprehension) then the message can change attitudes via the central route. If motivation and ability criteria are not met, but if a peripheral cue is perceived in the message (e.g., it prompts positive or negative affect) then it can change attitudes via the peripheral route (Petty & Cacioppo, 1986,). Although messages can change attitudes via central or peripheral routes, changes that occur via the central route generally result in greater cognitive elaboration and more enduring attitude changes that are stronger predictors of behavior (Haugtvedt & Kasmer, 2008).

While ELM has been used extensively across branches of psychology (e.g., consumer, social, political), it has not been widely used in dissemination research (Liang et al., 2017). In one instance, ELM was used to design and test a dissemination strategy that provided primary care physicians with evidence about smoking cessation in the form of a quiz to increase cognitive elaboration and central processing; however it had no effect on provider referrals to cessation services or intention to refer (Vogt, Hall, Hankins, & Marteau, 2009).

One way that ELM could aid dissemination research is by providing a framework for experiments that examine the routes via which messages about different types of evidence might be processed by various practice audiences. For example, if guidelines for youth antipsychotic medication prescribing are the evidence being disseminated, and child psychiatrists are the audience, a dissemination experiment might manipulate the extent to which dissemination materials emphasize the strength of evidence supporting the recommendations because most recipients will have motivation and ability to process the information (i.e., central route processing is likely). In contrast, if statistics about the prevalence and consequences of youth antipsychotic overprescribing are the evidence and state legislators are the audience, an experiment might manipulate the features of stories (i.e., narratives) about the problem because few recipients are likely to have prior knowledge or interest in the topic or allocate time to process the information (i.e., peripheral route processing is likely). As described below, message tailoring offers an approach to applying ELM theory to dissemination research.

Persuasion Knowledge Model

People, whether they are consumers or practice audiences, typically know when someone is trying to persuade them. The persuasion knowledge model (PKM) explores how this knowledge is activated, influences responses to persuasion attempts, and diffuses across cultural contexts (Friestad & Wright, 1994, 1999). PKM delineates between three types of knowledge: persuasion knowledge relating to beliefs about the motives of the persuader and appropriateness of persuasion tactics; agent knowledge relating to perceived competencies of the persuader; and topic knowledge relating to the message recipient’s expertise on the subject matter. In general, the effectiveness of persuasive attempts decreases as activation of persuasive knowledge increases, with suspicion of the persuader’s motives being especially detrimental to the effectiveness of persuasion attempts (Campbell & Kirmani, 2008).

PKM is relevant to dissemination research because it can help elucidate if and how research evidence is processed when packaged in different formats and sent from different sources. For example, relevant questions may relate to clinical psychologists’ perceptions of the motives behind the development and dissemination of clinical practice guidelines, how these perceptions vary according to the dissemination agent (e.g., professional societies vs. government agencies vs. insurance companies), and differ by psychologists’ level of expertise on the guideline topic. While PKM has been extensively studied in consumer psychology, it has not received attention in D&I research.

Message Tailoring

Tailoring entails manipulating elements of a message so that it fits, like a garment, to personal attributes of the recipient. Tailored messages are generally more persuasive than “one-size-fits-all” messages (Noar, Benac, & Harris, 2007) and online communication (e.g., e-mail, websites) has dramatically enhanced the efficiency and precision of tailoring (Kosinski, Stillwell, & Graepel, 2013). Dijkstra (2008) provides an overview of three types of “tailoring-ingredients of computer-tailored persuasion” that are relevant to dissemination research: personalization, adaptation, and feedback.

Personalization

Personalization involves incorporating one or more recognizable aspects of the message recipient into the message content. Personalization has no persuasive power in and of itself but is a cue that can positively affect engagement with and processing of information. For example, an experiment of 68,000 e-mails found that adding the recipient’s first name to the subject line increased the probability of opening the e-mail by 20% and reduced the probability of unsubscribing from future e-mails by 31% (Sahni, Wheeler, & Chintagunta, 2018). According to ELM, personalization could work via peripheral route mechanisms because people generally have positive attitudes towards salient features of themselves, as illustrated by studies on name similarity and generosity (Munz, Jung, & Alter, 2020).

Personalization could also function via central route mechanisms by signaling that a message is relevant to the target, therein increasing cognitive elaboration (Hawkins, Kreuter, Resnicow, Fishbein, & Dijkstra, 2008; Petty, Cacioppo, & Schumann, 1983; Wheeler, Petty, & Bizer, 2005). Personalization can be counterproductive, however, and decrease the persuasive power of messages if the target has negative reactions to the content (Dijkstra, 2008). Personalization can also be detrimental if the message is perceived as irrelevant, prompts information privacy concerns, or signals that a persuasive attempt is being made, therein activating persuasion knowledge (Friestad & Wright, 1994; Petty & Cacioppo, 1986).

To our knowledge, no studies have tested the effects of personalization on practice audiences’ engagement with or processing of dissemination materials. Although extant research generally suggests that it would be beneficial to personalize dissemination materials for basic recipient characteristics such as their name (Liu-Thompkins, 2019), this is an empirical question worthy of investigation. Beyond recipient name, other elements that could be personalized and tested in dissemination effectiveness research include characteristics such as National Provider Identifier number and areas of specialization when clinicians are the audience, or legislative district numbers and committees of membership when legislators are the audience.

Adaptation

Adaptation tailors the persuasive augments and content of messages to align with the psychological state of recipients (Dijkstra, 2008). By increasing “message-person congruence,” adaptation can enhance cognitive elaboration and thus message persuasiveness. Adaptation is different from personalization because it involves tailoring message content, not simply offering cues that could affect message processing. Messages can be adapted for recipient characteristics, such as demographics and personality traits. For example, a message adapted for extroverted personality types might emphasize benefits related to social attention from adopting an evidence-based intervention, while a message for risk averse individuals might emphasize benefits related to malpractice risk mitigation (Hirsh, Kang, & Bodenhausen, 2012).

The utility of adaptation as a tailoring strategy is contingent upon the ability to match recipients with appropriately adapted messages. The capability to do this efficiently has increased dramatically with the rise of social media in which people leave “digital footprints” that facilitate the rapid development, delivery, evaluation, and refinement of adapted messages (Kosinski et al., 2013), though privacy concerns need to be thoroughly considered using these data. Online message adaptation has been used to influence a range of outcomes, spanning from elections (e.g., Cambridge Analytica and the 2016 U.S. Presidential election; Illing, 2018) to purchasing behavior. For example, Matz et al. (2017) used Facebook “likes” to adapt advertisements according to viewers’ personality traits and found that the adaptations resulted in 40% more advertising clicks and up to 50% more purchases.

As with personalization, little research has assessed the effects of adapting the content of dissemination materials for the predicted psychological states of practice audiences. Although there would be value to testing such adaptations, a challenge to taking this approach to scale is limited data about the motivations and personality traits of individual practitioners, which are needed to match recipients with messages. Large scale adaptation of dissemination materials might be most feasible for audiences of elected officials because they regularly share their opinions in public forums and this information is readily available.

Feedback

Messages that are tailored with a feedback component provide recipients with information about themselves that relates to the target behavior of the persuasive attempt. Feedback as a persuasive communication (i.e., dissemination) strategy is similar to the implementation strategy of audit and feedback. The key distinction is that, as a dissemination strategy, it is typically limited to the unidirectional, asynchronous provision of information, whereas feedback often includes other components (e.g., training, facilitation) when used as an implementation strategy. According to feedback intervention theory (Kluger & DeNisi, 1996), negative feedback that includes feasible recommendations for improvement can be persuasive because it increases motivation. However, negative or positive feedback that does not include such recommendations, or feedback that is not relevant to recipients’ goals, could undermine motivation and be counterproductive.

The effects of feedback-tailored messages on clinician behavior have been assessed through dissemination experiments. For example, an experiment targeting off-label antipsychotic medication prescribing among high-volume prescribing primary care physicians embedded feedback in a letter from the Centers for Medicare and Medicaid Services (Sacarny et al., 2018). The letter quantified the volume of the physician’s prescribing of the drug relative to their peers and indicated that there could be regulatory consequences if they did not change their behavior. The study found that the feedback was effective at changing the target behaviors.

A Research Approach for Data-Driven Dissemination

Building on the overview of theory and findings from consumer psychology, as well as implementation science, this section offers a road map of the types of studies that are needed to conduct data-driven dissemination activities. The approach for data-driven dissemination is summarized in Table 1 and entails three types of dissemination studies: formative audience research, audience segmentation research, and dissemination effectiveness research. For any given intervention or issue, we recommend these studies be conducted through an iterative process because the evidence base of psychological science is constantly evolving, and the sociopolitical context in which practice decisions are made is always changing.

Table 1.

A Research Approach for Data-Driven Dissemination

Type of Study Objective Purpose
Formative audience research Characterize a target audience’s awareness about, adoption of, and attitudes towards an intervention, and preferences for receiving information about it, as well as other individual attributes that may influence practice behavior and perceptions of context (e.g., self-efficacy, injunctive social norms). Provide an empirical foundation to inform the design and distribution of dissemination materials.
Audience segmentation research Identify discrete and meaningful sub-groups within an audience that vary in terms of their awareness about, attitudes towards, adoption of, and preferences for receiving information about an intervention. Inform the adaptation of dissemination materials and modes of delivery for different audience segments.
Dissemination effectiveness research Test dissemination strategies to determine which are most effective at changing an audience’s awareness about, attitudes towards, and adoption of an intervention. Determine which dissemination strategies should be scaled-up.
Open in a new tab

Formative Audience Research

The purpose of formative audience research is to generate data that can provide an empirical foundation to inform the design of dissemination materials, how they are distributed, and adapted for different recipients (Slater, 1996). Per ELM, this research can shed light on the routes through which messages embedded in dissemination materials might be processed by recipients (Petty & Cacioppo, 1986). Formative audience research is typically early exploration phase, descriptive, and conducted with the objectives of characterizing a target audience’s awareness about, adoption of, and attitudes towards an intervention as well as their preferences for receiving information about it. These four domains reflect the objectives of dissemination strategies proposed by Leeman et al. (2017) but are not exhaustive and may vary according to the goals, timing, and context of the dissemination initiative and the attributes of the intervention.

For example, a formative audience research study might assess self-efficacy to deliver a new, complex intervention. This information could inform how dissemination strategies might improve self-efficacy to adopt the intervention by simplifying information about delivery. A study might also assess injunctive social nor-ms related to the delivery of a well-established evidence-based intervention. This information could inform the design of dissemination strategies that integrate adapted feedback and peer-comparisons (e.g., Sacarny et al., 2018) and harness the power of social influence (for a review of social infleunce and consumer psychology see Argo, 2020). The key consideration for selecting domains of variables to assess in formative audience research is the extent to which it seems plausible that dissemination materials could affect variables in that domain.

The theoretical framework used to select, measure, and analyze domains of variables in formative audience research should be chosen based on the specific research questions guiding the dissemination study. However, it should be noted that two of the formative audience research domains detailed below—attitudes and intentions to adoption—are core to the theory of planned behavior (TBP) (Ajzen, 1991). (See Ajzen 2018 for detailed discussion of TPB as it relates to persuasion and consumer psychology). TBP is widely used in implementation science and has been applied to inform the design (Breslin, Li, Tupker, & Sdao-Jarvie, 2001; Williams, 2015) and evaluation (Casper, 2007; Yoong et al., 2015) of dissemination strategies.

Awareness

Measures of awareness in formative dissemination research include assessments of an audience’s familiarity with the intervention and its components, evidence of its effectiveness, and where resources to support implementation can be obtained—all of which relate to “topic knowledge” per PKM (Friestad & Wright, 1994). Assessment of awareness constructs can produce data about specific knowledge deficits that should be addressed by dissemination materials. Per ELM, aligning the content of dissemination materials with knowledge deficits could convey that the information is relevant and promote central processing (Petty & Cacioppo, 1986). There can also be benefits to assessing awareness about the problem that the intervention seeks to address, as this information could also help dissemination materials be perceived as more relevant. For example, a survey of state legislators assessed awareness about the strong evidence supporting state mental health parity laws (e.g., that parity laws increase access to treatments and do not increase costs) and the potential effectiveness of mental health treatments, finding strikingly low levels of awareness (Purtle et al., 2019).

Adoption

Measures of adoption in formative dissemination research include assessments of the extent to which a target audience has adopted, has attempted to adopt, or intends to adopt the intervention. Measures of the extent to which an audience has engaged with intervention materials (e.g., visited a guideline website) can also be considered adoption metrics because the behaviors can be conceptualized as indicators of intention to adopt. Per TPB, intentions to adopt an intervention mediate the relationship between attitudes towards the intervention and actual adoption (Ajzen, 2018). Williams (2015) used TPB to develop the Evidence-Based Treatment Intentions scale, which measures mental health clinicians’ intentions to adopt evidence-based treatments. There are also other instruments that can measure intention to adopt evidence-based practices (e.g., the Measure of Innovation Specific Implementation Intentions) and be tailored for questions related to specific guidelines and interventions (Moullin, Ehrhart, & Aarons, 2018). Adoption activities can move the knowledge translation process from the exploration to the preparation phase of EPIS.

Attitudes

While attitudes are often measured as outcomes in implementation studies—particularly Rogers’ (2010) Diffusion of Innovations constructs of complexity and compatibility, which are akin to the implementation outcomes of acceptability and appropriateness, respectively (Proctor et al., 2011)—their measurement in a formative dissemination studies produces data about how dissemination materials can reduce attitudinal barriers to intervention adoption. Items assessing attitudes should be focused on the specific guideline or evidence-based intervention that is the focus of the dissemination activities.

Established attitudinal measures such as the Evidence-Based Practice Attitudes Scale (e.g., EBPAS-36; Rye et al., 2017) can be easily adapted to assess attitudes toward specific interventions. Measures of persuasive knowledge, which can be conceptualized as attitudes in formative audience research, can also be adapted for specific intervention (Ham, Nelson, & Das, 2015). Rogers’ (2010) constructs related to attributes of interventions could also be assessed as attitudes in formative audience research. For example, a formative audience research study might measure attitudes related to whether an evidence-based psychological intervention is perceived as superior to the clinical status-quo (Rogers’ “relative advantage”), perceived as being aligned with the realities of practicing psychologists (Rogers’ “compatibility”), perceived as being difficult to understand or requiring intensive training (Rogers’ “complexity” and “trialability”), or perceived as being able to produce tangible clinical results (Rogers’ “observability”). For example, a survey of state mental health agency directors used Rogers’ constructs to assess correlations between attitudes towards policies that incentivize the use of evidence-based treatments and their agency’s adoption of these strategies (Stewart et al., 2018).

Attitudes can serve as independent or dependent variables in formative audience research, and such decisions should reflect the theoretical framework that is guiding the dissemination study. For example, if using ELM, attitudes would serve as the dependent variable, and the independent variables would be constructs related to motivation and ability to process information contained in dissemination materials. If using TBP, attitudes would be one of three independent variables (along with perceived behavioral control and subjective norms), while intention to adopt an intervention or actual adoption behavior would be the dependent variable.

Preferences for Receiving Information

Lastly, measures related to preferences for receiving information about the intervention assess an audience’s opinions about what information should be in dissemination materials and how these materials should be distributed. For example, measures might assess how intervention materials should be packaged (e.g., PDFs, podcasts) and the relative importance of different content elements (e.g., details about methods used to generate intervention effect estimates vs. details about implementation). These data can provide concrete guidance for developing dissemination materials that are perceived as relevant (therein increasing the chances central processing per ELM) or appealing (increasing chances of peripheral processing). Measures in this domain might also assess the extent to which different sources are perceived as sufficiently competent to disseminate information about an intervention (“agent knowledge” per PKM) and their perceived motives for dissemination (“persuasion knowledge” per PKM).

Audience Segmentation Research

The purpose of audience segmentation research is to understand how dissemination materials might be tailored for different groups within a target audience. Audience segmentation involves analytic techniques, as opposed to data collection, and aims to identify discrete and meaningful sub-groups (i.e., segments) within the target audience that vary in terms of their awareness about, attitudes towards, adoption of, and preferences for receiving information about an intervention (Slater, 1996), other psychological traits (e.g., motivation, personality type; Dykstra, 2008), or other domains of variables that are relevant to the dissemination research question. Segmentation analyses provide the empirical bases for making decisions about how to adapt of dissemination strategies for different groups, therein improving “message-person congruence,” increasing cognitive elaboration, and thereby enhancing the effectiveness of dissemination strategies (Dijkstra, 2008). For example, a study could identify segments of psychologists that differ in their motivation to adopt a new evidence-based treatment. Then, informed by ELM, messages adapted for psychologists in high motivation segments might emphasize concrete recommendations for practice change to increase central processing of information, while a messages adapted for psychologists in low motivation segments might emphasize peripheral cues (e.g., include metaphors).

Within the EPIS framework, audience segmentation can help identify mutable determinants of behavior across the exploration-to-sustainment continuum and inform how determinants in the inner or outer context could be modified through dissemination strategies that are tailored for different practice audiences. While audience segmentation and message adaptation are routine in marketing and health communication (Liu-Thompkins, 2019; Noar et al., 2007), they not been widely used in D&I research. There are two main approaches to audience segmentation research: demographic separation and empirical clustering.

Demographic Separation

With this approach, an audience is simply stratified based on personal characteristics (e.g., gender, personality traits), professional role (e.g., clinical supervisors), or organizational context (e.g., Veterans Affairs hospitals) and differences in awareness about, attitudes toward, adoption of, and dissemination preferences about an intervention are compared across strata. For example, an audience research study of state legislators’ preferences for receiving mental health research stratified the sample by the personal characteristic of political party affiliation (Purtle et al., 2018). It was found that data about economic impact is more important to Republicans than Democrats and that the sources these legislators turn to for mental health research varies by political party. A benefit to demographic separation approaches is that segmenting variables are often readily observable, which facilitates the delivery of adapted messages to audience members within that segment. A downside, however, is that it can be overly simplistic and fail to identify the most meaningful sub-groups within the target audience (Smith, 2017).

Empirical Clustering

With this approach, more advanced statistical techniques are used to identify patterns in relationships between awareness, attitude, and adoption characteristics, and create audience segments that reflect clusters of these variables. Latent class analysis, one statistical technique, identifies “classes,” which reflect different clustering patterns; audience members are then assigned to the segment to which they have the highest probability of belonging. For example, an empirical clustering study used latent class analysis to identify three behavioral health audience segments of state legislators—finding that 47% of legislators belonged to a segment that was characterized by high levels of stigma towards people with mental illness, skepticism toward the potential effectiveness of behavioral treatments, and giving more weight to budget concerns than research evidence when deciding whether to support a behavioral health bill (J. Purtle, Lê-Scherban, et al., 2018). A benefit to empirical clustering is that the use of multiple variables can produce a more nuanced and meaningful understanding of sub-groups within the target audience (Smith, 2017). A downside is that the utility of empirical clustering hinges upon the extent to which non-latent and readily observable variables (e.g., demographics) are associated with segment membership. As noted above in reference to adaptation as a tailoring strategy, linkages to readily observable variables are critical to facilitating the identification of segment members within a practice population beyond the study sample.

Dissemination Effectiveness Research

Dissemination effectiveness research examines how to communicate evidence about a particular intervention to a specific practice population. The purpose of dissemination effectiveness research is to determine which dissemination strategies are most effective at changing an audience’s awareness about, attitudes towards, and, ultimately, adoption of an intervention. As described above, ELM could be a useful theoretical framework to guide the design of dissemination effectiveness studies. Such studies should use randomized-controlled designs (or related designs that allow for causal inferences) in which the content of dissemination materials, cues within them (e.g., personalization elements), or their mode or source of delivery are manipulated among a sample of the target audience. Dissemination effectiveness studies might also test tailored vs. non-tailored dissemination strategies (Dijkstra, 2008) or positive vs. negative adapted feedback (Kluger & DeNisi, 1996).

Dissemination effectiveness studies can assess many of the same outcomes of formative audience research studies. For example, an experiment testing dissemination of evidence-based treatments for bulimia compared the effects of dissemination materials that emphasized research evidence to those that included a clinical case study, finding that psychologists randomized to case studies were more likely to have positive attitudes towards the evidence-based treatments and willing to be trained in the treatments (Stewart & Chambless, 2010). Dissemination effectiveness studies can also assess outcomes via unobtrusive observation. A dissemination effectiveness study of clinical practice guidelines on opioid prescribing compared the effects of data-focused versus narrative dissemination materials on rates of opening e-mail and hyperlink clicks, findings that the narratives were more effective (Meisel et al., 2016). Such outcomes, which could also be used to determine the effects of personalizing dissemination materials, can be conceptualized as proximal indicators of intent to adopt the intervention. Although changes in behavior are generally more appropriately characterized as implementation than dissemination outcomes, behaviors can be considered as dissemination outcomes when the independent variable is exposure to an intervention that exclusively distributed informational materials to a specific practice audience.

The EPIS framework might also be useful for conceptualizing and designing dissemination effectiveness studies. For instance, a dissemination study targeting an audience in the exploration phase might test messages included in clinical practice guidelines materials and assess how perceptions of various messages are associated with inner and outer context characteristics of the audience. Then in the preparation phase, the most promising messages can be refined and further tested. In the implementation phase, those approaches can be tested with measures of adoption and behavior as the outcomes of focus.

A Case Study of Applied Dissemination Research: The APA Clinical Practice Guideline for the Treatment of PTSD in Adults

The three-part research approach outlined above can generate data to inform, and ideally enhance, dissemination activities. This section illustrates how one aspect of this approach—formative audience research—has been applied to dissemination of the APA Clinical Practice Guideline for Treatment of PTSD in Adults (APA, 2017). The section also discusses how these data can inform subsequent audience segmentation and dissemination effectiveness research.

APA Clinical Practice Guideline for PTSD and Dissemination Activities

In 2010, the APA Council of Representatives approved an initiative for the APA to develop clinical practice guidelines and the Steering Committee for the Development of Clinical Practice Guidelines was created. As of fall 2019, the APA has created three clinical practice guidelines: one for treatment of PTSD in adults (APA, 2017), one for treatment of obesity and overweight in children and adolescents (APA, 2018), and one for treatment of depression across the lifespan (APA, 2019). Information about the PTSD guideline was first disseminated via the APA PracticeUpdate newsletter in March 2017 and a webpage dedicated to the guideline launched in July 2017. A detailed list of dissemination activities is included as Supplemental Material A.

Formative Audience Research of APA PTSD Guideline Dissemination

A web-based survey was conducted in winter 2018–2019 to characterize psychologists’ awareness about, attitudes towards, and adoption of the PTSD guideline and assess their preferences for receiving information about the guideline in the future.

Data

To construct the survey sample frame, a random sample was drawn of 1,986 APA members/fellows who were classified as licensed psychologists in the APA database. Personalized e-mail invitations to complete the web-based survey were sent between December 13, 2018 and January 10, 2019. Each respondent was e-mailed up to four times and a $5 incentive was offered for survey completion. The survey was completed by 591 respondents (response rate= 29.8%.) Non-respondent analyses revealed that there were not statistically significant differences between respondents and non-respondents in terms of gender, race/ethnicity, or mean age.

Respondents who indicated that they were not currently providing psychological treatment and/or were not currently (i.e., in the past year) providing services to adults with PTSD were excluded from the analysis because survey questions about their awareness of, attitudes towards, and adoption of the PTSD guideline would be largely irrelevant to this audience. This resulted in a dataset with responses from 407 practicing psychologists who provide services to adults with PTSD. The survey instrument included as Supplemental Material B.

Respondent Characteristics.

Respondents included in the analyses identified as 57.2% female, 82.1% non-Hispanic white, and primarily worked in private practice (77.7%). The mean age was 59.52 years (SD= 9.60). Among these respondents, 38.7% identified guidelines as a source they regularly consult to guide clinical decision making. Because respondents that regularly consult guidelines are presumably more likely to adopt the PTSD guideline than respondents that do not regularly consult guidelines, survey results were stratified by this respondent characteristic. This segmentation might inform how future dissemination activities can be tailored for clinicians that do and do not regularly consult guidelines (i.e., clinicians who are at different points in the EPIS continuum).

Awareness of the PTSD Guideline

Seventeen percent of psychologists were familiar with the PTSD guideline, operationalized as a rating of 4 or 5 on a 5-point Likert scale (Table 2). Psychologists that regularly consult guidelines were two times more likely to be familiar with the PTSD guideline than psychologists that do not regularly consult guidelines (25.3% vs. 12.4%, OR= 2.39, p= .0001). Eleven percent of psychologists were familiar with the PTSD guideline creation process and there were not significant differences in familiarity with the process between psychologists that do and do not regularly consult guidelines (12.7% vs. 9.6%, OR= 1.37, p= .321)

Table 2.

APA Member, Licensed Psychologists’ Awareness and Adoption of APA’s PTSD Guideline (N = 407)

Awareness and Adoption of APA PTSD Guideline All Psychologists Psychologists that Regularly Consult Guidelines Psychologists that Do Not Regularly Consult Guidelines
n % n % n % OR p
Familiar with the guideline a 71 17.4 40 25.3 31 12.4 2.39 .001
Familiar with the process used to create the guideline 44 10.8 20 12.70 24 9.60 1.37 .321
Have used the guideline to inform clinical practice b 45 11.0 32 20.4 13 5.3 4.59 <.0001
Will use the guideline to inform clinical practice b 154 37.7 87 55.8 67 27.0 3.41 <.0001
Downloaded materials about guideline-recommended treatments c 118 28.9 77 48.7 41 16.4 4.85 <.0001
Talked with other clinicians about how to align practice with the guideline c 69 16.9 44 27.8 25 10.0 3.47 <.0001
Attended in-person trainings about treatments recommended in the guideline c 39 9.6 22 13.9 17 6.8 2.22 .017
Talked to current/prospective patients/families about the guideline recommendations c 27 6.6 22 13.9 5 2.0 7.93 <.0001
Decided to provide a guideline-recommended treatment over another option c 25 6.1 21 13.3 4 1.6 9.43 <.0001
Sought consultation to facilitate use of the guideline c 17 4.2 11 7.0 6 2.4 3.04 .025
Open in a new tab

Note. Licensed psychologists who are currently providing services to adults with PTSD.

a

Rating of 4 or 5 on 5-point scale: 1 = Not familiar at all, 5 = Very familiar

b

Rating of 4 or 5 on 5-point scale: 1 = have not used/will not use it at all, 5 = have used/will use to a very great extent

c

Psychologists instructed to select all of the activities that they engaged in “because of” the APA PTSD guideline.

OR= odds ratio. p values from χ2 tests (df= 1)

Adoption of the PTSD Guideline

Eleven percent of psychologists indicated they had used the PTSD guideline to inform their clinical practice (i.e., adopted the guideline) and 37.7% indicated that they would use the guideline to inform their practice in the future (i.e., intended to adopt) (Table 2). Only 6.4% of psychologists endorsed the statement that they had never used the guideline and would not consider using it. Psychologists that regularly consult guidelines had four times higher odds of using the PTSD guideline than psychologists that do not regularly consult guidelines (20.4% vs. 5.3%, OR= 4.59, p<.0001) and three times higher odds of indicating that they would use the guideline in the future (55.8% vs. 27.0%, OR= 3.41, p <.0001)

Forty-four percent of psychologists indicated that they had engaged in at least one-of-six activities because of the PTSD guideline (i.e., indicators of intent to adopt; Table 2). The most frequently identified activities were downloading materials about treatments recommended in the guideline (28.9%) and talking with other clinicians about how to align practice with guideline recommendations (16.9%). Most proximal to patient care, 6.1% of psychologists indicated that they decided to provide a guideline-recommended treatment over another option because of the PTSD guideline. Psychologists that regularly consult guidelines were significantly more likely to endorse each of the six activities because of the PTSD guideline than psychologists who report they do not regularly consult guidelines.

Attitudes Towards the PTSD Guideline

Because the ability to answer questions about the PTSD guideline is contingent upon being at least somewhat familiar with the guideline, the analysis of attitude items was limited to psychologists that indicated a familiarity of 3, 4, or 5 in response to the 5-point Likert scale question: “Prior to receiving the invitation to complete this survey, how familiar were you with APA’s Clinical Practice Guideline for the Treatment of PTSD in Adults?” (n= 174).

Of these psychologists, 69.8% agreed that the PTSD guideline was based on the best research available and 68.4% agreed that it was easy to understand, both operationalized as a rating of 4 or 5 on a 5-point Likert scale (Table 3). Less than half of these psychologists agreed that the guideline could be implemented without needing to acquire new skills (46.0%) or was compatible with the realities of practicing clinicians (41.6%).

Table 3.

APA Member, Licensed Psychologists’ Attitudes Towards the APA PTSD Guideline (n = 174)

Attitude Toward APA PTSD Guideline a All Psychologists (n= 174) Psychologists That Regularly Consult Guidelines Psychologists That Do Not Regularly Consult Guidelines
n % n % n % OR p
Is based on the best research available 113 69.8 73 83.0 40 54.1 4.14 <.0001
Is easy to understand 106 68.4 66 76.7 40 58.0 2.39 .012
Is relevant to my patients 100 60.2 64 71.1 36 47.4 2.74 .002
Can be implemented at low cost 84 58.3 51 62.2 33 53.2 1.45 .280
Can be adapted for the needs of specific patients 83 54.2 54 63.5 29 42.6 2.34 .010
Does not adequately represent all points of view 72 46.2 32 37.6 40 56.3 0.47 .020
Can be implemented without me needing to acquire new skills 74 46.0 42 47.2 32 44.4 1.12 .728
Is compatible with the realities of practicing clinicians 64 41.6 39 47.6 25 34.7 1.71 .107
Is unlikely to change my current practices 62 37.3 25 27.5 37 49.3 0.39 .004
Open in a new tab

Note. Licensed psychologists who are currently providing services to adults with PTSD and are familiar with APA’s PTSD Guideline.

a

Rating of 4 or 5 on 5-point scale: 1 = Strongly disagree, 5 = Strongly agree.

OR= odds ratio. p values from χ2 tests (df= 1)

Psychologists that regularly consult guidelines generally had more positive attitudes towards the PTSD guideline than psychologists that do not regularly consult guidelines. Psychologists that regularly consult guidelines were significantly more likely to agree that the PTSD guideline is based on the best research (83.0% vs. 54.1%, OR= 4.14, p<.0001), is easy to understand (76.7% vs. 58.0%, OR= 2.39, p= .012), is relevant to their patients (71.1% vs. 47.4%, OR= 2.74, p= .002), and can be adapted for the needs of specific patients (63.5% vs. 42.6%, OR= 2.34, p= .010). Psychologists that regularly consult guidelines were significantly less likely than psychologists that do not regularly consult guidelines to agree that the PTSD guideline does not adequately represent all points of view (37.6% vs. 56.3%, OR= 0.47, p= .020) and is unlikely to change their current practices (27.5% vs. 49.3%, OR= 0.39, p= .004).

When attitudes towards the PTSD guideline were treated as continuous variables, there were significant bivariate correlations between most items tapping into different components of attitudes (see Supplemental Material C; note, we are using the term attitudes broadly to reflect a range of evaluative reactions to the guideline). Two attitudes—agreement that the guideline is relevant to patients and agreement that the guideline is compatible with the realities of practicing clinicians—were strongly correlated (r >.700, p <.05) with three other attitudes. There were strong positive correlations between agreement that the PTSD guideline is relevant to patients and agreement that the guideline can be implemented at low cost (r = .789, p <.05), is compatible with the realities of practicing clinicians (r = .764, p <.05), and is based on the best research (r = .719, p <.05). There were also strong positive correlations between agreement that the guideline is compatible with the realities of clinicians and agreement that the guideline is based on the best research (r = .761, p <.05) and can be implemented at low cost (r = .742, p <.05).

Preferences for Receiving Information about PTSD Guideline

Most psychologists indicated that they preferred to receive information about the PTSD guideline via e-mail (69.8%) and in the form of printed materials (68.2%), operationalized as a rating of 4 or 5 on a 5-point Likert scale. Slightly less than half of psychologists indicated that they preferred receiving information about the guideline in the form of webinars (46.4%) and in-person trainings (44.2%). Audio summaries/podcasts were preferred least frequently (32.7%). There were not significant differences between psychologists that do and do not regularly consult guidelines in terms of their preferred modes of receiving information about the PTSD guideline.

More than three-quarters of psychologists indicated that it was important for information about the PTSD guideline to include details about the rationale for the inclusion of specific practices/treatments (85.7%), details about the rationale for the exclusion of specific practices/treatments (77.2%), and clinical case examples (76.5%), all operationalized as a rating of 4 or 5 on a 5-point Likert scale. There were not significant differences between psychologists that do and do not regularly consult guidelines in terms of the features of information about the guideline that they perceive as important.

Implications of Formative Audience Research About the APA PTSD Guideline

Findings from formative audience research about the APA PTSD guideline can inform future audience segmentation and dissemination effectiveness research. In general, future research about dissemination of APA guidelines would be strengthened by more direct links to theory related to persuasive communication and behavior change. In terms of future audience segmentation, a latent class analysis could focus on the sources that psychologists regularly consult to guide clinical decision making. This would expand on the finding that psychologists who regularly consult guidelines (38.7%) differed from those who do not in terms of their awareness, adoption, and attitudes related to the guideline. In addition to asking about guidelines, the survey also assessed whether psychologists turned to nine other sources to inform clinical decision making (e.g., academic literature= 85.5%, patients= 34.1%; full list in Supplemental Material B). This analysis could generate a typology of psychologists that vary according to the sources of information they turn to in order to guide clinical decision making.

In terms of dissemination effectiveness research, experiments could expand on the formative audience research finding that a large proportion (76.5%) of psychologists identified clinical case examples as an important feature of guideline dissemination materials. This result, which is consistent with prior research (Meisel et al., 2016; Stewart & Chambless, 2010), reinforces the importance of including case examples in dissemination materials about APA clinical practice guidelines. It is unknown, however, which types of case examples are most effective. This could be investigated through dissemination effectiveness studies that manipulate the types of case examples in dissemination materials sent to APA members.

Future Directions for Data-Driven Dissemination

There are at least three major areas where new research approaches and partnerships could advance data-driven approaches to dissemination. First, there is a need to better integrate theory and findings from the field of communications into dissemination research and practice. Dissemination is communication; and a rich body of knowledge in the field of communication research, and the sub-field of science communication in particular, can offer insights about how to package mental health evidence and also counter misinformation about mental health issues and treatments. Communication can be conceptualized as a transactional or transformative process, and it has been largely approached as the latter field of implementation science (Manojlovich et al., 2015). Similarly, there is a need to better integrate implementation science theories (e.g., EPIS) and theories related to persuasive communication (e.g., ELM and PKM) to guide dissemination research and practice.

Second, there is a need to understand how the “information environments” (e.g., news media, social media) that practice audiences are exposed to may influence their attitudes towards evidence-based interventions. Political science research has long studied how information environments shape attitudes related to social issues and political behavior (Aalberg, Van Aelst, & Curran, 2010). Information environments can be conceptualized within the outer “sociopolitical context” domain of the EPIS framework (Aarons et al., 2011) and could moderate the effects of, and also be influenced by, dissemination strategies.

Third, there is a need to think about dissemination as part of a larger mental health knowledge translation system. The research approach for data-dissemination outlined above might generate knowledge about how to disseminate more effectively, but this knowledge will have little or no impact unless entities invest the time and resources into dissemination practice. Unfortunately, dissemination activities are typically uncoordinated and undervalued in the field of mental health in the United States. Few entities in the field have the skills, incentives, and perceived obligation to disseminate evidence effectively (Kreuter & Bernhardt, 2009), a problem that pervades public health more broadly (Brownson et al., 2018).

Conclusions

The notion that practice decisions should be informed by data, instead of anecdote, is central to the enterprises of evidence-based practice in psychology and implementation science. This article describes an approach for “data-driven dissemination” that can help hold dissemination activities to this same standard of practice. In so doing, the discipline of psychology will better advance its science and meet the aim of improving population health.

Supplementary Material

Supplemental Material 1
Supplemental Material 2
Supplemental Material 3

Public Significance Statement.

Dissemination is understudied in the field of implementation science and there is little guidance about how dissemination research should be conducted. As a consequence, the designs of dissemination strategies are typically driven by anecdote, not evidence. We address this issue by detailing a three-part research approach for “data-driven dissemination.”

Acknowledgments

Funding: National Institute of Mental Health (P50MH11366201A1S1)

Biography

graphic file with name nihms-1697965-b0001.gif

graphic file with name nihms-1697965-b0002.gif

graphic file with name nihms-1697965-b0003.gif

graphic file with name nihms-1697965-b0004.gif

graphic file with name nihms-1697965-b0005.gif

graphic file with name nihms-1697965-b0006.gif

Footnotes

Editor’s note. This article is part of a special issue, “Expanding the Impact of Psychology Through Implementation Science,” published in the Xxxxxx 2020 issue of American Psychologist. Shannon Wiltsey Stirman and Rinad S. Beidas served as editors of the special issue, with Anne E. Kazak as advisory editor.

References

  1. Aalberg T, Van Aelst P, & Curran J. (2010). Media systems and the political information environment: A cross-national comparison. The International Journal of Press/Politics, 15(3), 255–271. [Google Scholar]
  2. Aarons GA, Hurlburt M, & Horwitz SM (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Ajzen I. (1991). The theory of planned behavior. Organizational behavior and human decision processes, 50(2), 179–211. [Google Scholar]
  4. Ajzen I. (2008). Consumer attitudes and behavior. In Haugtvedt CP, Herr PM, & Kardes FR (Eds.), Marketing and consumer psychology series: Vol. 4. Handbook of consumer psychology (p. 525–548). Taylor & Francis Group/Lawrence Erlbaum Associates. [Google Scholar]
  5. Argo JJ (2020). A contemporary review of three types of social influence in consumer psychology. Consumer Psychology Review, 3(1), 126–140. [Google Scholar]
  6. Becker SJ (2015). Direct-to-consumer marketing: A complementary approach to traditional dissemination and implementation efforts for mental health and substance abuse interventions. Clinical Psychology: Science and Practice, 22(1), 85–100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Breslin C, Li S, Tupker E, & Sdao-Jarvie K. (2001). Application of the theory of planned behavior to predict research dissemination: A prospective study among addiction counselors. Science Communication, 22(4), 423–437. [Google Scholar]
  8. Brownson RC, Eyler AA, Harris JK, Moore JB, & Tabak RG (2018). Research full report: getting the word out: new approaches for disseminating public health science. Journal of Public Health Management and Practice, 24(2), 102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Campbell MC, & Kirmani A. (2008). I know what you’re doing and why you’re doing it. Handbook of Concumer Psychology, 549–574. [Google Scholar]
  10. Casper ES (2007). The theory of planned behavior applied to continuing education for mental health professionals. Psychiatric Services, 58(10), 1324–1329. [DOI] [PubMed] [Google Scholar]
  11. Dijkstra A. (2008). The psychology of tailoring-ingredients in computer-tailored persuasion. Social and personality psychology compass, 2(2), 765–784. [Google Scholar]
  12. Friestad M, & Wright P. (1994). The persuasion knowledge model: How people cope with persuasion attempts. Journal of consumer research, 21(1), 1–31. [Google Scholar]
  13. Friestad M, & Wright P. (1999). Everyday persuasion knowledge. Psychology & Marketing, 16(2), 185–194. [Google Scholar]
  14. Grimshaw J, Thomas R, MacLennan G, Fraser C, Ramsay C, Vale L, . . . Shirran L. (2004). Effectiveness and efficiency of guideline dissemination and implementation strategies. [DOI] [PubMed] [Google Scholar]
  15. Ham C-D, Nelson MR, & Das S. (2015). How to Measure Persuasion Knowledge. International Journal of Advertising, 34(1), 17–53. doi: 10.1080/02650487.2014.994730 [DOI] [Google Scholar]
  16. Haugtvedt CP, Herr PM, & Kardes FR (2018). Handbook of consumer psychology: Routledge. [Google Scholar]
  17. Haugtvedt CP, & Kasmer JA (2008). Attitude change and persuasion. [Google Scholar]
  18. Hawkins RP, Kreuter M, Resnicow K, Fishbein M, & Dijkstra A. (2008). Understanding tailoring in communicating about health. Health education research, 23(3), 454–466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hirsh JB, Kang SK, & Bodenhausen GV (2012). Personalized persuasion: Tailoring persuasive appeals to recipients’ personality traits. Psychological science, 23(6), 578581. https://www.vox.com/policy-and-politics/2017/10/16/15657512/cambridge-analytica-facebook-alexander-nix-christopher-wylie. [DOI] [PubMed] [Google Scholar]
  20. Kluger AN, & DeNisi A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological bulletin, 119(2), 254. [Google Scholar]
  21. Kosinski M, Stillwell D, & Graepel T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110(15), 5802–5805. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Leeman J, Birken SA, Powell BJ, Rohweder C, & Shea CMJIS (2017). Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice. 12(1), 125. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Liang L, Bernhardsson S, Vernooij RW, Armstrong MJ, Bussières A, Brouwers MC, . . . Group, G. I. N. I. W. (2017). Use of theory to plan or evaluate guideline implementation among physicians: a scoping review. Implementation Science, 12(1), 26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Liu-Thompkins Y. (2019). A decade of online advertising research: What we learned and what we need to know. Journal of advertising, 48(1), 1–13. [Google Scholar]
  25. Loken B. (2006). Consumer psychology: categorization, inferences, affect, and persuasion. Annu. Rev. Psychol, 57, 453–485. [DOI] [PubMed] [Google Scholar]
  26. Lomas J. J. A. o. t. N. Y. A. o. S. (1993). Diffusion, dissemination, and implementation: who should do what?, 703(1), 226–237. [DOI] [PubMed] [Google Scholar]
  27. Manojlovich M, Squires JE, Davies B, & Graham ID (2015). Hiding in plain sight: communication theory in implementation science. Implementation Science, 10(1), 58. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Matz SC, Kosinski M, Nave G, & Stillwell DJ (2017). Psychological targeting as an effective approach to digital mass persuasion. Proceedings of the National Academy of Sciences, 114(48), 12714–12719. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. McCormack L, Sheridan S, Lewis M, Boudewyns V, Melvin CL, Kistler C, . . . Lohr KN. (2013). Communication and dissemination strategies to facilitate the use of health-related evidence Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]: Centre for Reviews and Dissemination; (UK: ). [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McHugh RK, & Barlow DH (2010). The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. American Psychologist, 65(2), 73. [DOI] [PubMed] [Google Scholar]
  31. Meisel ZF, Metlay JP, Sinnenberg L, Kilaru AS, Grossestreuer A, Barg FK, . . . Perrone J. (2016). A randomized trial testing the effect of narrative vignettes versus guideline summaries on provider response to a professional organization clinical policy for safe opioid prescribing. Annals of emergency medicine, 68(6), 719–728. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Moullin JC, Ehrhart MG, & Aarons GA (2018). Development and testing of the Measure of Innovation-Specific Implementation Intentions (MISII) using Rasch measurement theory. Implementation Science, 13(1), 89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Munz KP, Jung MH, & Alter AL (2020). Name similarity encourages generosity: A field experiment in email personalization. Marketing Science. [Google Scholar]
  34. Nilsen P. (2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10(1), 53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Noar SM, Benac CN, & Harris MS (2007). Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychological bulletin, 133(4), 673. [DOI] [PubMed] [Google Scholar]
  36. Petty RE, & Cacioppo JT (1986). The elaboration likelihood model of persuasion Communication and persuasion (pp. 1–24): Springer. [Google Scholar]
  37. Petty RE, & Cacioppo JT (1996). Attitudes and persuasion: Classic and contemporary approaches: Westview Press. [Google Scholar]
  38. Petty RE, Cacioppo JT, & Schumann D. (1983). Central and peripheral routes to advertising effectiveness: The moderating role of involvement. Journal of consumer research, 10(2), 135–146. [Google Scholar]
  39. Petty RE, Haugtvedt CP, & Smith SM (1995). Elaboration as a determinant of attitude strength: Creating attitudes that are persistent, resistant, and predictive of behavior. Attitude strength: Antecedents and consequences, 4(93–130). [Google Scholar]
  40. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, . . . Kirchner JEJIS. (2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. 10(1), 21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, . . . Research, M. H. S. (2011). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. 38(2), 65–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Purtle Lê-Scherban, Wang Shattuck, Proctor, & Brownson. (2019). State Legislator Support for Behavioral Health Parity Laws: The Influence of Mutable and Fixed Factors at Multiple Levels. The Milbank Quarterly In press. Accepted May 24, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Purtle J, Dodson EA, Nelson K, Meisel ZF, & Brownson RC (2018). Legislators’ sources of behavioral Health Research and preferences for dissemination: variations by political party. Psychiatric Services, 69(10), 1105–1108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Purtle J, Lê-Scherban F, Wang X, Shattuck PT, Proctor EK, & Brownson RC (2018). Audience segmentation to disseminate behavioral health evidence to legislators: an empirical clustering analysis. Implementation Science, 13(1), 121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Purtle J, Nelson KL, Bruns EJ, Hoagwood KE, Dissemination Strategies to Accelerate the Policy Impact of Children’s Mental Health Services Research. Psychiatric Services. Accepted April 24, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Rogers EM (2010). Diffusion of innovations: Simon and Schuster. [Google Scholar]
  47. Rye M, Torres EM, Friborg O, Skre I, & Aarons GA (2017). The Evidence-based Practice Attitude Scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples. Implementation Science, 12(1), 44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Sacarny A, Barnett ML, Le J, Tetkoski F, Yokum D, & Agrawal S. (2018). Effect of peer comparison letters for high-volume primary care prescribers of quetiapine in older and disabled adults: a randomized clinical trial. JAMA psychiatry, 75(10), 1003–1011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Sahni NS, Wheeler SC, & Chintagunta P. (2018). Personalization in email marketing: The role of noninformative advertising content. Marketing Science, 37(2), 236–258. [Google Scholar]
  50. Slater MD (1996). Theory and method in health audience segmentation. Journal of health communication, 1(3), 267–284. [DOI] [PubMed] [Google Scholar]
  51. Smith RA (2017). Audience Segmentation Techniques. [Google Scholar]
  52. Stewart RE, & Chambless DL (2010). Interesting practitioners in training in empirically supported treatments: Research reviews versus case studies. Journal of Clinical Psychology, 66(1), 73–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Stewart RE, Stirman SW, & Chambless DL (2012). A qualitative investigation of practicing psychologists’ attitudes toward research-informed practice: Implications for dissemination strategies. Professional Psychology: Research and Practice, 43(2), 100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Tybout AM, & Artz N. (1994). Consumer psychology. Annual review of psychology, 45(1), 131–169. [Google Scholar]
  55. Vogt F, Hall S, Hankins M, & Marteau TM (2009). Evaluating three theory-based interventions to increase physicians’ recommendations of smoking cessation services. Health Psychology, 28(2), 174. [DOI] [PubMed] [Google Scholar]
  56. Wheeler SC, Petty RE, & Bizer GY (2005). Self-schema matching and attitude change: Situational and dispositional determinants of message elaboration. Journal of consumer research, 31(4), 787–797. [Google Scholar]
  57. Williams NJ (2015). Assessing mental health clinicians’ intentions to adopt evidence-based treatments: reliability and validity testing of the evidence-based treatment intentions scale. Implementation Science, 11(1), 60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Yoong SL, Jones J, Marshall J, Wiggers J, Seward K, Finch M, . . . Wolfenden L. (2015). A theory-based evaluation of a dissemination intervention to improve childcare cooks’ intentions to implement nutritional guidelines on their menus. Implementation Science, 11(1), 105. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material 1
NIHMS1697965-supplement-Supplemental_Material_1.pdf (136.7KB, pdf)
Supplemental Material 2
NIHMS1697965-supplement-Supplemental_Material_2.pdf (129.7KB, pdf)
Supplemental Material 3
NIHMS1697965-supplement-Supplemental_Material_3.pdf (139KB, pdf)

RESOURCES