Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Feb 1.
Published in final edited form as: Exp Clin Psychopharmacol. 2020 Mar 19;29(1):90–98. doi: 10.1037/pha0000363

Integration of Motivational Interviewing and Behavioral Economic Theories to Enhance Brief Alcohol Interventions: Rationale and Preliminary Examination of Client Language

Benjamin O Ladd 1, James G Murphy 2, Brian Borsari 3,4
PMCID: PMC7501195  NIHMSID: NIHMS1566034  PMID: 32191069

Abstract

Motivational interviewing (MI) is a widely disseminated evidence-based therapeutic approach for engaging clients and motivating health behavior change, especially risky substance use. Refinement of MI theory over the past few decades has provided empirical evidence that the technical component of MI (in-session client language) is a promising mechanism of behavior change (MOBC). However, heterogeneous and small-to-moderate effect sizes suggest the need for refinement of MOBC measurement and consideration of other types of client language. The current manuscript presents a complementary integration of current MI theory and behavioral economic (BE) mechanisms to further understanding of in-session factors associated with subsequent behavior change. In this paper, we define some of the key MOBCs from MI and BE theories, describe our integrated framework, and present preliminary findings from a pilot study of the effectiveness and MOBCs of a novel BE-informed application of MI in risky college student drinkers. Results from preliminary coding development suggest that BE-informed measures of client language better predict response to a brief intervention in risky college students than traditional change talk measures. We posit that BE theory can offer insight into meaningful session content beyond the current MI constructs of change talk and sustain talk, which in turn may serve to enhance development of clinical practice and inform scientific investigations.

Keywords: motivational interviewing, behavioral economics, client language, MOBCs


Approximately 15.1 million US adults meet criteria for alcohol use disorder, but less than 7% of these individuals received treatment in the past year (SAMHSA, 2015). This article presents an integrated framework of motivational interviewing (MI: Miller & Rollnick, 2013) process and behavioral economic (BE: Bickel, Johnson, Koffarnus, MacKillop, & Murphy, 2014) content in the service of improving understanding of dynamic change during brief interventions and informing efforts to improve intervention approaches to reach the majority of risky drinkers who do not seek treatment for their alcohol use.

Motivational Interviewing

Motivational interviewing (MI) is an efficacious strategy for reducing drug and alcohol use and related problems (e.g., Lundahl & Burke, 2009; Tanner-Smith & Lipsey, 2015). MI is defined as a “collaborative, goal-oriented style of communication with particular attention to the language of change... designed to strengthen personal motivation for and commitment to a specific goal” (Miller & Rollnick, 2013, p. 29). Over time, MI theory has focused on two complementary processes: a relational component and a technical component (Miller & Rollnick, 2013; Miller & Rose, 2009). The relational component refers to the client-centered approach of the therapeutic relationship (i.e. MI “spirit”). The technical component refers to the specific techniques a clinician uses to differentially evoke and reinforce specific types of client language. While the two components are complementary and interactive, relational mechanisms of behavior change (MOBCs) are most likely influenced by common treatment ingredients, or effective aspects of the therapeutic relationship at work in all behavioral therapies (Ahn & Wampold, 2001; Longabaugh & Magill, 2011). The technical MOBCs, on the other hand, are theoretically influenced by distinctive, or even unique active ingredients of MI.

Change Talk and Sustain Talk

The technical component of MI identifies self-motivational verbal utterances as key MOBCs. Specifically, greater Mi-consistent therapist skills should result in increased change talk (CT), or statements supporting movement toward healthy behavior change, and decreased sustain talk (ST), or statements supporting maintenance of unhealthy behavior, which in turn should lead to desired health behavior outcomes (Miller & Rose, 2009; Miller & Rollnick, 2013). Recent meta-analyses (Magill, Apodaca et al., 2018; Pace et al., 2017) have indicated reliable effects of Mi-consistent skills on rates of both client CT and ST. However, the relationship between in-session client language and subsequent behavior change is weaker and more variable (Magill & Hallgren, 2019).

There are a number of advantages to examining client language. First, client language is observable and measurable, even when the psychological constructs it may reflect (e.g., motivation, therapeutic alliance) are not directly accessible. Second, in-session behavior is immediately available to the clinician and mirrors the dynamic nature of motivation, and enhanced understanding of client language can have implications in terms of improving the therapeutic process and clinician training. Third, measures of client language represent distinctive mechanisms of the technical hypothesis of MI and should be influenced by specific intervention techniques. That said, client language definitions have been developed from a “bottom up” approach that was largely atheoretical and reliant on accumulated clinical experience (Miller & Rollnick, 2012). As a result, there is a gap between broader MI theory and the current measurement of client language MOBCs that limits the value of information obtained from current MI process research. Specifically, while in practice, skilled MI therapists may spend time during session exploring a client’s values, goals, hobbies, and interests, until such content is explicitly linked to the target behavior change, these discussions are categorized as not being clinically relevant under the current coding systems for CT and ST. Consequently, large amounts of in-session language remain unexamined and currently are coded as “follow-neutral” because they do not fit an existing category (as much as 53-67% of utterances; Borsari et al., 2015; Moyers, Martin, Houck, Christopher, & Tonigan, 2009).

Although MI theory recognizes the importance of client language related to a variety of content domains (e.g., personal values, goals, enjoyable activities), client language measurement, and most MI treatment manuals, often focus on one target health behavior change (e.g., reductions in alcohol use). For example, during a brief alcohol intervention, a college student may discuss their academic engagement, hobbies, or future goals, and that discussion ultimately may be highly relevant to their decision to reduce their drinking, but these topics are not integrated in popular treatment manuals (Dimeff et al., 1999) and would be coded as follow-neutral in the absence of the client or therapist drawing a direct link between the content and the target behavior change. CT ultimately may be the desired outcome, but the majority of individuals who receive MI, particularly in prevention or opportunistic settings, may not voice a specific desire to change their drinking even after a thorough discussion of pros and cons and feedback on drinking-related risk factors. Other than reflecting this reality and emphasizing autonomy, an MI therapist may not have a theoretical roadmap for productively extending the session with an individual who does not voice any ambivalence or desire to change a problem behavior. These limitations of MI MOBC measurement could be addressed by BE theory, which posits that focusing on enhancing alternatives and future orientation could enhance motivation to change and support planned and ongoing changes in the target behavior (McKay, 2017).

Behavioral Economic Theory

Although there are a variety of behavioral economic (BE) theories, including cognitively focused theories that leverage ubiquitous decision-making biases to understand and modify suboptimal choice behavior (Thaler & Sunstein, 2009), our focus here is on the integration of operant learning theories and microeconomics to understand how individuals allocate their time and behavior (Rachlin, 2000). Operant behavioral economic theory assumes that the value of any activity will depend on a) the benefit/cost ratio of that activity, b) the availability and benefit/cost ratio of other activities, and c) intertemporal choice situations, in which an individual must choose between outcomes that vary in amount and delay of reinforcement. Intertemporal choice dynamics are especially relevant to understanding substance use given that substance use typically results in relatively immediate reinforcement (i.e., subjective effects are experiences within seconds or minutes of ingestion) whereas behavioral alternatives to drug use (e.g., academic or health-related behaviors) may be associated with salutary outcomes that are delayed and require sustained patterns of behavior. From this perspective, substance misuse, defined as a pattern of fairly consistent preference for the substance relative to other activities, is presumed to result from ongoing patterns of interactions between endogenous (e.g., physiological response to alcohol, elevated stress) and environmental/contextual factors (e.g., low availability of alternatives, low price of the substance, social contexts reinforcing use of the substance) (Tucker et al., 2012). An event limiting access to substance-free reward (e.g., unemployment, divorce, a medical condition) will increase the relative value of substance use rewards, and frequent substance use will further impair one’s ability to obtain substance-free rewards, thus leading to a vicious cycle of increasing use (Rachlin, 2000). Similarly, increases in access to rewarding alternatives to substance use will generally decrease the relative value of substance use rewards. Indeed, levels of substance use generally show inverse relations with a variety of different alternative reinforcers (Higgins, Heil, & Lussier, 2004; Lamb & Ginsburg, 2018), and chronic substance misuse is associated with diminished ability to experience natural rewards (Lubman et al., 2009; Meshesha, Pickover, Teeters, & Murphy, 2017). Successful recovery or treatment response is associated with increased engagement in alternatives to substance use (McKay, 2017; Murphy et al., 2005, 2015; Rogers et al., 2008), and multiple efficacious treatments attempt to increase engagement in rewarding alternatives to substance use (Daughters et al., 2018; Meyers, Roozen, & Smith, 2011).

As noted above, individuals who use substances may under-engage in constructive alternatives because the benefits of these activities generally are delayed. Delayed reward discounting refers to the level of decrease in subjective value associated with reward delay. Although the value of all rewards decreases as receipt is delayed, there are substantial individual differences in the degree to which delayed rewards are discounted, and this delay-discounting phenomenon may be a core feature of substance abuse (Snider, LaConte, & Bickel, 2016). Indeed, individuals who drink heavily or use illicit drugs generally discount the value of delayed rewards more steeply than control participants (MacKillop et al., 2011), and steep delay discounting predicts poor treatment response (MacKillop & Kahler, 2009).

Rachlin (1995) argued that steep discounting of delayed rewards is due in part to whether individuals maximize utility based on the relative value of discrete acts, such as one episode of drinking vs. one episode of attending class (referred to as local utility), versus temporally extended patterns of behavior (referred to as overall utility), such as working for years to obtain a job that requires an advanced degree. In this view, obtaining the higher valued, long-term preferences entails temporally extended patterns of behavior. But the development and maintenance of such patterns may be undermined because their component acts often are relatively less valuable than alternative particular acts that are inconsistent with the pattern. Thus, a key issue becomes whether the objects of choice are perceived as discrete acts or as patterns of acts, and this is determined in part by the degree to which delayed rewards are discounted. Delay discounting is malleable and can be reduced by enhancing the salience of future rewards via experiential exercises where individuals vividly imagine future outcomes (Bickel et al., 2014; Daniel, Stanton, & Epstein, 2013) and by providing feedback that “bundles” discrete choices into coherent patterns associated with delayed outcomes (e.g., Hofmeyr, Ainslie, Charlton, & Ross, 2011; Monterosso & Ainslie, 1999).

Therapeutic process research on BE MOBCs is notably absent. This is due primarily to the fact that operant behavioral economic research generally focuses on modifying observable behaviors (e.g., drug use) via the manipulation of direct contingencies (e.g., providing incentives for verified abstinence or engagement in treatment consistent behaviors) as is the case in efficacious contingency management interventions (Higgins et al., 2004; Meyers et al., 2011). Consequently, BE elements have not been integrated into MI theory, despite strong potential for theoretical complementarity and improved clinical outcomes. Indeed, two studies have observed promising results for a brief MI designed to increase engagement in patterns of substance-free activity associated with delayed rewards (e.g., exercising, attending class) (Murphy et al., 2012, 2019). Extending BE theory to brief and inexpensive verbally-mediated intervention approaches that target non-treatment seeking heavy drinkers could expand the public health impact of BE approaches, although further research is needed to establish the association between in session verbal behaviors and subsequent changes in drinking.

Integration of MI (Process) and BE (Content)

Independently, MI and BE have provided valuable contributions to the understanding of substance use and problems. We posit that an integration of MI and BE approaches and mechanisms could serve to capitalize on the advantages of each while addressing many of the limitations to the current state of client language measurement. MI and BE both have traditional behavioral underpinnings, particularly the importance of principles of reinforcement. For MI, the technical hypothesis is based on evocation and differential reinforcement of certain types of client language over others. BE emphasizes enhancing the value of competing alternative sources of reinforcement and altering evaluation of reinforcing value of various behaviors based on temporal frame. We believe an explicit union of MI and BE can provide an important next step in advancing provision of clinical services designed to promote health behavior change and represents a natural extension of MI process research and BE theory.

MI has identified four processes by which health providers can engage individuals and create an environment for the exploration of behavior change: engaging, focusing, evoking, and planning (Miller & Rollnick, 2013). One of the most effective ways to evoke change language is to evoke personal values and goals that are inconsistent with current behavior. This strategy is highly compatible with, or even implies, the explicit focus of BE on examining behavior in terms of temporally extended molar patterns, versus a more molecular approach of attempting to understand efficient (i.e. immediate proximal) causes for each behavior act (Vuchinich & Heather, 2003). An MI session may help an individual understand the costs and benefits of drinking over time, including how drinking relates to important life values, often formally addressed using the personal values card sort task (Wagner & Sanchez, 2002), and how it will impact the future (i.e. envisioning the future) versus thinking about each drinking episode in isolation. Personalized feedback on drinking patterns is commonly included in MI sessions and may facilitate this as well by aggregating individual decisions about drinking into patterns that can be understood in terms of personal or financial costs (e.g., calories consumed from alcohol, money spent on drinking). Recently, brief motivational interventions (BMIs) have incorporated a more BE approach by providing feedback on time allocation to drinking compared to other valuable activities (e.g., exercise, academics, family), which may provide an objective index of valuation, according with their personal values and long-term goals (Colby et al., 2018; Murphy et al., 2012, 2019). This feedback often stimulates a discussion of the impact of drinking on other life areas, and how time spent drinking might be re-allocated to other goals and values-consistent substance-free activities.

An integration of MI and BE might improve the effectiveness and efficiency of motivating health behavior change, especially during the evocation and planning stages of MI. Currently, MI skills are designed to promote a therapeutic environment wherein idiosyncratic motivation and alternative behaviors are evoked and explored; however, there are often commonalities across individuals depending on the clinical population and target health behavior which can be capitalized upon to facilitate the process of honing in on useful content for a given individual. BE offers explicit definitions for identifying these commonalities. For example, in college students, alcohol use and studying are generally competing reinforcers, given that heavy drinking is associated with lower academic performance, and thus academic engagement could be a go-to topic during a brief alcohol intervention. More generally, for non-student populations substance-free activities like exercise, religious activity, volunteer service, and hobbies are generally inversely associated with alcohol and drug use (Acuff, Dennhardt, Correia, & Murphy, 2019), and can be targeted even among individuals who are not motivated to change their drinking (Murphy et al., 2019). In other words, MI is the communication style that facilitates focused engagement on target behavior in an empathic context, while BE provides a theoretical framework for identifying specific session content, beyond direct discussions of personal drinking (which generate change and sustain talk), that may enhance the likelihood of sustained changes in alcohol use. The explicit BE emphasis on increasing substance-free alternative reinforcement and re-evaluation of immediate and future value extends current MI mechanism measurement by providing additional content domains beyond the discussion of ambivalence about personal substance use which may in turn expand the opportunistic settings in which clinical services designed to reduce alcohol use could be used effectively. Indeed, young adult heavy drinkers report very little interest in participating in existing alcohol-focused brief intervention sessions (Buscemi et al., 2010). Relatedly, current MI MOBC measurement does not differentiate language related to these alternative ways of experiencing reward. For example, while a skilled MI therapist may evoke valued non-drinking activities or goals, this content would not be categorized as change talk or sustain talk. This is a concern as developing alternative ways of experiencing reward is associated with long-term recovery (Laudet & White, 2008; Tucker et al., 2002), and increases in proportional reinforcement from substance-free activities has been shown to partially mediate the treatment gains associated with BMIs (Murphy et al., 2019).

BE also offers explicit alternative behaviors to address with clients who may be resistant to changing their drinking (consistent with the MI strategy of shifting focus). If the session leads to increased engagement in substance-free activities, even if there is no expressed increase in motivation to reduce drinking, this plan may indirectly reduce drinking given that there is considerable evidence that increasing engagement in substance-free activities is associated with reductions in drinking (Higgins et al 2004; Murphy & Dennhardt, 2016). For example, Correia and colleagues (2005) found that participants who increased their exercise or creative activities spontaneously reduced their drinking compared to control participants. And brief experiential exercises that encourage individuals to think about a positive future event (episodic future thinking) can enhance future orientation and reduce addictive and health risk behavior even without an explicit focus on substance use (Daniel et al., 2013; Bulley & Gullo, 2017). Therefore, client language associated with alternatives to drug use and with future goals/activities also should be reflected and captured as a positive proximal outcome in MI process research.

Although BE theory has informed MI and other brief intervention approaches, with promising preliminary results, no research to date has fully integrated MI practice and BE theory by examining the associations between in-session BE content and client language associated with clinically relevant change. Thus, an important step in establishing BE factors as active MOBCs is to develop and test dynamic in-session measures informed by BE theory during MI interventions.

Preliminary Efficacy and Mechanisms of an Applied MI+BE Integration

The Substance-Free Activity Session (SFAS) is a single-session intervention that supplements a standard alcohol or drug-focused MI session (Murphy et al., 2012, 2019). The SFAS delivers personalized feedback in an MI style in order to evoke discussions about BE mechanisms of substance-free reinforcement, delayed reward discounting, and enhance the perceived costs of drinking (e.g., time allocated to drinking, impact of drinking on future outcomes).

A pilot trial evaluated the incremental efficacy of the SFAS added to a standard BMI (BMI+ SFAS), compared to a BMI plus Relaxation control condition (Murphy et al., 2012). First-year college students were identified from classroom screenings and were not seeking treatment. Six-month follow-up results (94% follow-up rate) indicated the alcohol BMI+SFAS intervention resulted in larger reductions in heavy drinking (d= .44) and alcohol problems (d= .52) relative to the active control condition. Moderation analyses indicated that BMI+SFAS was also associated with significantly greater reductions in binge drinking among participants who at baseline reported low levels of substance-free reinforcement. These participants may have benefited from the focus on increasing goal-directed substance-free activities. The SFAS was also associated with significant increases in hypothesized MOBCs, including future time orientation, self-regulation, and participation in substance-free activities (Murphy et al., 2012; Soltis et al., 2018).

Measurement of MI+BE In-session MOBCs

To test in-session mechanisms, our research team has begun developing novel client language definitions consistent with our integrated MI+BE approach using a subset of alcohol BMI and SFAS session tapes from the pilot trial described previously (Murphy et al., 2012). Both the original treatment trial and the coding study received approval from the appropriate Institutional Review Boards (WSU IRB protocol #16231). Half (n= 10) of the pilot sample was selected a priori based on their status as high treatment responders (1+ SD decrease in baseline alcohol-related problems at 1- and 6-month follow ups) and the other half (n= 10) were randomly selected from individuals not categorized as high-responders. Coders were blind to outcome status. This study had two aims: 1) determine whether integration of MI and BE mechanisms would result in greater rates of client language hypothesized to be clinically relevant, and 2) establish preliminary predictive validity of a coding system informed by MI and BE frameworks.

To achieve these aims, we adapted the current Motivational Interviewing Skill Code 2.5 (MISC: Houck, Moyers, Miller, Glynn, & Hallgren, 2010) system and made two major adaptations, resulting in the MISC-BE coding system. Figure 1 provides a tentative framework for the new MISC-BE system compared to the coding structure of the existing MISC system. First, we developed definitions of client statements to assess a range of behavioral content. We defined four domains of client language guided by BE theory: a) target behavior pre-specified by the investigators (e.g., alcohol use), b) alternative reinforcers, or behaviors theorized to serve as economic substitutes to the target behavior (e.g., studying, participating in volunteer/service activities), c) independent reinforcers, or utterances that explicitly include discussion of a behavioral goal but is unclear if it serves as a competing reinforcer to the target behavior (e.g., making new friends, which could involve more or less drinking), and d) non-relevant content, or statements that do not refer to behavioral allocation/goals. For the alternative reinforcer code, certain behaviors were identified a priori as being universal alternatives given the study sample (risky college drinkers) and intervention components. These included statements pertaining to: physical activity/exercise, diet/weight management, and engaging in academic activities (studying, attending class, getting good grades, employment opportunities as a result of academic performance/success). In addition to these pre-established alternative reinforcers, the coding system was designed to reflect that economic associations differ across individuals. Thus, coders could flexibly rate additional statements as alternatives based on the context establishing the behavior being discussed as a substitute or competing reinforce to alcohol (e.g., if a participant indicates their family disapproves of drinking, then spending time with family could be rated as an alternative).

Figure 1. Comparison of MISC 2.5 codes and proposed MISC-BE coding system.

Figure 1.

*The MISC does offer the option to further categorize change and sustain talk into subcategories. Under the MISC-BE, the non-health/ambiguous code will serve a similar function to the MISC follow/neutral code as the catchall for clinically non-relevant client language, albeit with a narrower scope.

The second adaptation of the MISC 2.5 was the incorporation of definitions of temporal orientation consistent with BE theory and designed to capture the temporal nature in which the value/reward of a behavior is earned/received (i.e. past, present, or future). As noted earlier, devaluing or discounting future rewards in favor of the immediate reward associated with activities like substance use is a primary behavioral economic risk factor for drug and alcohol misuse (Bickel et al., 2014). To reduce coder subjectivity, as well as minimize dilution of future-oriented statements, coders were trained to use “present” as the default code, and to code statements as “past” or “future” only if those timeframes are clearly specified and statements do not also include present orientation (i.e. statements about change in general would be considered “present” as that could apply to the current situation as well as the future). Consistent with the current measures of CT and ST, client utterances were rated based on motivational state being discussed. However, because the desired direction of the motivational state changes based on the behavioral content (i.e. avoid target, but approach alternative), we used labels of “approach” and “avoid” rather than “change” and “sustain”. These adaptations resulted in a mutually exclusive parallel decision-making process wherein coders assign one code for each of the three areas (behavioral target, motivational state, and temporal orientation). Table 1 provides example utterances of certain MISC-BE codes. It is important to note that different approaches to quantifying client language (e.g., strength, frequency) have been tested; as the current adaptation emphasized novel content areas rather than different measures of the same underlying variable definitions, we opted to use a frequency-based approach which has more consistent empirical support currently (e.g., Houck et al., 2010).

Table 1.

Participant utterance examples and corresponding MISC-BE codes.

Client Statement Behavioral Target Motivational State Temporal Orientation
I don’t think drinking will affect where I end up in terms of a job. Target Approach Future
I want to cut down my drinking to only weekends. Target Avoid Current
Studying isn’t the most enjoyable or exciting thing, but in the long run obviously it’s gonna pay off. Alternative Approach Future
I’ve been really into playing basketball recently. It’s another way that I bond with my friends and I always feel good after. Alternative Approach Present
In high school, I found studying to be really boring. Alternative Avoid Past

MISC-BE coding procedures.

Two research assistants first were oriented to the relevant literature on MI, BE, and client language and reviewed the MISC 2.5 manual as a team. Next coders practiced parsing sessions into codeable units using the CASAA Application for Coding Treatment Interactions (CACTI: Glynn et al., 2012) and compared their parses to master files. Finally, coders assigned ratings independently and then compared their ratings to previously-established master codes during regular coder team meetings until acceptable reliability, defined as single-measures absolute-agreement intraclass correlations (ICCs) of 0.6 or greater (Cicchetti, 1994) was achieved. The same training strategy was used for the MISC-BE system, with the exception that as a novel coding system, master codes did not exist. Instead, differences in assigned codes were discussed with both coders and a senior coder as a means of achieving consensus. After acceptable reliability was demonstrated on the MISC-BE, coders rated 20 sessions (10 BMI, 10 SFAS, different participants for each session) using both coding systems. Ten sessions were randomly selected and double-coded for reliability purposes; coders were blind to reliability status.

Analytic strategy.

We identified two primary variables of interest for our pilot test, Avoid Target and Approach Alternative, as these variables were hypothesized to act similarly to conventional CT, with Avoid Target largely overlapping with CT, and Approach Alternative representing a new content code also associated with positive outcomes. Additionally, we calculated the amount of session content rated as follow/neutral across the two coding systems. For all analyses, frequencies of codes were divided by total number of client utterances to control for client verbosity and session length.

To test whether our novel coding system captured more clinically-relevant language than the MISC 2.5, paired t-tests were planned comparing rates of MISC CT and MISC-BE CT (Avoid Target + Approach Alternative). Rates of follow-neutral language were compared in the same manner. We also planned to examine correlations between MISC CT and each of the two categories of MISC-BE CT. To examine the predictive validity of the novel coding system, independent t-tests were conducted comparing rates of client languages based on treatment response status (high vs. other). We also examined whether client language differed based on session format (i.e. BMI vs. SFAS).

Coding Results

Coders demonstrated acceptable interrater reliability under both coding systems (Table 2). As expected, participants provided greater rates of MISC-BE CT compared to MISC CT, with a mean difference of 8.2% of total utterances (SD = 15.5%, t(19)= 2.38, p = .028, Cohen’s d = .54) (Table 2). An even greater difference was observed for follow-neutral language, with participants providing an average of 36.7% (SD = 22.7%) fewer follow/neutral utterances using the MISC-BE relative to the MISC (t(19)= 7.24, p < .001, d = 1.62). Avoid Target and Approach Alternative codes were significantly associated with MISC CT, but in opposite directions (r = .940, p< .001 and r = −.824, p< .001, respectively). In other words, sessions containing more traditional MISC CT also contained greater amounts of MISC-BE Avoid Target, while sessions with lower rates of MISC CT contained greater rates of MISC-BE Approach Target (likely reflecting a shift of language lumped into the MISC follow-neutral category into clinically relevant categories under the MISC-BE).

Table 2.

Rates of client language using the MISC and MISC-BE coding systems.

Code Total (n=20) ICC BMI (n=10) SFAS (n=10) t P


M% SD M% SD M% SD
MISC Change Talk 24.6 16.3 0.979 38.9 8.0 10.3 6.3 8.89 <.001
MISC Follow-Neutral 65.9 23.2 0.977 44.9 10.5 86.9 6.7 −10.69 <.001

MISC-BE Change Talk 32.9 9.0 0.650 35.1 7.6 30.1 10.1 1.10 0.285
Avoid Target* 18.1 15.9 0.962 32.1 9.1 4.0 3.1 9.26 <.001
Approach Alternative* 14.8 14.2 0.865 2.9 4.0 26.7 9.7 −7.12 <.001
MISC-BE Follow-Neutral 29.2 13.1 0.854 24.3 11.3 34.1 13.5 −1.77 0.093

Note: Means are reported as percentages of the total number of client utterances. Single-measures, absolute-agreement intraclass correlations (ICCs) are reported. t values indicate mean comparisons between the BMI and SFAS sessions.

*

t tests adjusted due to unequal variances based on significant Levene’s Tests.

MISC = Motivational Interviewing Skill Code, MISC-BE = Motivational Interviewing Skill Code-Behavioral Economics version, BMI = brief motivational interview, SFAS = substance-free activity session.

In terms of the effects of in-session language on treatment response, high responders offered greater rates of MISC-BE CT (Avoid Target + Approach Alternative) during the session, (t(18) = 2.86, p = .01, d = 1.28), but did not differ from the non-high responder group on conventional MISC CT (t(18) = 0.39, p = .70, d = .17). Not surprisingly, greater rates of MISC CT were observed in the BMI sessions compared to the SFAS sessions (Table 2). Overall, MISC-BE CT did not differ based on session type; however significant differences did emerge in the expected directions when considering Avoid Target and Approach Alternative codes separately.

Discussion

The explicit integration of MI and BE theories is intended to significantly improve MOBC measurement and clinical practice. Indeed, MI theory already incorporates a number of BE mechanisms; however, this overlap is largely implied rather than explicit and thus may not consistently manifest in actual MI practice. Thus, this integration has the potential to extend current MI practice. For example, if the impact of a target behavior change on personal values or goals (e.g., how reducing alcohol use could improve academic achievement) is not explicitly stated during a clinical interaction, this might be attributed to absence of MI skill in the session and theorized not to result in desired health behavior change. BE theory posits that a discussion that leads to an increase in engagement in substance-free activities may have an indirect effect of reducing substance use. Similarly, discussions that link current patterns of behavior with future outcomes may reduce delay discounting and thus have salutary effects on drug use even in the absence of a direct discussion of alcohol or drugs.

Relatedly, a fair amount of in-session content informed by broader MI theory is not being captured using current definitions of the key client language measures of CT and ST. In fact, some researchers have raised concerns that this gap between MI theory and MI process research has led to a premature focus trap (Hilton, Lane, & Johnston, 2016). The union of MI and BE mechanisms may help bridge this gap. While MI theory may guide non-substance discussions, currently while attempting to measure relevant client language utterances (i.e. CT and ST) “[c]oders should not infer a link between actions being discussed by the client and the TBC [target behavior change], unless it is clear from the context that the purpose of the behavior is to move toward or away from the TBC goal” (Houck et al., 2013, p. 38). Thus, this may restrict amount of in-session behavior considered to be clinically relevant, and may even result in categorization of discussions consistent with current MI theory as non-relevant. The proposed client language measures integrating MI and BE theories allows researchers to capture a wider range of clinically-relevant language as an explicit link to the target behavior change is not required, widening the potential contexts for intervention around risky alcohol use (e.g., routine healthcare, discussions with academic advisors). The preliminary data presented suggest the promise of using a behavioral economic approach to code MI sessions, and we wish to highlight several findings of note.

First, coders can be trained to reliably differentiate MISC-BE client language constructs. Notably, although for this pilot investigation training of coders co-occurred with development of the coding definitions, the amount of training required does not appear to be much greater than the traditional MISC 2.5 system based on recent training of a new coder on the MISC-BE system (training took approximately 40 hours). Second, significantly greater amounts of clinically-relevant client language were captured using the MISC-BE compared to the MISC 2.5 as demonstrated by the lower rates of follow-neutral language under the MISC-BE. Third, differences in rates of MISC-BE language emerged based on session content consistent with the MOBCs emphasized in the BMI versus SFAS sessions. Finally, MISC-BE change language was associated with subsequent reductions in alcohol-related problems while MISC 2.5 change talk did not differentiate between treatment responders and non-responders, providing preliminary support for the incremental predictive utility of the proposed coding system and, more generally, for MI approaches that target BE MOBCs. Additional exploration of BE-related change language is warranted, especially as current definitions of change talk do not consistently predict change (Magill, Bernstein et al., 2014; Magill et al., 2018; Pace et al., 2017). This supports the theorized mechanisms driving treatment response and suggests a promising avenue for further testing of MI+BE process and interventions (for example, experimental dismantling tests of brief interventions differentially emphasizing some MOBCs over others).

Given the small sample size of this pilot study, the results should be interpreted with caution. For example, a dichotomous treatment response outcome was used to maximize detection of an effect despite alcohol use often being measured as a continuous variable, and greater exploration of this novel coding system is needed. Taking such limitations into account, these initial MISC-BE findings demonstrate preliminary reliability, capture greater rates of clinically-relevant client language compared to the MISC 2.5, and suggest greater predictive validity than conventional CT alone. If these findings can be replicated and extended, the integration of MI and BE could serve to advance MOBC investigation and improve clinical training and practice in several ways. First, as clinicians can be trained to elicit and deepen client change language (e.g., Moyers, Houck, Glynn, Hallgren, & Manuel, 2017), it is likely that clinicians can be trained to selectively emphasize BE concepts during an MI session, especially during the evoking and planning processes. Personalized feedback provided during the session could facilitate this process, as has been found in alcohol-focused sessions (Amrhein et al.,2003). Second, the BE constructs provide a valuable alternative for clinicians who are working with clients who are resistant to changing the target behavior, as alternative substance-free activities or future goals could be mutually determined to be an appropriate focus of the session. This would be consistent with evidence that post-treatment changes in client reports of the proportion of their activity participation and enjoyment related to substance-related outcomes, relative to total activity participation and enjoyment mediates treatment response (Murphy et al., 2019). Third, if BE-focused language is determined to be predictive in subsequent work, there is a wealth of existing MI session data that could be mined to clarify precisely how BE-related change talk can facilitate changes in subsequent behaviors. To do so would require the follow-neutral codes of existing datasets to be re-examined and related to observed outcomes. As machine coding of MI sessions becomes more feasible and utilized in MI process research, this strategy will be possible without onerous human coding procedures (Hallgren et al., 2018).

In sum, MI has shown considerable promise as an empirically-supported treatment that can be widely disseminated. However, MI process research has not consistently identified the in-session client language that predicts change. BE provides framework for identifying multiple clinically-relevant content domains that, when evoked and incorporated into planning, may clarify which language is an appropriate focus for the clinician to deepen through the use of MI consistent techniques. Further examination and refinement of such an approach for understanding in-session MOBCs is necessary, although the preliminary approach and findings described in this study suggest the value of an integrated MI and BE framework.

Public Significance Statement:

This review presents background and rationale for the integration of motivational interviewing and behavioral economic mechanisms of change. Findings from a pilot study suggest adapting and expanding current measures of change talk can improve prediction of health behavior change in college student drinkers from in-session language during a brief alcohol intervention.

Disclosures and Acknowledgments

Funding: This work was supported by the National Institute on Alcohol Abuse and Alcoholism (R21 AA016304) and by funds provided for medical and biological research by the State of Washington Initiative Measure No. 171. Funding sources had no roles other than financial support.

Footnotes

Conflict of Interest Statement: No conflicts of interest exist for any author of this manuscript that could be interpreted as influential in this research.

References

  1. Acuff SF, Dennhardt AA, Correia C, & Murphy JG (2019). Measurement of substance-free reinforcement in addiction: A systematic review. Clinical Psychology Review, 70, 79–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ahn HN & Wampold BE (2001). Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. Journal of Counseling Psychology, 48(3), 251–257. [Google Scholar]
  3. Amrhein PC, Miller WR, Yahne CE, Palmer M, & Fulcher L (2003). Client commitment language during motivational interviewing predicts drug use outcomes. Journal of Consulting and Clinical Psychology, 71, 862–878. [DOI] [PubMed] [Google Scholar]
  4. Bem DJ (1972). Self-perception theory. In L. Berkowitz (Ed.), Advances in Experimental Social Psychology, Vol. 6 (pp 1–62). New York: Academic Press. [Google Scholar]
  5. Bickel W, Johnson M, Koffarnus M MacKillop J, Murphy JG. (2014). The behavioral economics of substance use disorders: Reinforcement pathologies and their repair. Annual Review of Clinical Psychology, 10, 641–677. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Borsari B, Apodaca TR, Jackson KM, Mastroleo NR, Magill M, Barnett NP, & Carey KB (2015). In-session processes of brief motivational interventions in two trials with mandated college students. Journal of Consulting and Clinical Psychology, 83, 56–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Buscemi J, Murphy JG, Martens MP, McDevitt-Murphy ME, Dennhardt AA, & Skidmore JR (2010). Help-seeking for alcohol-related problems in college students: Correlates and preferred resources. Psychology of Addictive Behaviors, 24, 571–580. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bulley A, & Gullo MJ (2017). The influence of episodic foresight on delay discounting and demand for alcohol. Addictive Behaviors, 66, 1–6. [DOI] [PubMed] [Google Scholar]
  9. Cicchetti DV. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. [Google Scholar]
  10. Colby SM, Orchowski L, Magill M, Murphy JG, Brazil LA, Apodaca & Barnett NP. (2018). Brief motivational intervention for underage young adult drinkers; Results from a randomized clinical trial. Alcoholism: Clinical & Experimental Research, 42, 1342–1351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Correia CJ, Benson TA, & Carey KB (2005). Decreased substance use following increases in alternative behaviors: A preliminary investigation. Addictive Behaviors, 30, 19–27. [DOI] [PubMed] [Google Scholar]
  12. Daniel TO, Stanton CM, & Epstein LH (2013). The future is now: Reducing impulsivity and energy intake using episodic future thinking. Psychological Science, 24, 2339–2342. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Daughters SB, Magidson JF, Anand D, Seitz-Brown CJ, Chen Y, & Baker S (2018). The effect of a behavioral activation treatment for substance use on post-treatment abstinence: a randomized controlled trial. Addiction, 113, 535–544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Dimeff LA, Baer JS, Kivlahan DR, & Marlatt GA (1999). Brief alcohol screening and intervention for college students: A harm reduction approach. New York: Guilford Press. [Google Scholar]
  15. Ellis A. (1976). Reason and emotion in psychotherapy. New York: Lyle Stuart. [Google Scholar]
  16. Glynn LH, Hallgren KA, Houck JM, & Moyers TB (2012). CACTI: Free, open-source software for the sequential coding of behavioral interactions. PLoS ONE, 7, e39740. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hallgren KA, Dembe A, Pace BT, Imel ZE, Lee CM, & Atkins DC (2018). Variability in motivational interviewing adherence across sessions, providers, sites, and research contexts. Journal of Substance Abuse Treatment, 84, 30–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hettema J, Steele J, & Miller WR (2005). Motivational interviewing. Annual Review of Clinical Psychology, 1, 91–111. [DOI] [PubMed] [Google Scholar]
  19. Higgins ST, Heil SH, & Lussier JP (2004). Clinical implications of reinforcement as a determinant of substance use disorders. Annual Review of Psychology, 55, 431–461. [DOI] [PubMed] [Google Scholar]
  20. Hilton CE, Lane C, & Johnston LH (2016). Has motivational interviewing fallen into its own premature focus trap? International Journal for the Advancement of Counselling, 38, 145–158. [Google Scholar]
  21. Hofmeyr A, Ainslie G, Charlton R, & Ross D, (2011). The relationship between addiction and reward bundling: An experiment comparing smokers and non-smokers. Addiction, 106, 402–409. [DOI] [PubMed] [Google Scholar]
  22. Houck JM, Moyers TB, Miller WR, Glynn LH, & Hallgren KA (2010). Motivational Interviewing Skill Code (MISC) version 2.5. Retrieved from http://casaa.unm.edu/download/misc25.pdf. [Google Scholar]
  23. Huh D, Mun EY, Larimer ME, White HR, Ray AE, Rhew IC &Atkins DC. (2015). Brief motivational interventions for college student drinking may not be as powerful as we think: An individual participant-level data meta-analysis. Alcoholism: Clinical and Experimental Research, 39(5), 919–931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lamb RJ, & Ginsburg BC (2018). Addiction as a BAD, a behavioral allocation disorder. Pharmacology Biochemistry and Behavior, 164, 62–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Laudet AB & White WL (2008). Recovery capital as prospective predictor of sustained recovery, life satisfaction, and stress among former poly-substance users. Substance Use & Misuse, 43, 27–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Longabaugh R & Magill M (2011). Recent advances in behavioral addictions treatment: Focusing on mechanisms of change. Current Psychiatry Reports, 13, 382–389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Lubman DI, Yucel M, Kettle JW, et al. (2009). Responsiveness to drug cues and natural rewards in opiate addiction: associations with later heroin use. Archives of General Psychiatry, 66, 205–12. [DOI] [PubMed] [Google Scholar]
  28. Lundahl B & Burke BL (2009). The effectiveness and applicability of motivational interviewing: A practice-friendly review of four meta-analyses. Journal of Clinical Psychology, 65, 1232–1245. [DOI] [PubMed] [Google Scholar]
  29. Magill M, Apodaca TR, Borsari B, Gaume J, Hoadley A, Gordon REF &Moyers T. (2018). A meta-analysis of motivational interviewing process: Technical, relational, and conditional process models of change. Journal of Consulting and Clinical Psychology, 86, 140–157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Magill M, Bernstein MH, Hoadley A, Borsari B, Apodaca TR, Gaume J, & Tonigan JS (2018). Do what you say and say what you are going to do: A preliminary meta-analysis of client change and sustain talk subtypes in motivational interviewing. Psychotherapy Research. Advance online publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Magill M, Gaume J, Apodaca TR, Walthers J, Mastroleo NR, Borsari B, & Longabaugh R (2014). The technical hypothesis of motivational interviewing: A meta-analysis of MI’s key causal model. Journal of Consulting and Clinical Psychology, 82, 973–983. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Magill M & Hallgren KA. (2019). Mechanisms of behavior change in motivational interviewing: Do we understand how MI works? Current Opinion in Psychology, 30, 1–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Magill M, Kiluk BD, McCrady BS, Tonigan JS, & Longabaugh R (2015). Active ingredients of treatment and client mechanisms of change in behavioral treatments for alcohol use disorders: Progress 10 years later. Alcoholism: Clinical and Experimental Research, 39, 1852–1862. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, & Munafò MR (2011). Delayed reward discounting and addictive behavior: A meta-analysis. Psychopharmacology, 216, 305–321. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. MacKillop J & Kahler CW (2009). Delayed reward discounting predicts treatment response for heavy drinkers receiving smoking cessation treatment. Drug and Alcohol Dependence, 104, 197–203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. McKay JR (2017). Making the hard work of recovery more attractive for those with substance use disorders. Addiction, 112, 751–757. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Meichenbaum D (1977). Cognitive-behaviour modification: An integrative approach. New York: Plenum Press. [Google Scholar]
  38. Meshesha LZ, Pickover AM, Teeters JB, & Murphy JG (2017). Substance-free reward deficits among young adult non-medical prescription opioid users. The Psychological Record, 67, 241–251. [Google Scholar]
  39. Meyers RJ, Roozen HG, Smith JE (2011). The community reinforcement approach: an update of the evidence. Alcohol Research and Health, 33, 380–388. [PMC free article] [PubMed] [Google Scholar]
  40. Miller W, & Rollnick S (1991). Motivational interviewing: Preparing people for change. New York: Guilford Press. [Google Scholar]
  41. Miller WR & Rollnick S (2009). Ten things that motivational interviewing is not. Behavioural and Cognitive Psychotherapy, 37, 129–140. [DOI] [PubMed] [Google Scholar]
  42. Miller WR & Rollnick S (2012). Meeting in the middle: Motivational interviewing and selfdetermination theory. International Journal of Behavioral Nutrition and Physical Activity, 9, 25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Miller WR & Rollnick S (2013). Motivational interviewing: Preparing people for change, 3rd ed. New York: Guilford. [Google Scholar]
  44. Miller WR & Rose GS (2009). Toward a theory of motivational interviewing. American Psychologist, 64, 527–537. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Monterosso J, & Ainslie G (1999). Beyond discounting possible experimental models of impulse control. Psychopharmacology, 146, 339–347. [DOI] [PubMed] [Google Scholar]
  46. Moyers TB, Houck J, Glynn LH, Hallgren KA, & Manuel JK (2017). A randomized controlled trial to influence client language in substance use disorder treatment. Drug & Alcohol Dependence, 172, 43–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Moyers TB, Martin T, Houck JM, Christopher PJ, & Tonigan JS (2009). From in-session behaviors to drinking outcomes: A causal chain for motivational interviewing. Journal of Consulting and Clinical Psychology, 77, 1113–1124. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Murphy JG, Correia CJ, Colby SM, & Vuchinich RE (2005). Using behavioral theories of choice to predict drinking outcomes following a brief intervention. Experimental and Clinical Psychopharmacology, 13, 93–101. [DOI] [PubMed] [Google Scholar]
  49. Murphy JG & Dennhardt AA (2016). The behavioral economics of young adult substance abuse. Preventive Medicine, 92, 24–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Murphy JG, Dennhardt AA, Martens MP, Borsari B, Witkiewitz K, & Meshesha LZ (2019). A randomized clinical trial evaluating the efficacy of a brief alcohol intervention supplemented with a substance-free activity session or relaxation training. Journal of Consulting and Clinical Psychology, 87(7), 657–669. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Murphy JG, Dennhardt AA, Skidmore JR, Borsari B, Barnett NP, Colby SM, & Martens MP (2012). A randomized controlled trial of a behavioral economic supplement to brief motivational interventions for college drinking. Journal of Consulting and Clinical Psychology, 80, 876–886. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Murphy JG, Dennhardt AA, Yurasek AM, Skidmore JR, Martens MP, MacKillop J, & McDevitt-Murphy ME (2015). Behavioral economic predictors of brief alcohol intervention outcomes. Journal of Consulting and Clinical Psychology, 83(6), 1033–1043. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Pace BT, Dembe A, Soma CS, Baldwin SA, Atkins DC, & Imel ZE (2017). A multivariate meta-analysis of motivational interviewing process and outcome. Psychology of Addictive Behaviors, 31, 524–533. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Rachlin H (2000). The lonely addict. In: Bickel WK and Vuchinich RE (eds), Reframing Health Behavior Change with Behavioral Economics. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 145–65. [Google Scholar]
  55. Rogers RE, Higgins ST, Silverman K, Thomas CS, Badger GJ, Bigelow G, & Stitzer M (2008). Abstinence-contingent reinforcement and engagement in non-drug-related activities among illicit drug abusers. Psychology of Addictive Behaviors, 22, 544–550. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Snider SE, LaConte SM, & Bickel WK (2016). Episodic future thinking: Expansion of the temporal window in individuals with alcohol dependence. Alcoholism: Clinical & Experimental Research, 40, 1558–1566. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Soltis KE, Acuff SF, Dennhardt AA, Borsari B, Martens MP, & Murphy JG (2018). Self-regulation mediates the effects of a brief behavioral economic intervention on alcohol-related outcomes: A preliminary analysis. Experimental and Clinical Psychopharmacology, 26, 347–353. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Substance Abuse and Mental Health Services Administration (SAMHSA). (2015). National Survey on Drug Use and Health (NSDUH) 2015 (NSDUH-2015-DS0001) Public-Use File Dataset. Available at: https://www.datafiles.samhsa.gov/study-dataset/national-survey-drug-use-and-health-2015-nsduh-2015-ds0001-nid16894. Accessed 6/15/17.
  59. Tanner-Smith EE, & Lipsey MW (2015). Brief alcohol interventions for adolescents and young adults: A systematic review and meta-analysis. Journal of Substance Abuse Treatment, 51, 1–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Thaler RH, & Sunstein CR (2009). Nudge: improving decisions about health, wealth, and happiness. Rev. and expanded ed New York: Penguin Books. [Google Scholar]
  61. Tucker JA, Vuchinich RE, & Rippens PD (2002). Predicting natural resolution of alcohol- related problems: A prospective behavioral economic analysis. Experimental and Clinical Psychopharmacology, 10(3), 248–257. [DOI] [PubMed] [Google Scholar]
  62. Tucker J, Roth D, Huang J, Crawford M, & Simpson C (2012). Effects of interactive voice response self-monitoring on natural resolution of drinking problems: utilization and behavioral economic factors. Journal of Studies in Alcohol and Drugs, 73, 686–698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Vuchinich RE, & Heather N (2003). Choice, behavioural economics and addiction. Elsevier. [Google Scholar]
  64. Wagner C, Sanchez F (2002). The role of values in motivational interviewing. In: Miller WR, Rollnick S, editors. Motivational Interviewing: Preparing People to Change. New York: Guildford Press. pp. 284–98. [Google Scholar]
  65. Yurasek AM, Dennhardt AA, & Murphy JG (2015). A randomized controlled trial of a behavioral economic intervention for alcohol and marijuana use. Experimental and Clinical Psychopharmacology, 23 (5), 324–331. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES