Advancing Analytic Approaches to Address Key Questions in Mechanisms of Behavior Change Research

Abstract

Objective:

Interest in studying mechanisms of behavior change (MOBCs) in substance use disorder (SUD) treatments has grown considerably in the past two decades. Much of this work has focused on identifying which variables statistically mediate the effect of SUD treatments on clinical outcomes. However, a fuller conceptualization of MOBCs will require greater understanding of questions that extend beyond traditional mediation analysis, including better understanding of when MOBCs change during treatment, when they are most critical to aiding the initiation or maintenance of change, and how MOBCs themselves arise as a function of treatment processes.

Method:

In the present study, we review why these MOBC-related questions are often minimally addressed in empirical research and provide examples of data analytic methods that may address these issues. We highlight several recent studies that have used such methods and discuss how these methods can provide unique theoretical insights and actionable clinical information.

Results:

Several statistical approaches can enhance the field’s understanding of the timing and development of MOBCs, including growthcurve modeling, time-varying effect modeling, moderated mediation analysis, dynamic systems modeling, and simulation methods.

Conclusions:

Adopting greater diversity in methods for modeling MOBCs will help researchers better understand the timing and development of key change variables and will expand the theoretical precision and clinical impact of MOBC research. Advances in research design, measurement, and technology are key to supporting these advances.

Recent years have brought substantial growth in research on mechanisms of behavioral change (MOBCs) in substance use disorder (SUD) treatments (Huebner & Tonigan, 2007; Longabaugh et al., 2005; Magill et al., 2015; Morgenstern & Longabaugh, 2000). A common goal in MOBC research has been to identify which variables mediate the effects of treatment on substance use outcomes. This work has identified several potential MOBCs (including, for example, increased readiness for change, enhanced abstinence self-efficacy, reduced craving, and increased social support for abstinence) that may partly explain why SUD treatments help people reduce their substance use (Kelly, 2017; Magill et al., 2015).

Much of the methodology underlying MOBC research has been based on statistical mediation analysis (Baron & Kenny, 1986; Kazdin, 2006; Kazdin & Nock, 2003; MacKinnon & Dwyer, 1993), and this mediation framework has provided a flexible and extensible methodology for studying change in diverse ways. Extensions of statistical mediation include methods that can model multiple mediators (Preacher & Hayes, 2008), moderated mediation (Bauer et al., 2006; VanderWeele, 2014), multilevel mediation (Krull & MacKinnon, 1999; Preacher et al., 2010), growth-curve and difference-score based mediation (Cheong, 2011; Cheong et al., 2003; Selig & Preacher, 2009; Valente & MacKinnon, 2017), change that follows nonlinear trajectories (Fritz, 2014), and nonnormal and time-to-event outcomes (Gelfand et al., 2016).

As the field better understands which MOBC variables mediate the effects of treatment on clinical outcomes, the stage is increasingly set for pursuing a finer grained understanding of when and how these MOBCs develop. In the present study, we aim to advance the agenda of MOBC research by articulating key research questions addressing (a) when MOBCs change, (b) when MOBCs exert their effects on clinical outcomes, and (c) how treatment processes facilitate improvement in MOBCs. We do not comprehensively review all statistical approaches that could help researchers address these questions; instead we introduce and illustrate several examples of statistical approaches that may advance this work. We also highlight recent studies that have used these methods and discuss theoretical and clinical insights that may be derived from work in these areas.

Toward a more thorough understanding of MOBCs: When do MOBCs change during treatment?

MOBC research commonly tests whether putative MOBC variables mediate the effect of treatment on SUD outcomes. However, it is considerably less common for this work to delineate when putative MOBC variables themselves change. The current lack of knowledge about the timing of change for many MOBC variables is likely attributable to multiple factors. For example, researchers have understandably favored identifying whether a variable is a likely MOBC before addressing finer grained questions about when it changes. Treatment study designs also commonly have had gaps as long as several months between repeated measurements of MOBCs, greatly limiting precision in assessing the timing of change. Moreover, clinical theories of change for SUDs and other psychiatric disorders have often lacked specificity regarding the timing of change in targeted MOBCs and, therefore, miss opportunities to guide research on the matter.

Better understanding the timing of change could improve the precision of theoretical models and facilitate insight into specific treatment processes that correspond with those changes. For example, understanding the timing of change in MOBCs in relation to key treatment events (e.g., initiating or terminating treatment), milestones (e.g., obtaining a 12step sponsor), or specific behavioral changes (e.g., initiating abstinence or achieving low-risk use) could help pinpoint the specific underlying treatment processes that facilitate change in key MOBCs.

Understanding the timing of change may also benefit patients and clinicians by providing valuable benchmark data about the expected course of change in the mechanisms targeted in treatment. For example, there are currently no standardized benchmarks to describe the timing and amount of change in MOBC variables that may be expected during treatment. This leaves clinicians and patients with minimal evidence-based information about when and to what degree many improvements in MOBC variables can be expected to occur. Having available benchmark data could help reassure patients as to when distressing experiences (e.g., craving or negative affect) typically improve or worsen for most people in SUD treatment. Benchmarking could also support ongoing tracking of change in MOBCs by helping providers and patients evaluate progress in comparison to established clinical benchmarks, which in turn could facilitate discussion of treatment goals and inform clinical decision making (Goodman et al., 2013).

Growth-curve modeling (Duncan et al., 2006; Preacher et al., 2008) is one statistical modeling framework that can help researchers more precisely describe the timing of change in MOBC variables. Growth-curve models can describe average rates of change over specific key periods, evaluate how rates of change are associated with other covariates, show sudden or gradual change in relation to specific events, demonstrate the extent to which change accelerates or decelerates over time, and quantify between patient heterogeneity in change trajectories. Table 1 provides examples of empirical knowledge that can be gained from this methodology and includes examples of MOBC studies that have helped delineate the timing of change in MOBCs during SUD treatments.

Table 1.

Example methods for studying the timing of change

Method	Types of knowledge gained	Example findings
Growth curve modeling	Timing of gradual and sudden change in MOBC variables	Craving, withdrawal, and negative affect did not change in the week before quitting tobacco, then increased suddenly on the tobacco quit day, then decreased gradually over the next week to return to pre-quit levels during a nicotine pharmacotherapy trial (Piper et al., 2008).
	Quantity of change in MOBC variables during periods of interest	Drinking urges occurred on 60%–85% of all observed days before quitting drinking, then reduced to 40%–60% of days immediately after quitting drinking, and further declined to 20%–40% of days by the end of treatment in two clinical trials of cognitive–behavioral therapies (Hallgren et al., 2016).
	Association of changes in MOBCs with other covariates	Approach-based alcohol craving decreased more rapidly for patients with less pre-treatment drinking (vs. patients with more pre-treatment drinking) in the 6 months after starting community-based alcohol treatment (Schlauch et al., 2013).
Time-varying effect modeling (TVEM)	Timing of when associations between MOBC variables are strongest	Associations between positive affect and self-efficacy were strongest in the first 3 to 5 days after quitting smoking but were weaker after that among newly diagnosed cancer patients who smoke (Tan et al., 2012).
	Timing of when treatment alters the associations between MOBC variables	During the first 2 days after quitting smoking, associations between repeated measures of negative affect and craving were weaker among participants who received nicotine patches, lozenges, and bupropion compared with those who received placebo (Lanza et al., 2014).
	Impact of changes in relationships among MOBCs on treatment outcome	Experiencing a gradual reduction in the association between repeated measures of negative affect and craving within the first 14 days of quitting smoking was associated with subsequent smoking abstinence (Shiyko et al., 2012).
	Timing of when baseline measures predict change in within-treatment MOBC measures	Having a pre-treatment goal of abstinence from marijuana was associated with postsession motivation for abstinence only during the first six sessions of community-based adolescent marijuana treatment (Chung & Maisto, 2016).

Note: MOBC = mechanism of behavior change.

When do MOBCs affect clinical outcomes?

In addition to understanding the timing of change in MOBC variables, there are opportunities to better understand when MOBCs exert their effects on clinical outcomes. For example, although craving, self-efficacy, and social support for abstinence may each affect subsequent substance use, it is possible that each construct exerts this effect at different times. Some variables may facilitate the initiation of behavioral change (i.e., initiating abstinence or reduction in substance use), whereas others may be more helpful in maintaining behavioral changes that have already occurred. Many theoretical models do not delineate the timing of when putative MOBCs are hypothesized to affect clinical outcomes, and the time lags used for testing these effects in MOBC research are commonly (but often not ideally) based on the timing of measurement lags within a given data set. Moreover, some mediation models are tested using entirely cross-sectional data, giving no consideration to the timing of mediator-outcome relationships and potentially overestimating the true strength of those relationships (Maxwell & Cole, 2007; Maxwell et al., 2011).

Time-varying effect modeling (Hastie & Tibshirani, 1993; Hoover et al., 1998) is one methodology that can help describe when a variable exerts stronger or weaker effects on another variable within a given period. It can be conceptualized as a type of regression model with coefficients that change continuously over time, allowing for the examination of time-varying associations between predictor and outcome variables (Lanza et al., 2016). For example, rather than modeling the association between craving and substance use as a fixed relationship throughout the course of treatment, it is possible to model the strength, direction, and significance of the association between craving and substance use as changing over time. Predictors of outcomes can be time varying (i.e., the relationship between repeated measures of a predictor and outcomes over time) or time invariant (e.g., the relationship between a predictor at baseline and repeated measures of the outcome over time). Table 1 provides examples of recent studies that have evaluated time-varying relationships between two or more MOBC variables. Of note, most empirical examples in this area have focused on relationships between different MOBC variables, rather than the relationships between MOBCs and clinical outcomes, and much of this work has taken place in the context of smoking cessation studies.

How do treatment events, actions, and processes facilitate change in MOBCs?

As the field learns how MOBCs activate and maintain changes in substance use, it will be increasingly important to also understand how specific events, actions, and processes that occur in SUD treatment give rise to change in those MOBCs. For example, if enhanced self-efficacy facilitates or maintains reductions in substance use, clinicians and researchers will likely wish to understand how specific behavioral, cognitive, social, biological, and therapeutic factors facilitate increases in self-efficacy so this MOBC could be targeted more effectively and efficiently. It will likely be a substantial undertaking to comprehensively understand how treatment facilitates change at multiple levels (behavioral, cognitive, social, biological), and we do not attempt to describe all of the methodologies that could facilitate such understanding here. Instead, we wish to highlight key theoretical and methodological considerations that may help guide efforts to conceptualize and test how treatment processes may facilitate change in MOBCs.

Distinguishing (and linking) momentary events and sustained change mechanisms.

Researchers may benefit from conceptually differentiating two types of change-related constructs that are often similarly described as putative MOBCs despite potentially reflecting different dimensions of the change process. One type of construct reflects relatively “momentary” actions, events, and processes that often occur in discrete instances not expected to be sustained over time, whereas the other reflects relatively “sustained” patient characteristics that are potentially more stable over time and are measurable even after a momentary action, event, or process has ended (DiClemente, 2003; Doss, 2004; Longabaugh, 2007; McKay, 2007). These momentary constructs may facilitate change in sustained MOBC constructs, which in turn help maintain longer term change in clinical outcomes even after the initial momentary action, event, or process has ended. For example, momentary constructs could include specific therapist actions that deliver a treatment’s “active ingredients” (Longabaugh, 2007) (e.g., teaching specific skills or encouraging exploration of reasons to change) as well as specific patient actions that constitute “processes of change” (DiClemente, 2003) (e.g., verbally exploring reasons to change, completing homework, or initiating a new social relationship). These momentary events may then give rise to more sustained changes in patient characteristics or skills. For example, within-session exploration of reasons to change may drive an increase in readiness to change that is sustained beyond the duration of the clinical session. Similarly, specific instances of skills training and practice may give rise to sustained improvements in self-efficacy and drink-refusal skills.

Although numerous studies have evaluated how clinical outcomes are predicted by momentary constructs, such as clinician and patient in-session behaviors or homework completion (Decker et al., 2016; Gonzalez et al., 2006; Magill et al., 2014; Pace et al., 2017), there is room for additional research linking these momentary constructs with changes in the specific, sustainable mechanisms that they intend to target. Recent work has begun to illuminate associations between these two types of MOBC variables. For example, Magill et al. (2016) showed that within-session change talk in Project MATCH (Matching Alcoholism Treatments to Client Heterogeneity) predicted next-session self-reported coping behaviors and involvement with Alcoholics Anonymous measured 3 months later. D’Amico et al. (2017) found that adolescents’ within-session sustain talk in group motivational interviewing for alcohol and risky sexual behavior predicted lower self-efficacy and readiness for change measured 3 months later.

Modeling changes in quantity and function.

Another consideration is whether the processes that give rise to change should be designated as affecting the overall level of a putative MOBC variable (e.g., severity of craving) or be seen as affecting the functional relationships between variables (e.g., association between craving and substance use). Several treatment models aim to help patients establish new functional relationships between variables that previously maintained substance use, for example by uncoupling learned associations between substance-related cues, internal states, and behavioral responses. For example, mindfulness-based relapse prevention (Bowen et al., 2011; Witkiewitz et al., 2005) does not explicitly aim to reduce the quantitative level of craving or negative affect that patient’s experience but instead aims to help patients reduce the functional relationship between experiencing craving or negative affect and using substances. Likewise, cognitive–behavioral therapies (Monti et al., 1999) and some pharmacotherapies (Miranda et al., 2016) may help patients change the functional relationships between alcohol cues and subsequent cognitive, affective, or behavioral reactions.

Most MOBC research has focused on changes in the level or quantity of mechanisms as opposed to changes in their function. However, moderated mediation, also called a conditional indirect effect (Preacher et al., 2007), has offered one analytic approach for modeling how treatment process variables affect the functional relationships between MOBC and outcome variables. For example, Witkiewitz and colleagues (Witkiewitz et al., 2011; Witkiewitz & Bowen, 2010) illustrated that completing treatment modules targeting alcohol craving and participating in mindfulness-based relapse prevention attenuate the impact of negative affect on craving and heavy drinking. Further, this attenuated relationship may itself play a mediating role in reducing posttreatment drinking outcomes. Methodological tools for examining whether a potential mechanism acts as a mediator, a moderator, and/or a conditional indirect effect could also clarify changes in both quantity and function (VanderWeele, 2014).

Modeling linear and dynamic change.

In addition, MOBC researchers may wish to consider potential advantages and disadvantages of modeling change processes via traditional linear models versus dynamic systems models. Mediation models often conceptualize change as a unidirectional process that unfolds from one variable to another (i.e., treatment affects mediator, which in turn affects clinical outcome). Extensions of the simple mediation model, including models of multiple sequential mediation, may further parse the change process into increasingly finer grained series of intermediary and unidirectional steps. In contrast, dynamic systems approaches can explicitly model the dynamic, reciprocal, and often nonlinear relationships between variables involved in the change process. A dynamic systems approach can help researchers understand the complex relationships among interconnected sets of variables and model changes in higher level systems as phenomena that emerge through dynamic interactions among their lower level components. Positive and negative feedback loops are often crucial components of dynamic systems models and can help explain how some change processes unfold nonlinearly, for example, by suddenly or catastrophically changing in response to relatively small proximal changes, returning to previous equilibria even after substantial momentary change, or cycling between different system states (Hunt, 2007; von Bertalanffy, 1968).

Many change process variables are likely reciprocally related and may therefore interrelate as dynamic systems. For example, enhanced self-efficacy may lead to greater use of behavioral coping skills and vice versa (Perkins et al., 2012), motivation for change may both influence and be influenced by therapeutic alliance (Cook et al., 2015; Maisto et al., 2015), and clinicians’ in-session behavior influences patients’ expressions of reasons to change and vice versa (Gaume et al., 2008).

There are a growing number of frameworks for modeling dynamic systems in SUD treatment. Chow et al. (2015) used longitudinal mixture modeling to test a dynamic cusp catastrophe model of alcohol relapse and remission. They illustrated that proximal risk factors—stress, difficulty abstaining, and craving—predicted transitions from remission to relapse within the next 2 weeks, but patients also tended to remain in a relapse state even after those proximal risk factors dissipated. Using an approach based on differential equations and control systems, Timms et al. (2014) modeled how positive feedback between smoking and craving could cause temporary increases in nicotine craving upon quitting smoking, followed by gradual but substantial decreases in craving after that. Others have used computer simulations (e.g., in the context of social networks) to model feedback loops created from friends mutually influencing each other’s drinking behaviors, which can lead to the emergence of heavy-drinking friendship clusters that may reinforce heavy drinking and inhibit the effectiveness of alcohol interventions (Fitzpatrick et al., 2016; Hallgren et al., 2017). Simulation-based approaches may be particularly advantageous for studying dynamic systems, as they allow researchers to re-create and experimentally manipulate systems in ways that may not be possible in the real world and can generate novel hypotheses that may not have been apparent from real-world data alone (Apostolopoulos et al., 2017).

Modeling change processes as linear processes and as dynamic systems may provide complementary insights, and both approaches incur advantages and disadvantages that should be considered. For example, linear change models may be analyzed using software and statistical approaches that are more accessible to applied researchers, whereas dynamic systems models often require more complex software and analytic approaches. The delineation of clear mediators within a linear change model may provide clearer guidance about which variables should be expected to change for treatment to be effective, whereas dynamic systems often focus less on highlighting a singular variable (or set of variables) that accounts for the observed outcome and instead focus on the systemic arrangement of relationships among variables.

Dynamic systems approaches may be more advantageous in modeling many of the complexities that are involved in the change process, including the multiple and potentially interrelated variables that are often related reciprocally and nonlinearly. Dynamic systems approaches may help researchers understand how lower level interactions give rise to emergent, higher level system change, which may help with understanding how lower level phenomena give rise to higher level outcomes. Some dynamic systems approaches allow researchers to simulate and experimentally manipulate parameters that may not always be manipulated in the real world, allowing greater experimental control over the component processes that facilitate change and potentially providing insights that would not be achievable through other approaches.

Design recommendations and conclusions

Understanding the factors that give rise to change in MOBC variables will require several measurement and design considerations. Such studies may necessitate measurement with relatively high temporal resolution (e.g., weekly, daily, or more frequent), which may require brief measures to reduce participant burden. Multiple factors can influence decisions about how frequently measures should be collected, including the temporal stability of the construct being measured and the hypothesized duration it may take one construct to subsequently affect another construct. Interactive voice response systems (Aiemagno et al., 1996), ecological momentary assessment (Shiffman, 2009), and passive data collection methods (Imel et al., 2017; Milward et al., 2015) may help with obtaining large volumes of MOBC and treatment process data. Relatedly, natural language processing methods that can automatically code within-session behavior may be useful in accelerating the pace and volume of tracking momentary changes that occur within session (Atkins et al., 2014; Tanana et al., 2016). Electronic health record systems may also be an untapped source for obtaining diagnostic and treatment service–related data for millions of patients engaged in frontline clinical services and for delivering MOBC-informed clinical support tools (Ghitza et al., 2013). Currently, most electronic health record systems are poorly designed for tracking behavioral health–related data (Lyon et al., 2016), and existing behavioral health measures in electronic health record systems may have limited reliability, validity, and temporal resolution. Thus, there are numerous opportunities to develop, test, and implement electronic health record tools that are informed by MOBC research to support the collection and tracking of behavioral health data to aid clinical decision making (Hallgren et al., 2017) and promote long-term recovery outcome monitoring (Scott & Dennis, 2009).

Years of MOBC research have shed light on which variables mediate the effects of SUD treatments on clinical outcomes. MOBC researchers should increasingly embrace methods that further illuminate when and how change unfolds. This, in turn, will triangulate a better understanding of how treatments work, providing more precise and actionable clinical insights. MOBC researchers have historically been strong advocates for advancing research designs and analytic methodologies (Kazdin & Nock, 2003; Magill et al., 2015), and continuing to embrace a diversity of methods will likely lead to improved specificity in understanding how patients successfully change. Although a greater diversity of analytic methods may limit the extent to which different study conclusions can be conclusively attributed to differences in analytic methods versus differences in populations, treatments, or contextual factors, this diversity of methods is also likely to provide complementary insights into the larger picture of how change unfolds.