Some Methodological and Statistical Issues in the Study of Change Processes in Psychotherapy

Abstract

As the number of psychotherapies with demonstrated efficacy accumulates, an important task is to identify principles and processes of change. This information can guide treatment refinement, integration, and future development. However, the standard randomized control trial (RCT) design can limit the questions that can be asked and the statistical analyses that can be conducted. We discuss the importance of examining the shape of change, in addition to identifying mediators and moderators of change. We suggest methodological considerations for longitudinal data collection that can improve the kinds of therapy process questions that can be examined. We also review some data analytic approaches that are being used in other areas of psychology that have the potential to capture the complexity and dynamics of change in psychotherapy.

The central question of interest in the study of psychotherapy is change over time. Patients come into therapy with certain behavioral, emotional, and/or cognitive difficulties, and they seek relief from these problems and an improved quality of life by the time therapy is completed. One way to determine this change is to assess problems prior to treatment (point A) and at the end of treatment (point B). When it comes to understanding change in psychotherapy, is it enough to know simply that there has been improvement from point A to point B? Process researchers argue no—that is, in addition to knowing that change occurs in response to a treatment, it is crucial to understand how individuals change from point A to point B. We present some important methodological issues to consider to improve the quality of data and statistical analyses in studies of change in clinical trials.

In the first section of this paper, we briefly describe the types of questions that can be the focus of process research. In the second section, we discuss some of the ways in which the standard outcome study design, the randomized control trial (RCT) design, can limit the types of process questions that can be asked and the statistical analyses that can be conducted. We present two methodological recommendations that can address some of these limitations. First, we recommend that psychotherapy researchers increase the number of repeated assessments of symptoms and putative mediators or covariates of change over the course of treatment and follow-up. Second, we recommend that psychotherapy researchers carefully consider the timing of the effect of an intervention so that assessments are taken over an appropriate period.

In the third section of this paper, we follow these methodological issues with a description of what might be considered the current statistical state of the art in assessing change and correlates of change, individual growth curve modeling. We then describe a way of grouping individuals into particular patterns or classes of trajectories called growth mixture modeling. Finally, we explore the potential application of dynamical systems modeling to longitudinal data with intensive assessments, as a relatively novel way of characterizing change in psychotherapy.

The Questions of Psychotherapy Process Research

There is a renewed interest among psychotherapy researchers in studying not only whether psychotherapeutic treatments work, but also for whom and under what conditions they work, how they work, and why they work. These latter questions reflect more focused attention on what occurs in the interval between pretreatment and posttreatment. Questions on the process of change had been overshadowed to some extent by the pressing need to demonstrate treatment efficacy. As empirically-supported therapies proliferate, it is also important to refine the therapies, integrate important principles and components that cut across therapies, and continue treatment development efforts.

Several recent papers have called for increased attention to the assessment and examination of mediators and moderators of treatment (Kazdin & Nock, 2003; Kraemer, Stice, Kazdin, Offord, & Kupfer, 2001; Kraemer, Wilson, Fairburn, & Agras, 2002). A 2002 workshop titled “Psychotherapeutic interventions: How and why they work” was held at the National Institute of Mental Health (NIMH). This workshop was designed to clarify the goals of such work and to recommend strategies for studying mediators of change. A follow-up to this workshop is planned again in the near future. Some psychotherapy researchers (e.g., Kazdin & Nock, 2003) have argued that an increased focus on process-oriented questions will be the most rapid way that significant progress will be made in the study of psychotherapy. Indeed, as Kopta, Lueger, Saunders, and Howard (1999) remind us: “Hundreds of studies have shown that psychotherapy works better than nothing. What is not so clear is whether psychotherapy works for reasons specified by theory.” (p. 443). This sentiment is also reflected in a recent NIMH Psychosocial Intervention Development Workgroup calling for the development of new and more effective interventions for depression and the use of designs that allow for the systematic study of the process of change to guide further treatment development (Hollon, Muñoz, Barlow, et al., 2002).

Consistent with the recent resurgence of interest in the process of change, psychotherapy process research can address three sets of questions. These questions focus on the course of change (i.e., what is the shape of change?), moderators of change (i.e., for whom and under what conditions does change occur?), and mediators of change (i.e., why is change occurring?). Below we discuss each set of questions and why each represents a central direction for psychotherapy process research.

The course of symptom change (What is the shape of change over time?)

Unlike the view of change as a pre-post increment, the course of change describes the shape of change over the course of therapy for each individual. This conceptualization of individual change is based on the idea that psychotherapy is a continuous process over time. Although measurements of symptoms may be taken at two points in time, a continuous process reflects the assumption that there is some evolving, underlying trajectory that describes the way change takes place between the two time points (Francis, Fletcher, Stuebing, Davidson, & Thompson, 1991). The focus here is on understanding and describing the individual trajectories and examining how well an overall trajectory of group change adequately represents the individual trajectories. Figure 1 depicts three very different trajectories of change over the course of therapy that cannot be distinguished by simple pre-post change scores.

Figure 1.

Three hypothetical trajectories of change over the course of therapy.

Knowing the course of change can provide researchers with a good sense of when the most change is taking place. For example, change can occur smoothly over the course of therapy (as in trajectory B in Figure 1), or most of the change can be occurring at the beginning or the end of therapy (as in trajectory C in Figure 1). An examination of the trajectories can reveal the rate of change and whether the rate is constant over the course of treatment.

The shape of change can be compared with what would be predicted by the underlying theory of change for the intervention. For example, a skills-based intervention might be expected to produce a monotonic decreasing trajectory of targeted symptoms (trajectory B), whereas an exposure-based intervention might be expected to produce an S-shaped trajectory (trajectory A), with symptoms getting worse before they get better.

Knowing the shape of change can also shed light on where attention should be focused over the course of therapy in order to identify variables that may be causally related to change. If most of the change is occurring at the beginning or end of treatment (trajectory C), what factors might be operating at those times to produce the rapid change? For example, process researchers in depression have identified periods of rapid early response (Ilardi & Craighead, 1999), “sudden gains” or improvement in depression severity (Tang & DeRubeis, 1999), and transient periods of symptom exacerbation that predict later symptom reduction (see Hayes, Laurenceau, Feldman, Strauss, & Cardaciotto, this issue). Each of these patterns of change may imply different mechanisms of change.

Studying the shape of change can also reveal differences between treatment responders and nonresponders. Such analyses can shed light on whether nonresponders made any gains at any point over the course of therapy, whether gains were made and then lost, or whether gains were variable over the course of treatment. Answers to questions related to the shape of change can have important implications for testing theories of change and for treatment development.

Moderators of change (for whom and under what conditions does change occur?)

Moderators clarify under what conditions and for whom an intervention works. This definition is consistent with traditional conceptions of moderation from Baron and Kenny (1986) and Holmbeck (1999). When applied to psychotherapy trials, an additional feature is that the moderator should be a factor that precedes treatment, as recently suggested by Kraemer et al. (2002). The features of and methods for testing moderators have been well outlined by several researchers. An increased understanding of the role of moderators of change in psychotherapy can aid process researchers in important ways. For example, identifying moderators of change can allow for improved matching of patients with particular background characteristics to treatments with demonstrated efficacy with that moderating factor. Moreover, identifying a moderator of change can help to uncover potential mediators of change. Moderators can identify subgroups for whom a mechanism of change contributes to improved outcome versus subgroups for whom the mechanism does not relate to outcome.

Mediators of change (why/how is change occurring?)

Identifying and examining mediators of treatment change is important because they can elucidate the ways in which psychotherapy has effects on outcomes. They may also be thought of as the constructs or factors that an intervention program is designed to change (Judd & Kenny, 1981). Mediators represent potential mechanisms of change through which an intervention has its effects. A putative mediator of change should precede change in symptoms, occur during the course of treatment (i.e., come between initiation of the treatment and the outcome), and be associated with future change in an outcome (Kraemer et al., 2002). The identification of mediators is an important first step toward revealing and examining mechanisms of change, which are processes or events in psychotherapy that lead to and cause change in an outcome. The distinction here is whether the mediator is a causal agent (i.e., a mechanism) of change or simply a covariate of change. Thus, all mechanisms of change are mediators, but not all mediators will turn out to be mechanisms of change.

There are several reasons why a better understanding of mediators of change is important to the field of psychotherapy. First, because even the best treatment packages do not help all patients, identifying the mediators of change can help to improve a treatment by ensuring that future trials contain the mediating component and do not contain unnecessary components (Kraemer, et al., 2002). Second, understanding the mechanisms of change in therapy can potentially improve the understanding of clinical disorders and the variables associated with their course. For example, Roemer and Orsillo (2002) have expanded CBT-based treatment of generalized anxiety disorder (GAD) by integrating mindfulness/acceptance approaches into the treatment, and an examination of how changes in these variables are associated with changes in anxiety can reveal information on the nature and function of worry. Third, collecting information about mediators of change across several studies can facilitate consolidation and parsimony across the variety of treatments for psychological conditions by helping to distill the important mechanisms of change that cut across different types of therapy (Kazdin & Nock, 2003). Collins, Murphy, and Bierman (in press) also describe how moderators and mediators of change that have been identified empirically can then be used as “tailoring variables” for adaptive treatment strategies. Adaptive treatment approaches involve repeated adjustments of treatment level and type in response to individual patient needs rather than the application of fixed treatment components. The mediators and moderators are monitored and adjustments are made so that the treatment is tailored to the individual. Adaptive treatment approaches represent a potentially powerful way to refine and optimize intervention packages.

Ultimately, clinical research that pursues all three types of process questions described above should be most likely to impact treatment evaluation, treatment development, and mental health policy. However, the design of the randomized control trial that is used in most treatment outcome research can pose challenges for researchers interested in studying the process of change.

Methodological Issues in the Study of Change

Randomized clinical trials and process research

There has been a growing consensus that the RCT is one of the best methods for obtaining convincing evidence for the efficacy of a psychological treatment (Haaga & Stiles, 2000). Some of the important components of the RCT include a well-defined population of patients from which a sample of sufficient size is drawn, randomization of patients to a treatment or to one or more control or comparison groups, and assessment of patients on a set of validly measured outcome variables prior to treatment, as well as upon completion of treatment (Kazdin, 2003; Kraemer, et al., 2002). At its essence, the RCT represents an application of experimental methods to the study of psychotherapy so that causal links between a treatment and an outcome can be approximated. In this way, results drawn from RCTs are used to establish the efficacy and effectiveness of a treatment, compare the efficacy of one treatment to another, and ultimately, to influence treatment selection in practice and public policy. RCTs are one of the standard methods for determining whether particular treatments for particular disorders/conditions meet criteria for being empirically supported (Chambless et al., 1998). Acceptance of the RCT is so widespread that there are now efforts to establish consistent standards for the conduct and reporting of such trials [i.e., CONSORT (Consolidated Standards of Reporting Trials); Begg et al., 1996] that are being applied at the level of journal article and grant review. Kraemer et al. (2002) recommend that RCTs routinely assess and report moderators and mediators of treatment outcome towards the goal of understanding why and for whom treatments work. However, some important methodological features of the typical RCT often restrict the clinical researchers’ response to this call.

There are two important features of the RCT design that can limit researchers’ ability to address questions of process. First, outcome is usually assessed at pretreatment and again at posttreatment. Although an exact count of pre-post designs across all published psychotherapy trials is not possible, several clinical researchers have made similar observations (e.g., Kazdin, 2003; Kazdin & Nock, 2003; Kraemer et al., 2002). When additional measurements are taken, they are often taken as follow-ups to the posttreatment assessment rather than as more intensive assessment during treatment. Follow-up assessments are crucial for making statements about the durability of treatment effects, but they do not allow researchers to answer confidently questions about processes of change over the course of therapy.

Second, even when an outcome is measured at multiple points between the beginning and end of treatment, few studies contain measures of putative mediators at points between pre- and post-treatment. Potential mediating variables are often assessed at the same time as the symptom variables at pre and at post. Unfortunately, this assessment schedule does not allow researchers to conduct an effective test of a variable as a mediator of change. In the case where the mediator is assessed at the same time as the pretreatment outcome, this putative mechanism variable may have had no chance to exhibit variability that would be related to change in the outcome. That is, the putative mediator of change should show variability at a point earlier in time than the assessment of the outcome the mediator is purported to influence (Kraemer et al., 2002; Baron & Kenny, 1986). In the case where the putative mediator is assessed at the same time as the posttreatment outcome, the variable could be an epiphenomenon of symptom change or may no longer be relevant to outcome change by the end of treatment.

Despite being part of an otherwise well-designed RCT, these two design features (i.e., pre-post outcome assessments and no putative mediators assessed between pre and post) can limit a study’s ability to address the three questions about the course of change, moderators of change, and mediators of change. While the amount of change from pretreatment to posttreatment can be estimated in a two time-point design, the shape of change in a symptom outcome is limited to a linear pattern. Moreover, the existence of measurement error at the pre- and post-assessments can make it difficult to precisely and reliably capture the linear change, unless more time points are obtained (Willett, 1988). It may also be difficult to evaluate pretreatment covariates as potential candidates for moderators of treatment, when the measure of change is a potentially unreliable pre-post difference. As discussed earlier, with only pre- and post- assessments of variables of interest, one cannot identify adequately mediators of treatment change. We now turn to recommendations for adjusting the design of the typical RCT in order to address better questions related to the process of change.

Methodological issues related to the temporal design of RCTs

If one of the central aims of psychotherapy researchers is to study the process of change, then an important design goal is to obtain sufficient assessments of putative mediators and outcomes to obtain an accurate and informative picture of how change unfolds over time. When a researcher is planning an RCT and wants to increase the number of assessments between pre- and post-treatment, the most straightforward approach is to include a mid-treatment assessment. Mid-treatment is typically an arbitrary placement for an additional assessment, but there are other ways to decide more strategically the timing of and intervals between these assessments.

The temporal design (Collins & Graham, 2002) of a longitudinal study is an important but often under-considered factor in RCT designs. Methodologists have long been aware of the issues related to the timing and spacing of assessments and the potential consequences of temporal design on the findings from longitudinal studies (e.g., Windle & Davies, 1999), but these issues are less often discussed in guidelines for designs of psychotherapy trials. For instance, the CONSORT guidelines, recently adopted by the American Psychological Association sponsored Journal of Consulting and Clinical Psychology, recommend assessments of outcome variables at pre- and post-treatment but do not present guidelines on how researchers can use multiple assessments to enhance process research.

The temporal design of a treatment trial can influence the magnitude of observed treatment effects. As a start, a treatment intervention study must be of sufficient duration for the putative treatment effects to be apparent. The treatment takes a certain amount of time to exert its effect on participants, so the timing of the endpoint is important in order to capture the change. For example, let us assume that an intervention trial has two repeated assessments, a pretreatment and posttreatment assessment and that the assessments are separated by six months. If the effect of the treatment is strongest at two months, the posttreatment measure is taken four months after the effect takes place. In general, the farther the assessment of an outcome is relative to the time of the maximal treatment effect, the weaker the observed treatment effect (Collins & Graham, 2002).

The timing of the posttreatment assessment can also be too early. For instance, anti-depressant medications often take three to four weeks to demonstrate their maximal impact on depressive symptoms. If the posttreatment assessment was taken after two weeks, the medication could appear to have a weak effect or be ineffective when, in fact, the measurement interval between pre and post was too short to adequately evaluate the effect. Clients who start taking anti-depressant medications can also make this error and discontinue treatment prematurely. Thus, one consequence of temporal design is that an observed treatment effect can be strong, weak, or zero, depending on how close the measurement interval matches the timing of the treatment effect.

In addition to influencing efficacy outcomes, the temporal design of a psychotherapy trial can have important consequences on the evaluation of the three process questions discussed earlier—the shape of change, moderators of change, and mediators of change. A consequence of having measurement intervals that are too large when studying the shape of change is that more complex patterns over the course of therapy can appear to be simpler, but may not necessarily be representative of the true shape of change. Examining Figure 1 once again, let us say that the true course of change in a given therapy emerges as cubic or S-shaped (trajectory A). However, if the assessments are taken only at pre- and post-treatment, the form appears to be a straight line (trajectory B), suggesting a smooth, continuous decrease in symptoms over the course of treatment—a misleading conclusion. Assessments taken at pre-, mid-, and post-treatment also would not reveal the complexity of the shape of change in this case.

Temporal design can also affect the evaluation of moderators of change. A putative moderating variable may show a small or nonexistent effect if the measurement interval of the pre-post difference does not approximately match the timing of the treatment effect. As discussed earlier, measurement error in the pre-post difference score results in lower reliability of this measure of change, thus attenuating its associations with potential moderators or other covariates (Aguinis, 1995).

The temporal design of a clinical trial can also have consequences on tests of mediation. While cross-sectional designs are not appropriate for testing mediation, two wave pre-post designs are not much better (Singer & Willett, 2003). It is preferable to obtain minimally three or more repeated measurements to adequately evaluate a mediation model. Nevertheless, even with pre-, mid-, and post assessments, the magnitude of a mediation effect can vary dramatically depending on the measurement interval used to assess the putative mediator. As cogently argued by Collins and Graham (2002), the often used pre-mid-post design may not be effective for examining mediation, if the middle assessment is far from the period when the treatment has its strongest effect on the mediator. Although the mediator may be a crucial part of the treatment, in the case of an arbitrarily placed middle assessment, a mistaken conclusion of no mediation or partial mediation can be made.

We now turn to a discussion of two methodological recommendations for improving the temporal design of RCTs and similar clinical trials, drawing from literatures oriented toward methodologists who work with longitudinal data (e.g., Collins & Graham, 2002; Singer & Willett, 2003; Willett, Singer, & Martin, 1998; Windle & Davies, 1999). We then explore ways of planning clinical trials with more intensive temporal designs that can allow for more sophisticated process analyses.

Increase the number of repeated assessments

More often than not, psychotherapy researchers should increase the number of repeated assessments of outcomes and putative mediators of change that are obtained over the course of treatment. While adequate for establishing the efficacy of a treatment, it is clear that two-wave pre-post designs for clinical trials do not provide psychotherapy researchers the opportunity to examine a range of process questions. Obtaining additional assessments of outcomes and putative mediators over the course of treatment allows for more complex shapes of change to be examined and increases the precision and reliability with which simpler (i.e., linear) shapes of change are estimated (Willett, et al., 1998). As discussed earlier, a more appropriate temporal design can help researchers to identify when the greatest amount of change is taking place and when a potential mediator may be having an effect on outcome. As succinctly stated by Collins and Graham (2002), “In most cases, careful consideration of the temporal design in a longitudinal study will lead to the conclusion that more observations, more closely spaced, are needed (p. 94).”

Select an appropriate timing of assessments

As discussed above, psychotherapy researchers should carefully consider the timing of the effect of an intervention so that assessments of outcomes (and putative mediators of change) are taken over an appropriate period and are not taken too early or too late. The timing of assessments can also have implications for uncovering treatment mediation effects. If the assessments are frequent enough and occur before a mediator of change exerts its effect on a treatment outcome, then the shorter but more frequent assessments of both outcome and mediator provide process researchers with the maximal chance of capturing a mediation effect (if it indeed exists). Moreover, when the putative mediator is assessed at multiple times, there is an opportunity to determine at which assessment interval the mediator has its maximal effect. Applying this rationale to the rapid response that occurs in many cognitive behavior therapy (CBT) trials (Ilardi & Craighead, 1999), Kraemer et al. (2002) caution that “…considerable thought must be given to mediators that might operate very early and intensively. These [mediators] would have to be measured early in treatment, perhaps in the case of psychotherapy, on a session by session basis (p. 883).” Thus, a mediator assessed at one point over the course of therapy may demonstrate a small effect, while the same mediator assessed at another point may demonstrate a larger effect (Collins & Graham, 2002; Windle & Davies, 1999).

In addition, a mediation effect may be different for different outcome variables. If multiple outcomes are assessed, then there is a unique opportunity to uncover a complex unfolding of treatment effects for different outcomes. For example, a mediator of change for treatment of depression may have different effects on negative emotion, positive emotion, and behavioral outcomes over time. These potentially interesting processes of change could not be detected with the typical pre-post designs.

How does one determine the appropriate temporal design?

Collins and Graham (2002) suggest the following general principle to guide researchers planning a study: “The temporal design should be chosen in relation to characteristics of the phenomena being studied (p. 86).” In the present context, the timing and spacing of repeated assessments should match a theoretically informed conceptualization of the treatment effect and theory of change for a particular psychotherapy.

As suggested by Collins and Graham (2002), researchers should turn to prior work, pilot data, and theory to develop an educated guess about specifying an appropriate measurement rate and interval. For example, as noted above, there is some evidence for a rapid response in symptom reduction in response to CBT approaches for a handful of disorders (Ilardi & Craighead, 1999). If researchers were planning an RCT using CBT versus some other intervention, it may be prudent to assess both putative mediators and outcomes multiple times and early in the treatment. Protocol clinicians may also have tenable hypotheses regarding the process of change and can collaborate with researchers to improve the study of change in psychotherapy (Goldfried & Wolfe, 1996). Clinicians can have insights and observations about what individual change will look like, and how fast change will take place. In general, it is recommended to take more observations when change is taking place more quickly and take less observations when change is more gradual (Collins & Sayer, 2000). Pilot work and initial open clinical trials of a treatment can provide information from which to obtain useful information to guide temporal design decisions. Although these sources of information may not guarantee the optimal temporal design for a particular study, they may be preferable to using an arbitrary, but typical, pre-mid-post design for studying all change processes.

Designs for increased and more closely-spaced assessments

One fairly direct approach to obtaining more frequent and closely-spaced assessments may come from monitoring forms and worksheets that are often used routinely in weekly psychotherapy sessions in many cognitive and behavioral therapies. An alternative source for multiple assessments of putative mediators of change are weekly essays over the entire course of therapy. For instance, in some of our research, we have participants in an open trial of an exposure-based cognitive therapy for depression write weekly essays about their depression. The content of these essays is then coded and can reveal how the new therapy might be having its effects and when the hypothesized change processes are activated (Hayes, Beevers, Feldman, Laurenceau, & Perlman, 2005).

Another approach to obtaining more frequent and closely-spaced assessments is the use of diary methods. Experience sampling (Csikszentmihalyi, Larson, & Prescott (1977), daily diaries (Eckenrode & Bolger, 1995), social interaction recording (Reis & Wheeler, 1991), and ecological momentary assessment (Stone, Shiffman, & DeVries, 1999) all refer to a class of methodologies for examining everyday experience known broadly as diary methods (for a review, see Bolger, Davis, & Rafaeli, 2003). The primary benefits of examining psychotherapy process using diary methods include a) an examination of links between events, emotion, cognition, and behavior in their natural, spontaneous context; b) attenuation of biases due to retrospection by minimizing the amount of time between the experience of an event and the rating of the event; and c) a detailed accounting of experience resulting in frequent and closely spaced measurements.

In recent work applying diary methods to an RCT, nightly diaries over an 8-week period were obtained from problem drinkers to elucidate mood processes associated with the effects of naltrexone on alcohol consumption (Kranzler, Armeli, Feinn, & Tennen, 2004). Although it may not be feasible to obtain daily assessments of putative mediators and outcomes across a whole course of therapy, it may be possible to have a series of daily assessments at the beginning and towards the end of therapy to examine changes in daily within-person processes associated with emotional reactivity to daily stressors (e.g., Gunthert, Cohen, Butler, & Beck, 2004). A diary approach, however, may not be appropriate for all psychotherapy research applications. Along with the benefits, researchers should consider the potential costs of using essay writing and diary methods (or other designs for collecting more frequent and closely spaced assessments), including increased costs, increased subject burden, and increased risk of drop-out (Bolger et al., 2003).

Statistical Issues in the Study of Change

In this section of the paper, we discuss some of the statistical issues and consequences of using a two-wave design for studying individual change. We then turn to a discussion of individual growth curve modeling as a method for depicting change as a continuous process, as well as a more recent extension of growth curve modeling for capturing classes of individual growth trajectories called growth mixture modeling. We then present dynamical systems modeling as a direction that may be on the horizon for the analysis of change processes.

Limitations of traditional approaches to analyzing change

With a two-wave pre-post design, many of the traditional data analytic approaches for evaluating the efficacy of psychotherapy trials are limited in their ability to examine processes of change over time (Collins & Sayer, 2000). A central assumption in pre-post RCTs is that when two groups of the same patients are randomized to treatment and control conditions, and if there are no mean pre-treatment differences across groups, then any differences in outcome must be due to the treatment independent variable. An ANOVA-based form of analysis (e.g., repeated measures analysis of variance (RANOVA), MANOVA, multiple regression) has been the traditional way of analyzing data from such a design. A problem, however, is that the individuals in the treatment and control groups are not exactly the same, are not treated exactly the same, and do not respond to the treatment or control conditions in exactly the same way. An important consequence of traditional data analytic approaches is that participants in the treatment group respond differently to the same to the treatment, but these individual differences are attributed to sampling or measurement error instead of to meaningful individual variability in change. The result is a fixed-effects model that assumes that everyone in a group has the same treatment effect (e.g., everyone changes in the same way).

Even when a design has more than two assessments over the treatment interval, ANOVA-based approaches are limited in several important ways. ANOVA approaches typically require fully balanced data, assume equally spaced intervals between repeated assessments, cannot accommodate continuous predictors of change, and cannot incorporate both time-varying and time-invariant covariates. Probably most importantly, ANOVA approaches do not focus on simultaneously modeling both intraindividual change and individual differences in intraindividual change (Collins & Sayer, 2000; Rogosa & Willett, 1985).

Thus, the typical focus of RCTs on comparing average effects between a treatment and one or more control groups and assessing change as an increment between two (or more) time points has also had an influence on data analytic techniques. Techniques are limited to those that focus on group-level change rather than on individual change and a continuous trajectory of change over the time course. Individual growth curve modeling allows researchers to address simultaneously both levels of change.

Individual growth curve modeling

There is a long history of methodologists who have questioned whether any attempts to assess change are valid or reliable and therefore have recommended that researchers try to frame their questions in other ways (e.g., Cronbach & Furby, 1970). It is now well-understood that in order to assess individual change appropriately over time, one needs repeatedly measured longitudinal data. With only two time points, change can only be viewed as an increment over time, and this measure of change can be unreliable due to potential difficulties in separating the underlying true change score from measurement error (Rogosa, 1988; Singer & Willett, 2003). A better alternative is to view change as a continuous process of development or learning over time. Again, this problem is significantly diminished with the implementation of more than two waves of assessment and the application of individual growth curve modeling.

Work by methodologists has demonstrated that there are well-founded, valid, and reliable ways to measure change (and examine correlates of change) in the form of individual growth curve modeling (Rogosa 1988; Singer & Willett, 2003), hierarchical linear modeling (Raudenbush and Bryk, 2002), and random coefficient modeling (Hedeker, in press). In order to take advantage of and apply appropriately these more modern methods for assessing change in psychotherapy, researchers need to have an appropriate temporal design.

In essence, individual growth modeling takes place at two levels. The first level consists of fitting a regression model (linear or nonlinear) to each participant’s repeated measures data (capturing intra-individual change). The second level consists of estimating average parameters of change and the amount of individual variability around these average parameters (capturing inter-individual variability in intra-individual change). It is this inter-individual variability in change that is typically treated as error variance in traditional ANOVA-based approaches. Individual growth modeling allows researchers to model (i.e., explain) this individual variability in change with covariates of interest that may predict the shape and rate of change.

Researchers are using individual growth curve modeling to address the three process oriented questions of shape of change, mediators of change, and moderators of change. Individual growth curve modeling provides a way for researchers to directly examine the shape of change in an outcome over time. The term “growth curves” implies that the outcome needs to be smoothly increasing, but this does not need to be so. The trajectory describing the outcome can be increasing, decreasing, quadratic, or cubic. Willett, Singer, and Martin (1998) recommend a minimum of three time points to specify a linear model, four points for a quadratic model, and five points for a cubic model. When more than the minimum number of time points are used, the precision with which parameters of change are captured is increased. Depending on the number of repeated measures taken, many growth curve models can provide more statistical power compared to ANOVA approaches, with the important feature of revealing the shape of change (Muthén & Curran, 1997).

A trajectory may also show some complex combination of these patterns over distinct pieces of the assessment period (Singer & Willett, 2003). This approach is an extension of growth curve modeling known as piecewise linear modeling (Raudenbush & Bryk, 2002) that allows the model of intraindividual change to be different for different portions of therapy. An application of piecewise linear modeling to intervention data can be found in Svartberg, Seltzer, Stiles, and Khoo (1995). In this study, the authors wanted to compare the rates of progress over the course of treatment versus during follow-up in patients receiving short-term dynamic psychotherapy versus a control treatment condition. Thus, the change during the treatment “piece” of the assessments was separated from change during the follow-up “piece.” Distress was assessed at pre-treatment, 10 sessions, post-treatment, 6 months, 12 months, and 24 months after treatment, using the Symptom Checklist-90. Piecewise growth analyses revealed that improvement was greater during treatment than after and that improvement after therapy was greater for participants who demonstrated less rigid personality functioning.

In addition to assessing the shape of change, researchers can also use individual growth curve modeling to examine moderators and mediators of change. A pre-treatment between-subjects covariate can be used as a predictor of variability in intraindividual change. If this covariate interacts with a treatment factor in such a way as to have an impact on an outcome, the between-subject covariate may be considered a moderator of individual treatment change. Examining links between a putative mediator as a time-varying covariate and an outcome in an individual growth curve modeling framework is one way of approaching the evaluation of mediation. A piecewise linear modeling approach may also be explored as a way of examining mediation, where the treatment factor influences the trajectory of a putative repeatedly measured mediator in the first piece of treatment, and the slope of the mediator trajectory influences the slope of the outcome in the second piece of treatment (Khoo, 2001). Methodologists are currently working on more formal ways of examining mediation in a growth curve modeling framework (e.g., Cheong, McKinnon, & Khoo, in press).

Growth mixture modeling

As discussed above, individual growth curve modeling focuses on capturing the typical trajectory for a sample, as well as the degree to which individuals differ in the parameters that define the typical trajectory. One assumption of this approach is that all individuals have been drawn from the same population, with common population parameters defining the typical trajectory. That is, the heterogeneity that may be observed in growth trajectories is captured by continuous individual differences in trajectory parameters. An approach called Growth Mixture Modeling (GMM; Muthén, in press; Muthén, 2001) allows for parameter differences to come from subpopulations (i.e., discrete classes) that may be reflected in the sample. Thus, the growth mixture modeling approach can allow for different classes of average growth trajectories, with individuals varying around these latent trajectory classes, as compared to assuming individual variability around a single average growth curve. Interestingly, class can be thought of as having a moderating effect, with the different classes of trajectory shapes indicating qualitative types of change. GMM represents a combination of continuous and categorical latent variables in a growth curve framework, where a latent categorical variable is used to estimate the probability of membership in a particular class.

Consistent with a GMM approach, some psychotherapy researchers have noted the importance of detecting and examining potential subgroups of trajectories beyond the average trajectory (e.g., Krause, Howard, & Lutz, 1998). When applied to data in a psychotherapy trial, GMM allows process researchers to ask the following intriguing questions: a) what is the number of discrete trajectory classes that can best describe the various ways (i.e., trajectory shapes) in which participants change over the course of therapy?; b) what are the parameters that best describe the shape of change for each class?; and c) what are the antecedent variables and distal outcomes associated with participants belonging to particular classes. An interesting application of GMM to psychotherapy data can be found in Szapocznik, Feaster, Mitrani, Prado, Smith, Robinson-Batista, Schwartz, Mauer, and Robbins (2004). In this application, GMM was used to explore whether there were differing classes of trajectories representing differential response to Structural Ecosystems Family Therapy in HIV-seropositive, inner-city, African American women. Three classes emerged, representing three trajectories of change in psychological distress. One trajectory class consisted of high baseline distress that decreased over time, one consisted of high baseline distress that increased over time, and one consisted of low baseline distress that decreased over time. Different change processes may underlie each of these classes.

GMM is a superior method for determining the number of trajectory classes compared to a post-hoc clustering approach because the resulting clusters from the latter assume error-free classification. GMM provides a probability of class membership that takes unreliable classification into account. However, at this point, we would recommend that clinical researchers conduct post-hoc exploration of trajectory clusters and explore potential correlates of these trajectory clusters. These analyses can form the basis for generating future tenable hypotheses for more stringent analysis. An important potential limitation to the application of GMM to data from psychotherapy trials concerns sample size and power considerations. At this point, relatively little is known about power and sample size requirements for GMM. Based on simulation data on an example growth mixture model, Muthén (in press) has used 300 participants to obtain reasonable power for analysis. GMM is probably best utilized with datasets from multi-site collaborative studies (e.g., Elkin et al., 1989; Keller et al., 2000), where sample size may be less of an issue. Another potential limitation of GMM is the need for clinical researchers to use specialized statistical software for this type of data analytic approach (see Muthén, in press).

Dynamical systems modeling

Despite the important advances made in growth curve modeling approaches, they have some limitations for studying change in an externally valid way. One important limitation is the assumption of unbounded increases or decreases in an outcome over long periods of time. Consider again the three trajectories depicted in Figure 1. Each of them implies a long-term trajectory that increases or decreases to infiniti—a course that is unrealistic in the real world. For example, a depressed individual may respond to a treatment by showing a general decrease in symptoms over the course of the treatment period, but may eventually have life stressors that cause an increase in depressive symptoms that recede again over time. Several episodes of reactivity to stressors may follow the treatment period. This type of pattern may be characterized by oscillations in mood around an equilibrium point rather than steady increases or decreases in an outcome. This type of dynamic behavior has been used to describe mood symptoms patterns over time in disorders such as depression and bipolar disorder (Boker, 1999, Johnson & Nowak, 2002). Another pattern of long term change in a depression outcome may be small increases and decreases (resembling a zig-zag), with a zero overall trend that then suddenly increases abruptly and remains high. Regular oscillations or sudden shifts in depression outcomes cannot be modeled effectively with growth curve modeling approaches (Boker, 1999; Boker & Nesselroade, 2002).

Dynamical systems represents an alternative approach to modeling these types of nonlinear and discontinuous trajectories (Kaplan & Glass, 1995). Growth and change in psychotherapy has been conceptualized as a dynamic process reflecting both stability and instability, and destabilization of a stable behavioral and emotional pattern is often a necessary ingredient of change towards amore adaptive pattern of functioning (Hayes & Strauss, 1998; Mahoney, 1999; Schiepek, Eckert, & Weihrauch, 2003). One application of dynamical systems modeling is cusp catastrophe modeling for describing discontinuous change (see Witkiewitz and Marlatt, this issue). The idea here is that the probability of relapse to drinking may not be proportional to linear increases in associated risk factors. Rather, a relapse to drinking may be best conceptualized as a sudden shift, or catastrophe, whereby small changes in a risk factor can result in a sudden change from abstinence to relapse.

Another application of dynamical systems theory is the modeling of self-regulation processes, using continuous time differential modeling (Boker, 1999). A system is taken to be a set of variables that change over time and have some natural inter-connectedness (i.e., such as all being measured from the same individual). A dynamical system is one where the state of a system at one time is dependent to some degree upon the state of the system at a previous time point. A general characteristic of dynamic systems is that a few parameters can underlie a system whose behavior can look very complex and/or random. In particular, differential models posit relationships between the value of a variable and derivatives of the variable with respect to time (i.e., position, velocity, curvature). When variables and their derivatives are used simultaneously in an equation, one can extract parameters representing the frequency of oscillations about an equilibrium point and the degree of friction (or damping) toward an equilibrium that define a family of trajectories, all tending toward a central equilibrium point (Boker, 1999).

Two basic differences between growth curve modeling and this type of dynamical modeling are that growth curve modeling defines a single trajectory (rather than a family of trajectories), and dynamical modeling implies that a putative attractor (i.e., an equilibrium point) exists, around which differential modeling trajectories tend toward unless perturbed by some external influence (e.g., life event, daily hassle).

Although existent in the developmental (Lewis & Granic, 2000) and social psychology literatures (Nowak & Vallacher, 1998), there are only a few studies that have applied dynamic systems modeling to the study of psychotherapy (e.g. Schiepek et al., 2003; Wikiewitz & Marlatt, this issue). One reason is that dynamical analytic approaches generally require very large numbers of observations per individual (Boker, 1999). However, it is likely that tens to hundreds of repeated measurements of key variables over time will be necessary because the basis of dynamic system models is the analysis of change, stability, and transitions over time. The inaccessibility of the dynamic system literature to psychotherapy researchers and the complexity of the dynamical system modeling approaches are other reasons why these approaches have not yet caught on in the psychotherapy literature. Nevertheless, it may simply be a matter of time before software is developed to facilitate applications of dynamic models to clinical data.

Although a more complete discussion of dynamic system modeling and its application to change in psychotherapy is beyond the scope of this paper, we believe that this approach holds strong potential for use in describing trajectories of outcomes that may have discontinuous or self-regulatory features and for understanding the putative intrinsic dynamics underlying these kinds of trajectories.

Conclusion

Understanding the processes of change in any psychotherapy is itself an iterative process that cannot be done in a single study. For example, researchers can identify putative mediators based on existing trials or datasets and then follow up with RCTs that include a higher dose of the mediator of change as part of the treatment to test the hypothesis that the enhanced treatment is associated with more change than the original treatment. Moreover, any modern method for the analysis of change cannot be divorced from the quality of the research design in order to lead to meaningful results. In particular, we reviewed how issues related to temporal design can influence the effects obtained from study results and lead to potentially erroneous conclusions regarding outcome, the shape of change, moderators of change, and mediators of change. The combination of a strong theory of the nature of change in psychotherapy, a well-conceived and planned research design (including temporal design issues), and the application of modern methods for the analysis of change will be most likely to produce a fruitful line of psychotherapy process research.