Methodological Innovation to Increase the Utility and Efficiency of Psychotherapy Research for Patients with Co-occurring Mental Health and Substance Use Disorders

Abstract

Psychotherapy research with chronic and difficult-to-treat populations such as those with co-occurring mental health and addictive disorders can employ flexible research designs, which allow for a systematic, yet non-linear relationship between efficacy and effectiveness designs. Outcomes research can bypass the efficacy-effectiveness dichotomy through use of a hybrid model (Carroll & Rounsaville, 2003) conducted in the context of community treatment settings in collaboration with community providers. We present the case for using this methodological approach as a means of advancing psychotherapy research and practice, while translating and disseminating empirically supported treatments with more efficiency. A hybrid model study conducted within the National Institute on Drug Abuse’s Clinical Trials Network illustrates the application. These findings inform new directions for future research.

There have been increasing calls to provide empirical support for psychotherapy treatments. Clinicians and treatment providers are eager for additional research showing which treatments are most effective, in what context, and for whom. Despite this consensus, debate remains regarding the standards and methods to evaluate the evidence for psychotherapy models. Whereas efficacy trials with rigid study parameters allow for maximum control, they often have limited generalizability. In contrast, effectiveness models which focus on testing treatments under real world conditions often compromise causal inference and conclusive results. Experience over the past ten years in testing interventions for substance abusing women with comorbid post-traumatic stress disorder (PTSD) and delivering community-based treatment have led to an acute awareness of these dilemmas.

In this article, we examine the limits of traditional efficacy research as applied to psychotherapy treatments with chronic and difficult-to-treat patient groups. We discuss how gaps between clinical research and practice have hindered the progression of evaluation and clinical application for such populations. We present a hybrid model (Carroll & Rounsaville, 2003) in which study conditions are more naturalistic but maintain essential features of efficacy trials (e.g., randomization to intervention conditions) to inform the next generation of treatment studies. A hybrid model trial within the National Institute on Drug Abuse (NIDA) Clinical Trials Network (CTN) illustrates the application which answers practical questions with respect to sample selection, design flexibility, and therapist/supervisor training. Finally, we discuss implications and continuing challenges for future psychotherapy research.

Limits of the Linear Stage Model for Chronic Comorbid Populations

For the last two decades, behavioral research has followed a stage model of scientific inquiry borrowed largely from the pharmaocological fields (Carroll & Rounsaville, 2003). Although the stage model of research offers a systematic, scientifically appropriate way of testing behavioral therapies, there are significant limitations. As described by Rounsaville, Carroll, and Onken (2001), there are three stages in the scientific study of clinical treatments: stage I encompasses treatment development and feasibility testing, stage II tests the efficacy of the treatment in a randomized control trial, and stage III looks at the effectiveness of the treatment in a “real world” setting. Stage III trials test the external validity of the treatment and answer questions that will be most relevant to providers who may wish to adopt the treatment (e.g., cost, training recommendations, moderating variable analysis). Finally, dissemination of empirically supported interventions into the community comprises stage IV research.

Efficacy trials are designed to evaluate the scientific merit of a given intervention in a controlled manner. Guidelines have been developed in order to promote conditions under which observable post-intervention changes can be attributed to the intervention being tested and alternative explanations (e.g. passage of time, spontaneous remission, effects of assessment) can be ruled out with some degree of confidence. In order to control the extent to which these external influences can threaten the validity of the treatment intervention, rigorous methodological requirements must be met, including the comparison of the treatment condition to a comparable control by way of random assignment. Additionally, the nature and duration of the treatments are well specified, as are certain characteristics of participants receiving the treatment and of therapists providing the treatment.

Although efficacy randomized controlled trials (RCTs) have been accepted as the gold standard in clinical research, the very conditions that allow them to test for causality raise questions about their applicability to chronic comorbid populations. There have also been challenges moving from stage II to stage III research, especially in behavioral research. The complexity that arises during stage III research often limits effect sizes due to clinician training and fidelity, diverse client characteristics and presenting diagnoses, and the introduction of outside factors, such as other service use (e.g., Carroll & Rounsaville, 2007; Morgenstern & McKay, 2007). Common parameters set around patient characteristics (who is receiving the intervention), treatment implementation (the format in which the treatment is being delivered), and clinician training and qualifications (who is administering the treatment) may limit the relevance to treatment as it is practiced in community settings. The patient population of substance abusing women with PTSD serves as a prime example with which to highlight the need for creative alternatives to the traditional stage model of research. Once considered “special”, this population is now viewed as representative of a substantial subgroup of treatment seeking patients in substance abuse and mental health programs (Hufford, 2000).

Behavioral Treatment Research with Women who Comorbid PTSD and Addiction

Patient Characteristics

Stringent selection criteria typically applied in efficacy trials is one constraint that can be particularly problematic for research on chronic comorbid populations. While a circumscribed and relatively homogenous participant sample makes interpretation of results less complicated, it is often at the cost of generalizability and a representative patient sample. Instead many clinical trials are composed of stable, higher functioning patients with minimal if any psychiatric comorbidity (Bernal & Scharron-del-Rio, 2001; Borkovec & Castonguay, 1998; Schulte & Eifert, 2002). Patients seen in the community, however, tend to have multiple problems and a wide range of symptoms (Persons & Silberschatz, 1998; Seligman, 1995).

This discrepancy is especially apparent in substance abuse treatment research with women. Narrow inclusion criteria limit representation of this patient population, as a substantial portion of potential participants are ineligible (Humphreys & Weisner, 2000). The vast majority of these women present with multiple chronic disorders, polysubstance use, and complex trauma histories (Dansky, Sladin, Brady, Kilpatrick, & Resnick, 1995; Hien & Scheier, 1996), with documented rates of PTSD between 14–60% (Brady, Dansky, Back, Foa, & Carroll, 2001; Donovan, Padin-Rivera, & Kowaliw, 2001; Najavits, Weiss, & Shaw, 1997; Triffleman, 2003). These interrelated disorders influence each other and affect the course of treatment. Unlike patients with single disorders, comorbid populations typically require comprehensive, multimodal treatment to address co-existing problems and symptoms. Often in efficacy trials, in an effort to isolate the impact of the intervention being tested, participants are often asked to suspend or refrain from receiving additional behavioral or pharmacological treatments during the active study phase. This is often an unrealistic request for the vast majority of patients.

Treatment Implementation

Efficacy trials are based on assumptions about treatment implementation and participant treatment behavior. For example, there are often expectations that patients will receive all or most treatment sessions, in a specific sequence, and that they are stable enough to wait multiple weeks for randomization to a closed psychotherapy group. Setting a tight frame around treatment enrollment and sequencing in the service of maintaining a high level of internal validity may ultimately have limited utility in informing treatment. In addition to multiple mental health problems, women with PTSD and substance use disorders (SUD) tend to face numerous additional difficulties (e.g., limited social and economic resources, ongoing exposure and revictimization, physical health problems). Such obstacles affect treatment attendance and compliance and make patients less likely to benefit from treatments that build directly on attendance at previous sessions. Efficacy trials by definition offer little flexibility in term of these parameters. Patients may be discontinued from the study for missing sessions or may choose to drop out of treatment if they feel overwhelmed by study requirements.

Poor study retention has implications for methodology typically applied to efficacy RCTs. Inadequate numbers (e.g., less than 50 participants per condition) make it difficult to draw statistically sound causal inferences about treatment outcomes. This mismatch between the optimal and actual sample size plagues the field as a whole, and is evidenced by meta-analyses showing 12 to be the average sample size per condition in clinical trials (Kazdin, 2001). Again, this problem is particularly acute with SUD populations where treatment dropout rates are typically high (i.e., up to 50 or 60%). Although statistical methods such as intent-to-treat designs have been used to manage high rates of attrition, they are imperfect solutions with regards to practical questions about treatment effect (Lachin, 2000).

Clinician Training

Finally, clinical staff participating in efficacy trials are not necessarily representative of clinicians practicing in the community. Therapists who participate in these studies typically have advanced degrees and receive extensive training in study interventions prior to seeing patients, as well as intensive, ongoing supervision once the trial begins. Study clinicians typically have fewer assigned patients in the service of being able to attend more closely to those participating in the study. In addition, therapists in a controlled trial often have the luxury of focusing on a specific problem that they can address with a tailored intervention (Chambless & Hollon, 1998).

In contrast, community substance abuse and mental health clinics are often staffed by Bachelor-level clinicians or paraprofessionals who have less access to training, receive inconsistent supervision, and manage large caseloads. Community substance abuse and mental health clinics vary greatly in the resources needed to address comorbid conditions, such as PTSD and SUD. Community agencies also have varying philosophies about service delivery, especially within substance abuse treatment programs. For example, clinicians often feel strongly that trauma or PTSD should not be addressed early in recovery out of fear of relapse.

If therapists who actually deliver treatment in most practice settings are not represented in research, applicability of study findings is limited. Effectiveness in general is important, but it is also necessary to know if community clinicians can be reliably trained to deliver the treatment effectively. Though having a diverse range of therapists might require investigators to relinquish some degree of experimental control, it would allow for greater emphasis on questions related to dissemination and implementation of the treatment.

Common parameters for patient characteristics, treatment implementation, and therapist training make the use of RCTs problematic for women with PTSD/SUD as they are often in direct conflict with the clinical realities of treating this population. While many investigators continue to believe that efficacy and RCT standards should be the structure for future treatment research (Hollon, 1996), others advocate the use of more naturalistic research methods, such as effectiveness studies within the community. In effectiveness studies participants may be free to choose the treatment they prefer and the parameters of treatment are left uncontrolled or less specified. Outcome variables can be more diverse, focusing on symptom reduction and other relevant areas of functioning.

Recently clinical researchers have begun to develop flexible models to address the gap between efficacy and effectiveness research (i.e., Carroll & Rounsaville, 2003; Tunis, Stryer, & Clancey, 2003). These models allow for more naturalistic conditions, but maintain selected hallmarks of efficacy research, including RCT methodology. Carroll and Rounsaville’s hybrid model represents one such example for treatment research in substance using populations.

Methodological Innovation to the Linear Stage Model of Behavioral Research

The hybrid model was designed to increase ecological validity and the efficiency of treatment research. It has been identified as a way to link efficacy and effectiveness research in substance abuse treatment (Carroll & Rounsaville, 2003). This approach can inform behavioral research in terms of (a) treatment efficacy in community settings, (b) improving efficiency in translating empirically supported treatments into the community, and (c) highlighting directions for further treatment development via collaboration with community providers. Though there are other approaches with similar aims, such as the practical clinical trial (Glasgow, Magid, Beck, Ritzwoller, & Estabrooks, 2005; March et al., 2005; Tunis et al., 2003), the hybrid model was developed specifically to test behavioral intervention effectiveness. Practical clinical trials place greater emphasis on matching treatments to specific patient sub-populations and advocate for comparing clinically relevant treatments (i.e., two active treatment arms).

The hybrid model’s main characteristics are: (1) increased attention to training issues with clinicians of diverse backgrounds, (2) enhanced variation in setting and patient characteristics, (3) attention to community-level outcomes of interest (e.g., cost-effectiveness, acceptability, patient/provider satisfaction), and (4) retention of key RCT design features. Importantly, and in contrast to effectiveness research (stage III) which may not use a controlled trial methodology (see Seligman, 1995 for further discussion), the hybrid model retains many of the essential features of the efficacy RCT while allowing greater flexibility for multi-problem, psychiatric populations: selection criteria, retention, and implementation. Hybrid models differ from other well-designed studies that do not meet RCT criteria by the specificity of elements retained from the efficacy study template. By purposefully retaining randomization and stringent training and adherence criteria, researchers are able to answer important causal questions. Other studies which do not meet efficacy thresholds often are forced to forego randomization or enhanced adherence monitoring due to setting constraints. In addition, methodological features, including standardized, objective outcome measures, intention to treat analyses, and closely monitored treatment fidelity are encouraged.

As a consequence of discrepancies between efficacy RCT design and real world realities, there have been many obstacles to the successful translation and dissemination of empirically supported treatment, especially for chronic and comorbid patient populations. This has often put the field of psychotherapy research in the curious position of adopting treatments with limited empirical support. The conflict between the time it takes for a given treatment to progress through years of development and testing to definitively receive the designation of an “efficacious or “probably efficacious” treatment (Task Force on Promotion and Dissemination of Psychological Procedures, 1995) and the importance of providing treatment for at-risk populations is an all too common experience for providers. The hybrid model is one method for evaluating a promising new treatment, while reducing the time between stages I to III and answering practical community level questions and concerns.

A growing concern is the lack of significant focus on dissemination and implementation research within the translational research model (e.g., Glasgow, Lichtenstein, & Marcus, 2003; Gotham, 2004). The hybrid model, utilizing important effectiveness study components and highlighting external validity can better set the stage for dissemination and implementation research. For example, implementation research questions often focus on acceptability of behavioral interventions in the community and training features that are not too intensive and can be more easily applied within an agency. By addressing some of these questions earlier in the research process, both researchers participating in dissemination and providers interested in implementing innovative interventions will be better informed (Glasgow et al.).

Application of the Hybrid Model: The NIDA CTN WTS

One recently developed infrastructure testing treatments in a “real world” context is the NIDA CTN. The CTN is a cooperative research group whose mission is to make progress in the treatment of drug abuse through clinical trials in community-based settings. The CTN has nodes located across the country. Each node is coordinated by a major research center with several affiliated community-based drug abuse treatment programs (CTPs). Using this collaborative system of research centers and CTPs, the CTN conducts nationwide clinical trials to determine the effectiveness of drug abuse treatments in a broad range of treatment settings and client populations. The CTN supports studies of behavioral, pharmacological, and combined psychopharmacological treatment interventions with demonstrated efficacy in previous research.

The CTN “Women and Trauma” study (WTS) illustrates one application of the hybrid model and how it was used to address common limitations of efficacy RCTs for comorbid populations, maintaining important methodological elements. In this study (Hien, Wells, et al., under review), a 12-session version of Seeking Safety (SS; Najavits, 2002), a cognitive-behavior treatment, was tested against an attention control women’s health education intervention (WHE) delivered in group format. The WTS used a randomized control, repeated measures design to assess the effectiveness of adding a trauma informed intervention to a platform of substance abuse treatment. Participants were 353 treatment seeking drug dependent women who met PTSD criteria. Trained counselors from seven CTPs across six states conducted the treatments which occurred twice weekly over a 6-week period (12 sessions total for both interventions). The impact of treatment on drug and alcohol abstinence and PTSD symptom severity was assessed pre- and post-treatment and over 3-month, 6-month and one year of follow-up.

Below we describe how the hybrid model increased ecological validity and generalizability of the study findings by allowing for increased flexibility in participant characteristics, treatment implementation procedures, and therapist training.

Participant characteristics: Heterogeneity in diagnosis and concurrent treatment

In contrast to narrow inclusion criteria of efficacy trials, the WTS applied broader inclusion criteria to allow for maximum participation and to more closely mirror the clinical reality of PTSD/SUD populations. Specifically, participants with comorbid disorders were not ruled out (with the exception of psychotic disorders) and the number and type of substances used/abused were not limited. The diagnostic criteria for PTSD were also expanded to include participants with subthreshold diagnoses, functional impairment considered clinically indistinguishable from those meeting full criteria. In addition, participants were not required to have specific types of trauma histories (e.g., childhood physical or sexual abuse). Table 1 displays participant demographic and diagnostic characteristics.

Table 1.

WTS Study Participants’ Demographic and Diagnostic Characteristics (Baseline)

Variables	Total N=353	Seeking Safety n=176	Women’s Health Education n=177
Age, years (M, SD)	39.2 (9.3)	39.3 (9.5)	39.0 (9.1)
Race/Ethnicity (%)
Black/African American	34.0	33.0	35.0
White	45.6	47.2	44.1
Latina/Hispanic	6.5	4.0	9.0
Mixed/Other	13.9	15.9	11.9
Married (%)	17.56	14.77	20.34
Education, years (M, SD)	12.5 (2.44)	12.7 (2.32)	12.4 (2.56)
Employed (%)	40.23	40.34	40.11
Number of prior treatment episodes (M, SD)	5.0 (7.9)	5.1 (7.4)	5.0 (8.2)
Current substance use dependence (%)
Cocaine	70.5	72.7	68.2
Stimulants	7.7	8.5	6.8
Opiates	25.6	25.6	25.6
Marijuana	27.2	27.8	26.6
Alcohol	56.1	59.7	52.5
Current full PTSD diagnosisa (%)	80.4	76.7	84.2
CAPS total severity scoreb (M, SD)	62.87 (19.4)	61.56 (19.36)	64.16 (19.4)

^aPTSD = post traumatic stress disorder.

^bCAPS = Clinician Administered PTSD Scale, total range of scale = 0–136.

WTS participants who were receiving or who were interested in receiving additional treatment were not excluded. This allowed for the fact that SUD/PTSD patients are often in need of comprehensive, longer-term services. Allowing participants to attend platform addictions and psychopharmacologic treatments while experiencing an experimental treatment lends far more applicability to study conclusions (Hufford, 2000). For example, during the study treatment phase, participants attended on average 1–2 additional mental health visits each week (M = 1.3, SD = 1.6 for SS and M = 1.5, SD = 2.7 for WHE) and 3 12-step meetings (M = 3.4, SD = 4.1 for SS and M = 2.8, SD = 3.7 for WHE). Using the hybrid design allowed us to examine the effectiveness of interventions with typical patients receiving substance abuse treatment.

Treatment implementation: Practical guidelines regarding participant treatment participation

Patients dealing with addiction and trauma often have chaotic lives and may have trouble staying engaged in treatment and maintaining consistent appointment schedules. In recognition of the many obstacles to regular treatment attendance in PTSD/SUD patients, the WTS allowed participants to use treatment in a more flexible way.

Specifically, common attendance and retention problems were addressed by allowing for: (1) open group enrollment, (2) free sequencing of sessions, and (3) leniency over missed sessions. Participants were entered into group treatment following standard clinic admission policy; groups were designed to run continuously as they would in a typical substance abuse treatment setting. Prior to group entry, participants had one individual session with the therapist who oriented the woman to group structure and rules. The treatment was delivered in a repeating sequence (i.e., sessions were ordered from 1–12 but seen as free-standing so participants could enter the groups at any time point). What may have been sacrificed in control of session order and group process was made up for by increasing feasibility and generalizability and quickly engaging participants into the treatment process. In addition, research questions regarding the impact of session order and group member attendance upon outcomes is a new area of interest among psychotherapy researchers and statisticians (e.g., Morgan-Lopez & Fals-Stewart, 2006).

Missed treatment sessions were handled flexibly. Participants were terminated from the treatment phase of the study only if they missed four consecutive sessions and had no contact with study staff. Understanding the multiple competing demands and emotional states of the population, this policy promoted contact but did not penalize participants for being unable to attend treatment regularly. Instead, participants were given some leeway to be able to adjust treatment exposure. Participants who did miss a session were able to view a video tape of the group to increase exposure to treatment content as well as retain connection to the group process (permission to video tape was received from both the therapist and other group members).

Attendance and retention outcomes were encouraging and suggest that the latitude afforded by following tenets of the hybrid model was advantageous. Specifically, the mean number of sessions attended for both groups was about 7 out of a possible 12 (7.5 for SS and 6.8 for WHE). In addition, only 31 (9%) of the 353 participants dropped from the study and had no available follow up data.

Therapist training: Using community clinicians in a train-the-trainer model

Rather than hiring outside clinicians with specific training for this study, the WTS recruited counselors already practicing in the clinic environment where the study was conducted (See Miele et al., under review, for discussion of WTS training and fidelity). Counselors had a range of training, experience, and years at the CTP (see Table 2). Using a train-the-trainer model, counselors and supervisors were trained and certified on study treatments guided by the treatment manual for each condition (SS: Najavits, 2002; WHE: Miller, Pagan, & Tross, 1998). Supervisors were certified to conduct adherence monitoring and provide supervision to the counselors. Study supervisors also received additional support and oversight from the coordinating site. Over the course of the study, supervisors rated the adherence of 257 SS video taped sessions and 193 WHE sessions. The coordinating site co-rated 60 SS and 71 WHE sessions. Overall adherence was high, with inter-class correlation coefficients in the good to excellent range (Cicchetti, 1994; see Table 3). These findings clearly show that community-based counselors can be trained to deliver trauma-specific treatment with fidelity. The monitoring of adherence to the manualized treatments and reliability checks between CTP supervisors and the coordinating site increased the capacity of the CTP to sustain intervention delivery after study completion and be able to train other staff and service providers in their area.

Table 2.

WTS Study Counselor and Supervisor Demographic Characteristics

	Counselor n=18	Supervisor n=18
Age: M	38.0	41.8
Race: N (%)
White	9 (50.0)	12 (66.7)
Black/African American	5 (27.8)	5 (27.8)
Hispanic/Latina	4 (22.2)	1 (5.5)
Yrs working in substance abuse treatment: M	4.8	9.0
Years at current program: M	3.9	4.8
Highest degree: N (%)
>Bachelors degree	1 (5.5)	1 (5.5)
Bachelors degree	7 (38.9)	2 (11.1)
Master’s degree/Doctorate	10 (55.6)	15 (83.3)
In recovery: N (% Yes)	4 (22.2)	2 (11.1)

Table 3.

WTS Study Adherence and Reliability Ratings for Seeking Safety and Women’s Health Education Interventions

Variable	Seeking Safety	Women’s Health Education
Total sessions rated	257	193
Adherenceab (M, SD)	3.8 (0.27)	4.0 (0.66)
Number of sessions co-rated by site supervisor and lead supervisor	60	71
Agreement (%)	96.7	94.4
Internal consistency (alpha)	.82	.98
ICCcd	.73	.77

^aMean score was calculated using full sample of sessions rated by site supervisors, valid cases only (no missing data).

^bAdherence calculated using a 5-point scale.

^cICC = intra-class correlation coefficient, average measure reliability, mean rating of all raters.

^dICC calculated using co-rated sessions (SS=60 [23.3%]; WHE=71 [36.8%]).

The hybrid model design allowed for the examination of training effectiveness with community clinicians delivering the study intervention to a typical substance abuse treatment population, while providing information on the transportability of these interventions.

WTS Outcomes

While it is beyond the scope of this paper to describe all the outcomes of the WTS, some detail is important to evaluate the utility of the hybrid model. Indeed, a series of publications and manuscripts demonstrate the richness and flexibility of the data. Papers detailing the impact of the two treatment groups include a primary outcome paper presenting clinically significant reductions in PTSD symptoms in both SS and WHE (Hien, Wells, et al., under review) and significant reduction in HIV sexual risk behavior among more risky women in the SS group (Hien, Campbell, et al., under review). Of particular importance to community programs is an analysis of adverse events showing that SS, the trauma-specific treatment, was no different than WHE in the number of women reporting study-related adverse events (Killeen et al., 2008). Given the common concern that directly addressing trauma histories might result in relapse among women in substance abuse treatment, this finding has much relevance. Additional papers focus on the examination of helping alliance across treatment groups and as a predictor of outcome (Miele, Hien et al., in preparation) and the demonstration of the importance of reducing PTSD symptoms in order to have a significant impact upon substance use outcomes (Hien, Jiang et al., in preparation). The implementation of the hybrid model in the WTS has allowed us to answer research questions in a meaningful and applied way across a number of domains.

Sustainability

Based on the primary outcome results indicating that both SS and WHE significantly reduced PTSD symptoms, all of the CTPs who completed the trial have elected to continue using these treatments in their programs. Two sites have offered training on the interventions to other programs in their areas, and in one case in another state. Several of the sites are finding ways to adapt the treatment for use with different populations within their programs. For example, one of the CTPs trained seven additional staff members and plans to conduct groups with adult men, women in residential treatment, and adolescent girls. The CTP described that patients were initially wary of joining a trauma group, but by the end of the second group had become very enthusiastic, and talked positively about the group to other patients (M. Miller, personal communication, February 8, 2008). One aspect of implementing an empirically supported treatment that has been more difficult to maintain is monitoring outcomes through assessment, such as PTSD symptom frequency and severity using self-report. CTP clinical staff was not responsible for participant assessment during the study and this may be an additional burden that will require more oversight and practical solutions. Finally, one site is implementing a new trauma-specific program within their agency as a result of participating in the WTS.

Psychologists and mental health professionals interested in becoming involved in research using hybrid models should consider networking and collaborating with academic researchers and universities who can provide grant writing and methodological expertise. In addition professional associations are increasingly interested in the integration of research and practice to study psychotherapy treatment and disseminate findings into community settings. As more research funding is earmarked for health services and dissemination and implementation research, there should be increased opportunities for clinician/researcher collaboration.

Limitations and Next Steps

Despite the advances that can be achieved from conducting empirically based treatment research using more flexible research designs, it is critically important to note the realities and limitations of any treatment research design with a comorbid population. First and foremost, the nature of chronic illness by definition involves the need for long term treatments. Yet, funding pressures and logistical implementation challenges typically limit the length of treatments under study. On average, most CBT treatments are delivered over a 3–4 month window, whereas the average length of clinical treatment for a patient with comorbid addictions can be many years. In the WTS example, participants were given only six weeks of twice weekly group therapy. One can easily see that setting up a clinical trial with such a limited dose of treatment sets the bar very high for the performance of that treatment. Short of conducting a more lengthy clinical treatment phase, which may be unlikely given cost and feasibility issues (e.g., participant attrition and staff turnover), the fundamental mismatch remains that short-term treatment is being tested on long-term problems. Studies that focus on short-term change in a few problem areas, neglecting the long-term clinical picture, will continue to have more limited practical use.

One way to minimize misinterpretation of a clinical trial with short-term findings would be to exercise caution in giving any one study undue weight, particularly in a negative direction. It is possible that Type II error (i.e., failure to detect a significant outcome for an effective treatment) might obscure what is in reality a significant finding due to limited power as a consequence of the complexities inherent in behavioral research. Empirical support for any psychotherapy with comorbid populations should always be considered in the context of a body of work with a particular treatment, and never rest upon the negative or small effect sizes from one or two trials. Given the need for multiple trials of a promising treatment, the hybrid model may be a more efficient alternative.

Other types of secondary analysis to support and supplement primary outcome treatment effects are warranted to answer complex questions about who the treatment works for and in what context (e.g., Rapkin, 2002). Statistical methodologies can answer some questions, such as moderation and mediation analyses (e.g., Kraemer, Wilson, Fairburn, & Agras, 2002), and support greater design flexibility (e.g., Morgan-Lopez & Fals-Stewart, 2006). Because psychotherapy effect sizes are likely to be small (e.g., underestimated due to the limited dosage), researchers should recognize the potential significance of even a small effect and further examine the benefit among subsets of individuals. Expectations and criteria for interpreting outcomes of psychotherapy research should also be reevaluated.

Conclusion

Our findings support using a hybrid model with comorbid, chronically ill populations. These types of designs can maximize external validity (while maintaining internal validity controls) and yield many meaningful findings. When a psychotherapy treatment is found to be effective using a hybrid model, clinical program staff already trained in the treatment can begin dissemination to other staff within their own organization and to the larger community. In this way, cost effectiveness is maximized while dissemination grows over time. Using a PTSD/SUD population as a case in point, we illuminate the inefficiency of the traditional linear stage model of behavioral therapy given the diversity of population, predictably irregular treatment participation, and relevance of training and supervision implementation issues. These findings advance our understanding of treatment delivery research for this comorbid population.

Biographies

DENISE A. HIEN received her PhD in clinical psychology from Teachers College, Columbia University. She is a professor in clinical psychology at City University of New York. She is also an adjunct senior research scientist at Columbia University College of Physicians and Surgeons and was the founding executive director of The Women’s Health Project Treatment and Research Center at St. Luke’s\Roosevelt Hospital in New York City. Dr. Hien also maintains a private practice in New York City. Her research interests are in treatment development for women with co-occurring substance abuse disorders and post traumatic stress. Contact information: North Academic Campus-Room 8-131, 138th Street and Convent Avenue, New York, NY 10030, dah26@columbia.edu

LISA R. COHEN received her PhD in clinical psychology from Yale University. She is a research scientist at the New York State Psychiatric Institute, Columbia University Medical Center and at the City College of New York. She maintains a private practice in New York City. Her areas of research include development and testing of interventions for women with posttraumatic stress and substance use disorders.

AIMEE N. C. CAMPBELL received a M.S. in social work from Columbia University School of Social Work (CUSSW) and is currently a doctoral candidate at CUSSW. She is a Senior Research Associate at CUSSW Social Intervention Group and a Project Director with the Long Island Node of NIDA’s Clinical Trials Network. Her research interests include gender specific interventions for women in substance abuse treatment, HIV prevention intervention, and innovative dissemination and implementation strategies for empirically supported treatments.