Beyond the constraints of an RCT: Naturalistic treatment outcomes for anxiety-related disorders

Abstract

Despite considerable data from randomized controlled trials supporting use of behavioral therapies for anxiety disorders and anxiety-related disorders, there is a relative scarcity of data demonstrating that such findings are generalizable to patients in nonresearch settings, and a lack of standardized repeated outcome measurement in such settings. Using one of the largest examinations of naturalistic outcomes of behavioral therapies in treatment-seeking patients (N = 489), we examined the clinical characteristics and treatment outcomes of patients seeking treatment for anxiety and anxiety-related disorders in the past 3 years. Patients seeking treatment at a clinic specializing in cognitive-behavioral therapy (CBT) completed self-report questionnaires via an electronic data capture system and diagnostic interview at baseline, and were reassessed at mid- and posttreatment. Patients with anxiety and related disorders were assessed for changes in symptom severity and secondary outcomes (impairment/functioning, quality of life, and depression) over the course of therapy. Patients showed clinically significant and statistically reliable improvement in anxiety symptom severity scores over treatment (p < .001), after controlling for number of sessions received. Patients also showed significant improvement in depression, quality of life, and functioning (p values ≤ .001). We also found significant improvement in disorder-specific symptoms, including obsessive-compulsive disorder, posttraumatic stress disorder, generalized anxiety disorder, and social anxiety disorder (p values ≤ .001). Importance of, and ways to facilitate, integration of more routine assessment of a broader range of symptoms via online assessment systems and methods to better determine the effectiveness of CBT in naturalistic clinics are discussed.

Introduction

Despite numerous randomized controlled treatment (RCT) studies demonstrating the efficacy of cognitive-behavior therapy (CBT) for adult anxiety disorders (e.g., generalized anxiety disorder [GAD], social anxiety disorder [SAD], panic disorder, and specific phobias) and anxiety-related disorders (obsessive-compulsive disorder [OCD] and posttraumatic stress disorder [PTSD]; e.g., Norton & Price, 2007;Olatunji, Cisler, & Deacon, 2010), a contentious debate continues over the “real-life” applicability of these treatments. Clinicians often feel that the circumstances necessary to conduct an RCT compromise the external validity of the treatments. There is a prevailing belief that several variables, including the type of patients, settings, and therapists delivering treatment in RCTs, are not clinically representative of the average clinical setting. A number of researchers agree with these concerns and have advocated for more effectiveness studies to complement efficacy studies in order to support the generalizability of RCT results in more naturalistic settings (e.g., Hunsley & Lee, 2007;Stewart & Chambless, 2009) and to determine optimal treatment in the “real world” (Leichsenring, 2004).

Effectiveness studies examine how well a “typical” patient in a naturalistic clinical setting will benefit from a therapeutic treatment. However, there is tremendous heterogeneity in “typical” clinical patients, settings, and therapists. For example, what is a standard patient seeking treatment in a community mental health center, in a private practice, in a primary care setting, or in a hospital? A number of patients receive outpatient therapy in each of these settings, and yet it is unclear which of these patient groups are considered “typical.” This ambiguity is particularly important because, as mentioned above, community clinicians have posited that research patients look different on a number of factors (e.g., comorbidity profiles, symptoms severity, complexity of psychosocial stressors) than solely treatment-seeking patients. Similarly, professional therapists vary by level of experience, training, degree of specialization, and intensity/frequency with which they provide therapy (e.g., weekly, biweekly, or flexibly), and there is a concern that the therapists conducting treatment in the context of RCTs do not resemble clinicians in “real-world” settings.

As a result of this perceived heterogeneity in patients and clinicians specifically in nonresearch settings, effectiveness studies have defined “naturalistic” using different criteria. For example, Hans and Hiller (2013) required six common criteria for effectiveness studies to be included in their meta-analysis, which included studies where patients are referred clinically and not experimentally, and where patients receive treatments from therapists with no training specific to the treatment provided using a flexible structure. A meta-analysis of 11 effectiveness studies (van Ingen, Freiheit, & Vye, 2009) followed similar criteria, only including studies that were conducted in a nonuniversity setting and requiring inclusion of studies with patients who had comorbid conditions. This meta-analysis found that in routine clinical practice, cognitive-behavioral interventions were significantly associated with anxiety symptom improvement. In a larger meta-analysis of 56 anxiety disorder treatment studies, Stewart and Chambless (2009) rated effectiveness of clinically representative treatment research based on nine criteria, including the allowance of medication and unrestricted disorder comorbidities. Results showed large effect sizes for treatment across all anxiety disorders included in the meta-analysis, indicating positive symptom improvement at posttest. While effectiveness meta-analyses judge clinical effectiveness differently, the results of these studies have been comparable to the results from RCTs. However, more effectiveness studies with larger sample sizes, more stringent effectiveness criteria, and additional varieties of “naturalistic” settings are still needed.

As indicated from these meta-analyses, effectiveness studies with anxiety and related disorders in particular have indeed been conducted in a variety of “naturalistic” settings, including outpatient clinics (e.g., DiMauro, Domingues, Fernandez, & Tolin, 2013;Hahlweg, Fiegenbaum, Frank, Schroeder, & von Witzleben, 2001;Wootton, Bragdon, Steinman, & Tolin, 2015), anxiety specialty clinics (Franklin, Abramowitz, Kozak, Levitt, & Foa, 2000), private practices (Gaston, Abbott, Rapee, & Neary, 2006) and primary care settings (Craske et al., 2009). Wootton et al. (2015) collected data from 98 patients with a variety of anxiety disorders in an outpatient fee-for-service clinic. This effectiveness study is one of the few to explore long-term outcomes. Results indicated that treatment gains were maintained across 3 years of follow-up with large effect sizes at posttreatment. In a naturalistic study conducted in a similar clinic, DiMauro et al. (2013) collected data from 181 patients presenting with a variety of anxiety disorders. Patients who had completed at least three sessions of CBT were included in the study and completed a follow-up at 1 year. Results indicated that the majority of patients (around 62%) showed symptom improvement at posttreatment, and the majority of these responders (around 77%) maintained their responder status at the 1-year follow-up. In both studies, the measures used to gauge improvement were general ones related to broader functional impairment and not specific to the primary disorder that was being treated.

Another study by Hahlweg et al. (2001) collected data from 416 exclusively agoraphobic patients from three outpatient clinics. Follow-up at 6 weeks and 1-year posttreatment indicated significant improvements in symptom severity. However, although the authors utilized more diagnosis-specific measures to assess improvement, this effectiveness study focused only on a brief, intensive treatment for agoraphobia, thus these results are not necessarily generalizable to more commonly delivered treatments of a wider range of disorders.

Effectiveness studies of exposure therapies for anxiety-related disorders are particularly important given their wide evidence base and yet concurrently low rates of utilization (Asnaani, Gallagher, & Foa, 2018). Specifically, therapists show reluctance in utilizing and implementing exposure therapies in their practice given the belief that patients studied with these treatments in RCTs are significantly different from the “real-world” clients they see, for all the reasons delineated above. Therapists have expressed concerns that exposure therapy, in particular, will cause excessive distress and increase the likelihood of patient dropout (Becker, Zayfert, & Anderson, 2004;Olatunji, Deacon, & Abramowitz, 2009), despite evidence that there are not higher dropout rates in exposure therapy as compared to other CBT techniques (e.g., Hembree et al., 2003). Furthermore, therapists have reported doubt over whether research treatment protocols utilizing exposure therapies can effectively be generalized to nonresearch clinical settings (Warren & Thomas, 2001). Cahill, Foa, Hembree, Marshall, and Nacash (2006) posited that these concerns, in addition to low training rates, may contribute to the lack of widespread dissemination of exposure-based therapies. Studies also suggest that rates of exposure therapy implementation may be lower than that of other evidence-based practices (EBPs; Harned, Dimeff, Woodcock, & Contreras, 2013;Jensen-Doss, Cusack, & de Arellano, 2008), showing staggeringly low uptake rates in clinical practice (e.g., 17–20%; Becker et al., 2004). Further exploration of the severity and comorbidity profiles of patients in naturalistic clinical practice could address this perceived difference between naturalistic and RCT patient samples. Furthermore, such an examination within the context of flexibly delivered exposure therapies may allay clinicians’ concerns about the effectiveness of such therapies in more naturalistic clinical settings. This study incorporates both of these critical (and currently understudied) areas in order to promote implementation of exposure therapies in practice settings.

In addition, one reported weakness of most RCTs is the focus only on symptom improvement as the major outcome of interest. Specifically, meta-analyses reviewing the efficacy and effectiveness of CBT for anxiety and related disorders have largely focused on outcome as a function of reductions in disorder-specific symptom severity (Hofmann & Smits, 2008;Stewart & Chambless, 2009). Many in the field are moving toward considering a more nuanced view of recovery and success of psychotherapy, including constructs such as functional impairment and quality of life (e.g., Hofmann, Wu, & Boettcher, 2014). For example, Asnaani et al. (2017) measured improvement in quality of life and functioning in patients receiving exposure treatment for OCD and found that improvements in quality of life and functioning were significantly associated with OCD symptom improvement. Similarly, an RCT of 325 male Vietnam veterans suggested PTSD symptom severity was associated with reduced quality of life (Schnurr, Hayes, Lunney, McFall, & Uddo, 2006). Results indicated that change in PTSD symptoms was subsequently associated with change in quality of life. Studying such secondary outcomes is a relatively recent development and more research is needed to determine the improvement seen in these areas following treatment, particularly in the context of naturalistic, effectiveness studies of CBT (Stewart & Chambless, 2009).

This study was intended to address a number of these gaps in the literature (i.e., a scarcity of findings on a broader range of outcomes, use of large naturalistic samples, and utilization of treatment providers who vary in their levels of experience and training). Thus, the current study differs from the handful of previous studies in this area by incorporating several elements that have not been combined together in one examination thus far: use of a large sample of patients seeking CBT treatment for a variety of anxiety disorders and anxiety-related disorders (OCD and PTSD), multiple data collection points throughout treatment, the use of both diagnosis-specific and broader secondary outcome measures, variety in the experience level of therapists, and a flexible number of permitted sessions. Furthermore, all therapists were trained in workshops for OCD and PTSD treatment, but were not monitored for manual adherence. Finally, there was no formal training for the anxiety disorders (GAD, SAD, panic disorder, and specific phobias) treated at the clinic, and treatment content and structure were not dictated on a systematic level.

Specifically, this study had several major aims: First, the study explored sample characteristics in this naturalistic setting, including the overall symptom severity and comorbidity patterns in the sample before the start of treatment to provide a detailed description of the clinical features of this naturalistic sample. Related to this, important clinical characteristics of the sample, such as treatment length, dropout rate, and number of sessions received, were also examined. Second, improvements in primary anxiety symptoms and more global outcomes (depression, functional impairment, and quality of life) over the course of therapy were examined, and finally, these outcomes were compared across primary anxiety disorders (OCD and PTSD), using sophisticated higher-level statistical models (including mixed models and reliable change [RC] statistics).

Method

Participants

Participants were 489 adults ages 18 and older (M = 31.73, SD = 11.71, ranging from 18 to 73 years old) who received a primary diagnosis of a DSM-5 anxiety or anxiety-related disorder at an outpatient, fee-for-service specialty anxiety clinic in Philadelphia, Pennsylvania. (See the Supplement for details on how primary diagnosis was determined.) Patients presenting for treatment were slightly more likely to be female (52.1%), and the majority were White (76.7%). See Table 1 for full demographic descriptives of the sample. Inclusion criteria for study participation were (a) male and female patients ages 18 and older; (b) DSM-5 diagnosis of GAD, SAD, specific phobia, panic disorder, OCD, PTSD, and/or other/unspecified anxiety disorder; (c) symptom duration of at least 3 months; (d) appropriateness for receiving outpatient care; (e) the ability to consistently attend a weekly or biweekly treatment schedule; and (f) willingness and ability to provide written consent for study participation after study procedures have been explained. Exclusion criteria were (a) primary diagnosis other than the anxiety or related disorders listed above, (b) active suicidality, (c) schizophrenia without stable medication regimen or patient deemed to have more than mild current symptoms of psychosis even while on stable antipsychotic medication regimen, (d) intellectual disability, (e) psychosis, (f) severe autism, and (g) primary substance or alcohol dependence, or significant substance use symptoms that required specialized treatment before anxiety symptoms could be addressed.

Table 1.

Sample demographics

	N	%
Gender
Female	255	52.10%
Male	213	43.60%
Transgender	1	0.20%
Other	3	0.60%
Missing	17	3.50%
Race
White	375	76.70%
Asian	28	5.70%
Multiracial	22	4.50%
Hispanic	15	3.10%
African American	14	2.90%
Other	5	1.00%
Missing	30	6.10%
Level of education
No degree	6	1.20%
High school diploma/GED	129	26.40%
Associate’s degree	18	3.70%
Bachelor’s degree	188	38.40%
Master’s degree	74	15.10%
Doctoral degree	44	9.00%
Missing	30	6.10%
Relationship status
Single	194	39.70%
In a relationship	84	17.20%
Living with partner	52	10.60%
Married	114	23.30%
Separated	6	1.20%
Divorced	9	1.80%
Missing	30	6.10%
Employment status
Full-time	188	38.40%
Part-time	59	12.10%
Retired	14	2.90%
Student	130	26.60%
Disabled	9	1.80%
Unemployed, looking for work	23	4.70%
Unemployed, not looking for work	25	5.10%
Other	11	2.20%
Missing	30	6.10%

Note. GED = general equivalency diploma.

Procedure

All study procedures were approved by the Institutional Review Board at the University of Pennsylvania. As part of the clinic’s normal procedures, every patient seeking treatment between 2015 and 2017 at the clinic first completed a brief (~ 20–30 minutes) initial phone screen with a research assistant (RA) who assessed primary anxiety symptoms and clinical eligibility for treatment, with special attention to inclusion and exclusion criteria. If eligible, patients were scheduled for an in-person 2-hour intake evaluation that consisted of tailored structured and unstructured interview components, as described in detail in the Supplement.

Following the phone screen, patients completed a pretreatment self-report questionnaire packet using a secure online database (REDCap; Harris et al., 2009) prior to their intake evaluation. These questionnaires were selected by the clinical team and were chosen based on clinical and research utility. Clinicians used these measures in session to assess patient progress on primary and secondary outcomes, including quality of life, global disability, and so on. All patients presenting to the clinic were given the option to consent to the study during the intake evaluation, during which the evaluating clinician reviewed the informed consent form with the patient. All patients completed both the online self-report and in-person components of clinical care, regardless of whether they consented to the study or not, as part of routine clinical procedures. Data presented in the current study are only from patients who consented to having their data being used as part of the study and who initiated treatment (i.e., completed one or more treatment sessions).

Patients who were recommended to begin treatment at the center were placed on a wait-list, and started treatment typically within 4–6 weeks after the initial evaluation. Patients were also asked to fill out self-report questionnaires identical to the pretreatment questionnaires via another secured online link at midtreatment (roughly corresponding to week 7 after the start of treatment) and posttreatment (roughly corresponding to 19 weeks after the start of treatment, or when the patient terminated, whichever came first). A week prior to their patients’ midtreatment or posttreatment points, clinicians were asked to fill out a short questionnaire describing features of the treatment (such as primary diagnosis treated, treatment protocol used, number of sessions attended, and length of time of entire treatment episode). Details about systematic reminders provided to patients and clinicians for completion of questionnaires by the study team is detailed in the Supplement.

At any point during a patient’s treatment, if the patient had terminated or dropped from treatment, the clinician informed the study RA of the termination, so that the study RA could send the patient a posttreatment survey. At various points during a patient’s treatment (e.g., after the clinical intake, and after the conclusion of treatment), a member of the study staff entered information about the patient’s current diagnoses and treatment status at the clinic into the patient’s confidential study record. This information was used from a research perspective to corroborate patients’ self-reported symptom data, and clinicians’ report of their patients’ diagnoses and treatment status on the mid-/posttreatment checklists.

Treatments

All patients presenting to the clinic were treated for their primary diagnosis utilizing various CBT-based treatment protocols, approximately 92% of which were exposure-based protocols. Among these exposure treatments, these evidence-based protocols primarily included treatments such as exposure and response prevention (Ex/RP) for OCD (Foa, Yadin, & Lichner, 2012) and prolonged exposure (PE) for PTSD (Foa, Hembree, & Rothbaum, 2007). Other protocols utilized included CBT for panic and unspecified anxiety (Craske & Barlow, 2006), and SAD (Hofmann & Otto, 2017), as well as mindfulness for GAD and unspecified anxiety (Orsillo & Roemer, 2011). For detailed information about clinician experience and training, please see the Supplement.

Patients typically completed 10–20 sessions of individual therapy, consistent with the guidelines for treatment length in the aforementioned protocols. A patient was designated as a “treatment dropout” based on the treating provider’s clinical judgment, which took several factors into account, such as whether the patient stopped coming to treatment before he or she had reached his or her stated treatment goals, were noncompliant with treatment recommendations, or were resisting exposures at the top of fear hierarchies despite ongoing significant interference or distress.

Measures

Beck Anxiety Inventory (BAI)

The BAI (Beck, Epstein, Brown, & Steer, 1988) is a 21-item self-report measure assessing for symptoms of past-week anxiety and panic. Each item, which evaluates how much clients are bothered by their symptoms, is rated on a 4-point scale ranging from 0 (not at all) to 3 (severely, I could barely stand it). The questionnaire was initially developed to discriminate between anxious and nonanxious diagnostic groups and is moderately correlated with the Hamilton Anxiety Rating Scale (r = .51), and mildly correlated with the Hamilton Depression Rating Scale (r = .25). As this is a naturalistic setting, measures collected were periodically evaluated and replaced over the course of the 3-year data collection period to reduce patient burden. In the current study, the BAI was replaced with the Panic Disorder Severity Scale (PDSS) after about the first 184 patients enrolled in the study.

Beck Depression Inventory II (BDI-II)

Depressive symptoms were assessed using the BDI-II (Beck, Steer, & Brown, 1996). The BDI-II is a widely used 21-item scale, with each item scaled from 0 (no disturbance) to 3 (maximal disturbance). Item scores are summed for a total severity score ranging from minimal depressive symptoms to major depressive symptoms.

Generalized Anxiety Disorder–7 Item (GAD-7)

Symptoms of GAD were assessed using the GAD-7 (Spitzer, Kroenke, Williams, & Löwe, 2006), which is a brief, seven-item self-report scale assessing the frequency of anxiety symptoms rated from 0 (not at all) to 3 (nearly every day). The final item assesses overall difficulty of items that were previously endorsed rated from not difficult at all to extremely difficult.

Obsessive-Compulsive Inventory—Revised (OCI-R)

The OCI-R (Foa et al., 2002) is an 18-item self-report questionnaire that assesses for the severity and presence of OCD symptoms. This measure is a shortened version of the original OCI scale. The OCI-R includes six subscales: washing, checking, ordering, obsessing, hoarding, and mental neutralizing.

Panic Disorder Severity Scale (PDSS)

Panic symptoms were assessed using the self-report version of the PDSS (Houck, Spiegel, Shear, & Rucci, 2002). The PDSS is a seven-item questionnaire measuring seven dimensions of panic disorder and related symptoms, including frequency, distress, interoceptive and agoraphobic fear and avoidance, and social and work functioning impairment. Client responses are rated on a scale of 0 (none) to 4 (extreme).

Penn State Worry Questionnaire (PSWQ)

The PSWQ (Meyer, Miller, Metzger, & Borkovec, 1990) is a 16-item self-report questionnaire that measures pathological worry and correlates well with other measures of worry. Items were rated on a 5-point scale of 0 (not at all typical of me) to 4 (very typical of me). In the current study, the PSWQ was replaced with the briefer GAD-7 to reduce patient burden after about the first 187 patients enrolled in the study.

Posttraumatic Diagnostic Scale for DSM-5 (PDS-5)

The PDS-5 (Foa et al., 2016) is a 24-item self-report questionnaire that assesses PTSD symptoms and severity according to DSM-5 criteria. Responses are rated on a 5-point Likert scale from 0 (not at all) to 4 (six or more times a week/severe). The first 20 items of the PDS-5 yield a total score of 0–80 and includes four subscales based on the DSM-5 symptom clusters: reexperiencing, avoidance, changes in mood and cognition, and hyperarousal. The final four items assess onset, distress, and duration of PTSD symptoms.

Quality of Life Enjoyment and Satisfaction Questionnaire—Short Form (Q-LES-Q-SF)

The Q-LES-Q-SF is an 18-item questionnaire based on the original Q-LES-Q (Endicott, Nee, Harrison, & Blumenthal, 1993). This measure assesses the degree of enjoyment and satisfaction in daily functioning and life.

Sheehan Disability Scale (SDS)

The SDS (Leon, Olfson, Portera, Farber, & Sheehan, 1997) is a three-item instrument assessing impairment across three domains: work, social life/leisure activities, and family life/home responsibilities. Clients rate subjective impairment on a scale of 0 (not at all) to 10 (very severely).

Social Phobia Inventory (SPIN)

Symptoms of social phobia were assessed using the SPIN (Connor et al., 2000), which is a 17-item self-report measure of fear, avoidance, and physiological discomfort in social situations. The measure assesses past-week discomfort on a 5-point scale ranging from 0 (not at all) to 4 (extremely).

Data analysis

All analyses were conducted using IBM SPSS Statistics, version 25. First, we report descriptive information about the sample. Then, mixed-effects models were used to analyze symptom change over the course of treatment in this naturalistic setting. We used the linear mixed-model procedures outlined in Heck, Thomas, and Tabata (2014) and Shek and Ma (2011). Measures of anxiety symptom severity (BAI, GAD-7, OCI-R, PDS-5, PDSS, PSWQ, and SPIN), and secondary measures, including depressive symptoms (BDI-II), quality of life (Q-LES-Q-SF), and disability (SDS), were collected at baseline, midtreatment, and posttreatment. Analyses were intent-to-treat and included all participants (completers and noncompleters), as mixed-effects models are robust to missing data due to dropout. One hundred twenty-one participants (24.7%) did not complete treatment as recommended by their therapists in this study and were considered treatment dropouts.

In order to examine symptom reduction across measures with different scales, we computed the percent of maximum possible for all anxiety symptom measures as well as all secondary measures (see Supplement for details; Moeller, 2015). Prior to analyzing symptom change over time, model fitting was performed to determine the best fitting shape of the growth trajectory in each measure (see Supplement Table 1). Additionally, we controlled for the number of treatment sessions due to the variable number of sessions completed by each participant in this naturalistic setting. Change over time in each symptom measure was analyzed in separate models in the form: Symptom measure = number of sessions + time.

Finally, to investigate disorder-specific symptom reductions, we examined change in symptom measures in four diagnostic groups with sufficient sample sizes to be tested individually (OCD, PTSD, GAD, and SAD). For all analyses, we report the RC as a measure of clinically significant change. RC scores greater than 1.96 are equivalent to a p value < .05, suggesting that the change is statistically reliable and unlikely due to measurement error (Jacobson & Truax, 1991). We also include effect sizes calculated with Cohen’s d for all measures. Notably, due to the lack of a control group in this data set, effect sizes may be inflated as RCTs typically calculate the effect size as the pre–post difference in symptom severity minus that of the control group.

Results

Descriptive information about the sample and treatment

Demographics of the sample are shown in Table 1. The most common primary diagnosis in this sample was OCD, followed by GAD, PTSD, SAD, panic disorder, and specific phobias (for the percentages of each primary, secondary, and tertiary diagnosis, see Supplement Table 2). During the intake interview, 17.0% of the patients reported that they were taking psychiatric medications, and 16.2% reported they had taken psychiatric medications in the past. In terms of therapy, 14.9% reported being in therapy at the time of the intake and 54.2% had previously been in treatment. Of the sample, 13.1% had previously been in a psychiatric inpatient, residential, or intensive outpatient program. On average, 66.1% of therapists provided treatment once per week, 9.6% provided treatment twice per week, and 7.4% provided treatment less than once per week. Of the therapists, 46.0% reported that the standard length of the session was 60 minutes, while 38.0% reported providing 90-minute sessions. The number of sessions completed by posttreatment was mostly commonly reported as 11–15 sessions (28.0%) or 7–10 sessions (23.3%). Of the sample, 24.7% did not complete treatment (noncompleters = 121, completers = 368). Completers and noncompleters did not significantly differ in terms of age, gender, anxiety symptoms, depressive symptoms, quality of life, or degree of disability (p values ≥ .228).

Change over the course of treatment

The number of patients who completed each outcome measure at each time point are shown in Table 2. Using a composite score of all anxiety symptom measures calculated as the percent of maximum possible, patients with a primary anxiety diagnosis showed significant reductions in anxiety symptoms over treatment, B = –7.41, SE = .43, t (279.22) = –17.07, p < .001, RC = 2.47, d = 1.06. The results show a relatively linear decline in anxiety symptoms from baseline to posttreatment (see Figure 1 A). Over the course of treatment, patients with a primary anxiety disorder, PTSD, or OCD diagnosis significantly improved on individual measures of anxiety symptoms, with a pre–post mean percentage improvement of 44.6%, surpassing the threshold for significant RC and demonstrating a large effect size (greater than 1 standard deviation). In other words, anxiety symptoms were reduced by almost half on average, and this change was both reliable and clinically significant.

Table 2.

Means and Standard Deviations for Each Measure at Baseline, Midtreatment, and Posttreatment

	Baseline			Midtreatment			Posttreatment
	N	M	SD	N	M	SD	N	M	SD
Anxiety composite	454	37.67	16.99	291	29.13	17.28	237	20.87	14.59
1. OCI-R	451	16.35	12.94	281	13.22	11.49	235	8.65	8.31
2. GAD composite	453	63.22	26.14	283	48.15	27.48	234	37.3	25.98
a. GAD-7	307	12.47	5.87	191	8.81	5.93	164	6.31	5.21
b. PSWQ	146	61.64	12.54	92	55.04	13.29	70	50.74	12.91
3. PDS-5	190	27.54	20.92	123	23.55	20.61	87	12.85	13.47
4. Panic composite	450	33.31	24.57	284	21.78	19.1	234	15.83	16.83
a. PDSS	304	7.84	6.98	196	5.92	5.67	164	3.9	4.66
b. BAI	146	27.98	12.38	88	14.63	10.28	70	12.77	10.42
5. SPIN	450	22.58	16.81	291	19.19	14.87	231	13.64	13.32
BDI-II	453	20.71	12.63	277	12.82	12.17	232	9.67	10.41
Q-LES-Q-SF	302	42.38	9.63	188	46.73	10.35	159	50.06	10.35
SDS	442	15.84	7.05	277	9.77	7.31	228	7.82	6.74

Note. Anxiety composite, GAD composite, and panic composite mean scores are represented by calculated percent of maximum possible scores, and all other mean scores are derived from the raw scores on each individual measure. OCI-R = Obsessive-Compulsive Inventory—Revised; GAD = generalized anxiety disorder; GAD-7 = Generalized Anxiety Disorder–7-Item; PSWQ = Penn State Worry Questionnaire; PDS-5 = Posttraumatic Diagnostic Scale for DSM-5; PDSS = Panic Disorder Severity Scale; BAI = Beck Anxiety Inventory; SPIN = Social Phobia Inventory; BDI-II = Beck Depressive Inventory–II; Q-LES-Q-SF = Quality of Life Enjoyment and Satisfaction Questionnaire—Short Form; SDS = Sheehan Disability Scale.

Figure 1.

Patients with any primary anxiety diagnosis showed significant improvement in (A) anxiety symptoms, (B) depression symptoms, (C) quality of life, and (D) disability over the course of treatment.

Patients also demonstrated significant reductions in their depressive symptoms, B = –22.56, SE = 1.45, t (338.74) = –15.57, p < .001, RC = 2.18, d = .95, which showed a more curvilinear decline. Figure 1 B shows that the greatest decline in depressive symptoms occurred between baseline and midtreatment. Additionally, quality of life increased over the course of treatment, B = 5.84, SE = .59, t (188.85) = 9.94, p < .001, RC = 2.13, d = .77 (see Figure 1 C). Patients also showed a significant reduction in disability, B = –36.30, SE = 2.13, t (284.22) = –17.06, p < .001, RC = 2.42, d = 1.16, in a similar curvilinear pattern as depressive symptoms (see Figure 1 D). Patients showed significant improvements in these secondary outcomes with a pre–post mean percentage improvement of 53.3% for depression, 27.0% for quality of life, and 50.6% for functioning.

Change in diagnosis - specific measures

Four disorders (OCD, PTSD, GAD, and SAD) had sufficient sample sizes at each time point to examine change in the relevant diagnosis-specific measure. In those with a primary diagnosis of OCD, OCI-R scores decreased linearly over the course of treatment, B = –8.33, SE = .75, t (116.41) = –11.08, p < .001, RC = 2.19, d = 1.18, as can be seen in Figure 2 A. PTSD patients also showed a significant linear decrease in PDS-5 scores from baseline to posttreatment, B = –17.30, SE = 1.81, t (38.55) = –9.58, p < .001, RC = 5.81, d = 2.01 (see Figure 2 B). Additionally, of those with a primary diagnosis of GAD, generalized anxiety symptoms as measured by the PSWQ or GAD-7 decreased significantly over the course of treatment, B = –28.29, SE = 5.68, t (46.45) = –4.98, p < .001, RC = 2.08, d = .83. However, unlike OCD and PTSD, the GAD symptoms declined in a more curvilinear pattern (see Figure 2 C). SAD patients showed a significant decrease in SPIN scores in a fairly linear manner with treatment, B = –10.11, SE = 1.41, t (28.70) = –7.16, p < .001, RC = 2.80, d = 1.04 (see Figure 2 D). In addition, those with a primary diagnosis of OCD, PTSD, GAD, or SAD significantly improved on their disorder-specific measures over the course of treatment, with PTSD showing the greatest amount of improvement (mean percentage improvement of 68.7%) while OCD, GAD, and SAD showed mean percentage improvements of 54.6%, 32.2%, and 39.5%, respectively.

Figure 2.

Patients with a primary diagnosis of (A) OCD (obsessive-compulsive disorder) symptoms, (B) PTSD (posttraumatic stress disorder) symptoms, (C) GAD (generalized anxiety disorder) symptoms, or (D) SAD (social anxiety disorder) symptoms show significant reductions in the relevant disorder-specific measure over the course of treatment.

Discussion

This study examined the primary and secondary outcomes for a large naturalistic sample of treatment-seeking patients receiving CBT with an emphasis on exposure-based treatments for anxiety disorders (OCD and PTSD). The sample was greater in size than most prior studies, presented with considerable comorbidity, and was fairly severe, as evidenced by high baseline primary diagnosis measure scores, high disability scores, and poor quality of life before the start of treatment. Thus, the sample presented with several clinical characteristics strongly resembling “real-world” samples (Leichsenring, 2004;Stewart & Chambless, 2009). In addition, almost half of the sample completed treatment lengths that were in line with prescribed treatments (11–20 sessions; Hunsley & Lee, 2007). The sample demonstrated a dropout rate of 24.7%, which is similar to research study samples using exposure protocols (e.g., Foa et al., 2005;Hunsley & Lee, 2007) and precisely within range of the average pooled dropout rates (18–29%) reported in large meta-analytic studies for RCTs of anxiety disorders (OCD and PTSD; Carpenter et al., 2018;Imel, Laska, Jakupcak, & Simpson, 2013). We had expected that being able to offer specialty, evidence-based treatment for anxiety symptoms (OCD and PTSD) would allow us to retain more patients in this setting, but this was not the case. Regardless, our dropout rate was similar or within the range found in previous meta-analyses examining nonrandomized effectiveness studies of CBT for anxiety disorders (Hans & Hiller, 2013;van Ingen et al., 2009), further supporting the notion that our sample was comparable to other studies in this regard.

Over the course of treatment, patients with a primary anxiety disorder, PTSD, or OCD diagnosis significantly improved on individual measures of anxiety symptoms. Anxiety symptoms were reduced by almost half on average, and this change was both reliable and clinically significant. In addition, those with a primary diagnosis of OCD, PTSD, GAD, or SAD significantly improved on their disorder-specific measures over the course of treatment, with PTSD showing the greatest amount of improvement. This finding is in line with previous studies with considerably smaller samples (Foa et al., 2005;Norton & Price, 2007;Stewart & Chambless, 2009;Wootton et al., 2015), but the analytic methods used here are more sophisticated than previous analyses, allowing for better accounting of missing data and more accurate measurement of the pattern of changes in outcomes over time. Importantly, patients also showed significant improvements using this analytic approach in crucial secondary outcomes, including depression, quality of life, and functioning, which have typically not been examined in such detail. Indeed, as our field increases its recognition for the value of understanding more holistic improvements in patients that extend beyond simply symptom reduction (e.g., Asnaani et al., 2017;Hofmann et al., 2014), the findings of reliable and clinically significant improvements in these secondary outcome measures as a result of CBT are important. Specifically, such findings build further support for the wider benefits of using evidence-based treatments, such as exposure protocols in naturalistic settings.

The current study raises several important issues as we continue to struggle in our efforts to improve implementation of exposure protocols and other evidence-based treatments in the community. As noted elsewhere (e.g., Asnaani et al., 2018), despite considerable support for such therapies in RCTs, therapists in naturalistic and community settings have been reticent to adopt such treatments. One major barrier revolves around the belief that such treatments lack the ability to address the problems of “real-world” patients and that the samples in which they have been studied for research studies are overly manufactured and far too exclusionary (Hunsley & Lee, 2007;Leichsenring, 2004;Stewart & Chambless, 2009). The current study’s results serve to mitigate these concerns, and support the effectiveness of exposure treatments in patients similar to who practitioners see in their offices in community settings. In addition, the hospital-based clinic in which these data were examined did not constrain providers (who were of all different expertise levels and training backgrounds) in the length or format in which exposure treatments were delivered. The clinic also did not restrict providers from incorporating other treatment skills from nonexposure CBTs, such as mindfulness, cognitive therapy, and distress tolerance skills, into their primarily utilized exposure protocols, allowing practitioners to be as flexible as they would be in community settings. From that perspective, this study greatly adds to the small but growing number of studies examining naturalistic outcomes, and importantly, other outcomes beyond symptom reduction that matter in anxiety (OCD and PTSD) samples, such as functioning and quality of life (e.g., Asnaani et al., 2017;Hofmann et al., 2014;Schnurr et al., 2006).

Despite the utility of the current findings in addressing the current research–practice divide, the present study is not without its limitations. First, the sample was not particularly diverse racially, which may not be representative of some community samples. This issue was likely due to the fee-for-service structure and acceptance of only limited insurance plans in order to receive treatment. However, the sample was diverse on other demographic measures (e.g., education level, employment status, and relationship status). In addition, while baseline data collection was quite high (near 93.3%) because this was part of the normal intake process, response rates for midtreatment and posttreatment data were considerably lower (near 56.4% and 48.4%, respectively). While the analytic approach employed allowed us to minimize the adverse effects of such missing data, this occurrence is certainly reflective of studies with naturalistic samples in general (Hans & Hiller, 2013;van Ingen et al., 2009). Due to the less structured and rigid evaluative process in naturalistic settings, it can be expected that fewer data will be acquired, particularly in comparison to RCTs that have stricter data collection requirements in order for patients to retain their place in treatment (e.g., Schulz, Altman, Moher, & the CONSORT Group, 2010). Another limitation to naturalistic studies is the lack of a control group, which may result in inflated effect sizes as RCTs typically calculate the effect size as the pre–post difference in symptom severity minus that of the control group. Additionally, as the treatments used in naturalistic settings are varied and tailored to the patient’s specific concerns, this precluded us from analyzing the effect of specific treatment protocols on outcomes, which would be an informative variable for future similar studies to examine more closely.

That said, the use of a computerized data collection tool that sent automated reminders to patients and allowed them to fill out measures in several sittings increased the accuracy and convenience of collecting information on how patients are progressing. It is possible that the battery of measures could be daunting given its size—however, our high compliance rate at intake would suggest that patients are more than willing to fill out these measures. Indeed, many patients in our clinic report interest in seeing their results at each assessment point, which may increase patient engagement in their own treatment. Finally, all of the measures were self-report, and this format may be particularly susceptible to a variety of biases, including response bias (Safer & Keuler, 2002). However, self-report measures also show high association with clinician-administered measures, while being considerably less burdensome to use in low-resourced naturalistic settings (e.g., Asnaani et al., 2017;Sulkowski et al., 2008). Current studies are under way at our clinic to incorporate several autonomous, brief, and objective computerized tasks to supplement these self-reported data, and to examine the potential mechanisms underlying better or worse outcome in our naturalistic sample.

Conclusions

Overall, this study showed significant effectiveness of short-term, evidence-based treatment for anxiety and related disorders delivered in a flexible manner in a sample that showed considerable diagnostic severity and comorbidity. Such improvements were observed both in terms of symptom reduction and in more global outcomes, such as life functioning and quality of life, and were achieved in the context of treatment with providers of a range of experience levels. Further, while this was not an explicit emphasis in the current paper, it is notable that this study was more broadly conceptualized as a way to examine how clinical research and outcome evaluation can be seamlessly integrated into routine clinical practice using a computer-based platform. Given the emphasis on disseminating and implementing EBPs in community settings, it was regarded as important to evaluate the naturalistic outcomes of exposure therapy using a technology-enhanced measurement across a comorbid and representative treatment-seeking sample.

Further, this study serves as a model for how repeated evidence-based assessment can be helpful over the course of routine clinical care—therapists consistently reported using the scores for a variety of outcomes at mid- and posttreatment (which were easily retrievable from the online REDCap data collection portal) to provide feedback to patients and to guide continued treatment or relapse prevention planning. To reduce burden on patients due to filling out large batteries at multiple times through treatment, clinicians in community clinics could opt to simply have patients fill out the few measures they need and still use this or a similar computer-based platform to obtain information on patients at several points over the course of treatment in order to continually inform evidence-based care and treatment planning.