The Impact of Social Skills Training For Social Anxiety Disorder: A Randomized Controlled Trial

Abstract

Objective:

Social anxiety disorder (SAD) impacts social, occupational and academic functioning. Although many interventions report change in social distress, improvement in social behavior remains under-addressed. This investigation examined the additive impact of social skills training (SST) for the treatment of SAD.

Method:

Using a sample of 106 adults who endorsed SAD across numerous social settings, participants were randomized to exposure therapy (imaginal and in vivo) alone, a combination of SST and exposure therapy known as Social Effectiveness Therapy (SET), or a wait list control. The assessment strategy included self-report measures, blinded clinical ratings and blinded assessment of social behavior.

Results:

Both interventions significantly reduced distress in comparison to the wait list control and at post-treatment, 67% of patients treated with SET and 54% of patients treated with exposure therapy alone no longer met diagnostic criteria for SAD, a difference that was not statistically significant. When compared to exposure therapy alone, SET produced superior outcomes (p<.05) on measures of social skill and general clinical status. In addition to statistical significance, participants treated with SET or exposure reported clinically significant decreases on two measures of self-reported social anxiety and several measures of observed social behavior (all ps < .05).

Conclusions:

Both interventions produced efficacious treatment outcome, although SET may provide additional benefit on measures of social distress and social behavior.

Introduction

Social anxiety disorder (SAD) is a marked and persistent fear of scrutiny in social or performance situations (American Psychiatric Association [APA], 2013). Individuals who experience social distress across a broad range of social settings¹ have severe social and general anxiety, social inhibition, fear of negative evaluation, avoidance, fearfulness, and self-consciousness and may account for up to 70% of patients seeking treatment (e.g., see Beidel & Turner, 2007 for a review).

Without treatment, SAD results in long-term functional impairment, but evidence based interventions do exist. Meta-analytic and qualitative reviews (Butler, Chapman, Forman, & Beck, 2006; Hofmann, 2010; Jørstad-Stein, & Heimberg, 2009; Ponniah & Hollon, 2008) and recent individual comparative trials (Clark et al., 2006; Mörtberg, Clark, Sundin, Åberg Wistedt, 2007; Rapee, Gaston, & Abbot, 2009; Stangier, Schramm, Heidenreich, Berger, & Clark, 2011) suggest that cognitive behavioral interventions are efficacious treatments for SAD, based on self-report and clinician ratings of improvement. Despite these positive reports, enthusiasm for current CBT outcomes must be tempered by several important limitations. First, statistically significant symptom improvement does not meet the threshold for diagnostic remission and second, outcome assessment strategies that fail to objectively assess social behavior change does not allow an assessment of changes in functional impairment. Specifically, extent outcome data are reliant on self-report and clinician ratings, which document that CBT results in perceived decreases in social distress (e.g., Clark et al., 2006; Mörtberg et al., 2007; Strangier et al., 2011). Few studies have examined actual changes in impaired social functioning/behavior, which is an important element in SAD’s clinical presentation (Beidel, Rao, Scharfstein, Wong, & Alfano, 2010). Even among the few investigations that included behavioral tasks in their assessment battery, most used the tasks only to assess social anxiety, not social behavior (Clark et al., 2006; Herbert, Gaudiano, Rheingold, Myers et al., 2005; Rapee et al., 2009).

Given the plethora of available treatment trials for SAD, why the lack of attention to assessing objective social skill? First, conducting observational assessments is clearly more challenging and time-intensive than completion of subjective measures. Another reason, however, is that some conceptualizations of SAD begin with the premise that people with SAD possess adequate social skills but their ability to focus on social interactions and use the skills appropriately is hindered by anxiety. This suggests that SAD is associated with a performance deficit, not a skill deficit (Hopko, McNeil, Zvolensky, & Eifert, 2001). Theoretically then, eliminating social anxiety should allow for adequate/appropriate social skills to emerge, but few studies have directly addressed this issue. One investigation (Hope, Herbert & White, 1995) reported that group CBT (with no formal social skills training) improved social skills across both DSM-IV generalized and non-generalized subtypes. This would suggest support for the performance deficit model, but the small sample size and the limited assessment of social skill (a one item Likert scale) limits the conclusions of this investigation.

A recent review (Poniah & Hollon, 2008) reported that social skills training (SST) alone is not efficacious for improving social skills in adult SAD. This conclusion would be consistent with the accepted practice that exposure therapy is an essential component of treatment for anxiety disorders (Craske, Treanor, Conway, Zbozinek, & Vervliet, 2014). However, extant efficacy data for adding SST to established treatments are contradictory. On one hand, Stravynski et al. (2000) reported that SST did not enhance treatment outcome to an interpersonal approach and the majority of individuals remained symptomatic at outcome. In contrast, Turner et al. (1994) treated thirteen individuals with SAD were treated with SST (12 sessions) followed by exposure therapy (12 sessions). Patients showed significant improvement on measures of social anxiety/distress as well as improvement in social behavior. Blinded ratings indicated that social effectiveness and social skill improved after SST but before exposure therapy. After additional exposure therapy, gains were maintained, but there was no further improvement in social skill. However, the small number of participants and the lack of randomization prohibit drawing firm conclusions about the additional nature of SST.

A more recent randomized controlled trial (Herbert et al., 2005) compared group CBT (CBGT) to SST plus CBGT. In the combined condition, SST included education, modeling, and behavior rehearsal in the context of the simulated exposure exercises, feedback and cognitive restructuring that is characteristic of CBGT. The results indicated that adding SST enhanced outcome over CBGT alone. Blinded observer ratings of social skill revealed statistically significant differences favoring the combined group, and significantly more individuals treated with the CBGT plus SST were judged as treatment responders when compared to CBGT alone (79% vs 38%, respectively); at 3 month follow-up, the difference remained but was no longer significant (83% vs 57%). As the authors noted, despite these improvements, post treatment scores on a self-report inventory of social anxiety fell well above the mean for non-clinical samples, suggesting continuing impairment, and the need to continue the search for efficacious treatment strategies. Although the less than optimal outcome might be due to a myriad of factors, one important consideration is that the addition of SST resulted in less time being devoted to other elements of the treatment package. Optimally, a comparative treatment trial should assess all treatment elements at full strength.

To summarize, current interventions for SAD have focused primarily on CBT in various iterations but most studies do not directly address how these interventions affect impaired social functioning. This is significant shortcoming in the existing literature because functional impairment is now a critical factor when determining the presence/absence of a psychiatric disorder (American Psychiatric Association [APA], 2013). Additionally, the inability to behave as desired is perhaps the reason why most individuals seek treatment. Furthermore, not all individuals with SAD respond to CBT suggesting that alternative strategies are necessary. Although Herbert et al. (2005) provide evidence that SST may enhance treatment outcome, those findings require replication and the interventions (including exposure therapy) must be provided at optimal strength.

In this investigation, we compared exposure therapy (EXP), a well-established treatment for SAD to a multi-faceted intervention (group social skills training plus individual exposure), known as Social Effectiveness Therapy (SET) for people with SAD. In addition to self-report and blinded clinician ratings, we attempted to address several limitations in the extant literature. First, we included direct observation of behavioral skill using several different behavioral tasks. Second, unlike most previous investigations, we assessed clinical significance as well as statistical significance, using a normative comparison group. We hypothesized that (a) both SET and EXP would produce positive treatment outcome when compared to wait list control, (b) SET would produce superior treatment outcome to EXP alone, particularly on measures of observed social behavior, and (c) treatment gains would be maintained at follow-up.

Method

Participants

The protocol was approved by the University IRB. Study personnel explained the project verbally to each participant, who was then given time to review the written consent. Questions were answered and no part of the study protocol was conducted until the participant signed the consent form.

One hundred nineteen (119) adults with SAD who participated in a study examining social skills deficits in SAD (Beidel et al., 2010) were invited to participate in the treatment program. Participants were recruited via clinician referral or newspaper advertisements and following an initial telephone screen, were interviewed by doctoral level psychologists or doctoral students in clinical psychology using the Structured Clinical Interview for DSM-IV (SCID; First, Spitzer, Williams & Gibbons, 1997) and the Structured Clinical Interview for DSM-IV Axis II (SCID-II; First, Gibbons, Spitzer, & Williams, 1997). SAD had to be the primary diagnosis and symptom duration had to exceed 6 months. Diagnostic exclusions included the presence of psychosis, bipolar disorder or depressive disorder with active suicidal ideation and Axis II diagnoses of Borderline, Schizoid, Paranoid, or Schizotypal Personality Disorder. All other comorbid diagnoses were included. Participants on selective serotonin reuptake inhibitors (SSRIs) were allowed to continue on their medication as long as the dosage remained stable throughout the treatment phase. Three (3) participants were excluded on the basis of comorbid depression with active suicidal ideation and 10 potential participants chose not to enter the treatment protocol. Twenty percent of the diagnostic interviews were videotaped and rated by a second clinician for the purposes of calculating inter-rater reliability. For the diagnosis of SAD, agreement was excellent (κ=.92).

Of the 119 potential participants, 106 were randomized to a treatment condition (see Figure 1, CONSORT diagram). Participants ranged in age from 19 to 78 years (mean age = 36.39, SD = 13.99). Demographic data and clinical symptom data, including comorbid disorders are presented in Table 1 by group. The mean Clinical Global Impressions Scale – Severity subscale rating was 5.16 (markedly ill), and on an 8 point Likert scale assessing behavioral avoidance, the average rating was 4.43, indicating moderate to severe avoidance of social encounters. Eleven percent (11%) of the participants were taking antidepressant medication during the study.

Figure 1.

CONSORT 2010 Flow Diagram

Table 1.

Means and Frequencies for Demographic and Clinical Characteristics

	SET (n=46)	EXP (n=41)	WL (n=19)
Age	36.7 (14.4)	36.1 (13.8)	36.4 (14.2)
Sex
Male	21	23	7
Female	25	18	12
Race/Ethnicity
Non-Hispanic Caucasian	32	31	12
African American	8	4	5
Hispanic	2	3	0
Asian	3	2	2
Biracial	1	1	0
CGI Severity of Illness	5.2 (1.0)	5.2 (0.7)	5.2 (0.9)
Additional Axis I Dx
None	70%	78%	63%
GAD	4%	5%	16%
Specific Phobia	9%	2%	5%
OCD	2%	2%	0%
MDD	9%	2%	5%
Dysthymia	7%	7%	5%
Alcohol Abuse	0%	2%	0%
Axis II Dx
None	37%	29%	58%
Avoidant PD	57%	61%	42%
Obsessive-Compulsive PD	9%	10%	0%

Participants were randomized using a 2:2:1 randomization schedule to one of three groups: Social Effectiveness Therapy (SET, a combination of SST and exposure therapy; n=46), exposure therapy alone (n=41), or waitlist control (n=19). Sample size was calculated using an effect size of d =.70, and setting alpha at .05, resulting in a needed sample size of 33 subjects per active treatment group to yield a test with >80% power (Cohen, 1988). There were no demographic or clinical status differences across the groups (see Table 1).

Measures

Self-report instruments

Participants completed the Social Phobia and Anxiety Inventory (SPAI; Turner et al., 1989), which assesses severity of social phobia symptoms across a range of social and performance situations. The SPAI has high test retest reliability, has good concurrent, external, and discriminant validity (Beidel et al., 1989a; 1989b; Turner et al., 1989) and the ability to detect both statistically reliable and clinically significant treatment changes (Beidel et al., 1993).

Clinician ratings

Doctoral level clinicians completed the 7-point Clinical Global Impressions Scale (CGI; Guy, 1976) severity and global improvement subscales (the latter was not completed at pretreatment) as well as an 8 point rating scale of behavioral avoidance. Clinicians also completed the Brief Social Phobia Scale (BSPS; Davidson et al., 1991), an 11-item clinical rating scale designed to rate fear and avoidance of seven common social evaluative situations and four common physiological symptoms and the Hamilton Rating Scales for Anxiety (HAMA) and Depression (HAMD; Hamilton, 1959, 1960). At post-treatment and all follow-up assessments, ratings were completed by interviewers blinded to assessment time (post vs follow-up) and treatment condition.

Social skill assessment

Three behavioral tasks provided a comprehensive assessment of social skills. The three tasks selected allowed for an assessment of molecular and molar social skills across various social settings. The first task used role play interactions with a trained confederate in 8 different social scenarios (expression of disapproval or criticism, social assertiveness, confrontation and anger expression, opposite sex interaction, interpersonal warmth, conflict or rejection, interpersonal loss, receiving compliments; Richardson & Tasto, 1976). For each interaction (approximately 3 minutes each), the examiner described the scene, the confederate read a prompt, and the participant responded, followed by a second prompt and participant response. Four of the interactions were with a male confederate and 4 scenes with a female confederate.

Second, there were two Unstructured Conversation Tasks (UCT; Turner et al., 1994), one involving interaction with an opposite sex confederate (“pretend you are at a dinner party and get to know the person next to you”) and one with a same sex confederate (“you just moved into a new house and see your neighbor in the back yard”). Each scenario was 3 minutes in length. Unlike the SSIT, there were no specific prompts and confederates were trained to respond to the participant, but not to assume the burden of the conversation.

Third was the Impromptu Speech Task (IST). Participants were asked to deliver a 10 minute impromptu speech using up to 3 topics (provided by the experimenter) to a 3 person audience. Participants were given 3 minutes to prepare their speech and could terminate the speech anytime after 3 minutes by holding up a stop card (see Beidel et al., 1989c).

In addition to the ratings by independent observers (described below), each participant rated their anxiety at baseline and after the SSIT, UCT and IST using a 9 point Likert rating scale, where 1=no distress and 9=extreme distress.

Ratings of Behavioral Skills

Assessments were videotaped and rated by independent raters unaware of diagnostic status or phase of treatment. Each interaction was rated for participant’s degree of anxiety and skill using a 5-point Likert scales. Higher ratings reflected better skill. Lower ratings reflected less anxiety. Additionally, for the IST, speech duration was timed (in seconds) and the percentage in each group able to complete the speech was calculated. To determine inter-rater reliability, 25% of all assessment videotapes were rated by a second independent rater. All reliability coefficients were above r =.80.

Using these three behaviors, self-rated anxiety, observer-rated anxiety, and observer-rated skill, we calculated three latent factor scores (BAT Self-rated Anxiety Factor, BAT Observer-rated Anxiety Factor, BAT Observer-rated Skill Factor). Latent factor scores are considered the gold standard for use in outcomes analyses because they substantially increase power by (a) decreasing the number of variables to be analyzed, and (b) eliminating random and task-specific error (i.e., only shared variance attributable to the construct of interest is included in the factor score; Shipstead, Redick, & Engle, 2010). Factor scores are interpreted as z-scores. BAT IST Total Time was the total duration of each participant’s IST speech.

Treatment Credibility

Four questions (Borkovec & Nau, 1972) assessed treatment credibility scales (10 point Likert rating scales): how logical the treatment appears, confidence in the treatment, expectancy of success, and estimate of the treatment’s success for a different fear. Both active interventions were rated as equally logical (M[SET]=7.9 vs. M[EXP]=7.2, p>.05), had equal treatment confidence ratings (M[SET]=6.7 vs. M[EXP]=6.7, p>.05), had equal ratings regarding expectancy of success (M[SET]=7.9 vs. M[EXP]=7.5, p>.05), and were rated as being equally successful for treating a different fear (M[SET]=7.5 vs. M[EXP]=7.0, p>.05).

Treatment Responder

The primary outcome variable was designated at the percentage in each group who did not meet diagnostic criteria at post-treatment. However, consistent with many randomized controlled trials, we used a second primary outcome variable to allow study results to be compared to other RCTs. We defined a treatment responder was defined a priori as a participant who was rated by the independent evaluator as very much improved (1) or much improved (2) on the CGI Improvement Scale at posttreatment.

Treatment

Social Effectiveness Therapy (SET; Turner et al., 1994)

SET is a multi-component behavioral program specifically designed to decrease social anxiety, improve interpersonal skill, improve social performance (i.e., public speaking skill), and increase participation in social activities. SET consisted of twice weekly sessions (one group SST, one individualized imaginal exposure).

SET includes 12 individual and 12 group treatment sessions, which run concurrently for a total of 24 sessions over 12 weeks. SST occurred weekly in groups of 4-6 participants and topics included basic conversational skills, establishing new friendships and social networks, assertiveness, and effective public speaking. SST used a standard skills training format including instruction, modeling, behavioral rehearsal, and feedback. Each group was 90 minutes in duration and was led by two therapists. In addition to group sessions, participants were assigned homework designed to allow the skills to generalize to their daily social interactions. These practice assignments were short in duration (about 10 minutes per day), and thus, did not constitute exposure sessions (e.g., habituation of anxiety was not the goal).

The second component of SET is exposure, which was conducted once weekly in individual sessions. The exposure sessions, which averaged 90 minutes in length, consisted of scenes designed to address the patient’s unique fears. Exposure was not hierarchical in nature, but used a flooding format where the patient was immediately exposed to the most extreme fear. The session was continued until habituation occurred. Exposure was conducted using an imaginal format. Although a number of investigations have reported positive outcome using in vivo exposure for the treatment of SAD, there are some instances where imaginal exposure is necessary/preferable. Specifically, efficacious exposure treatment requires the inclusion of all aspects of the fear, including all associated parameters (Lang, 1968). For many individuals with SAD, this includes the need to address fears of rejection, ridicule or disapproval from others. For instance, one patient, a lawyer avoided appearing in court due to fear that he would be unable to speak or would physically stumble, and his supervisor, who would be sitting in the court room, would decide he was incompetent and fire him. Clearly, it would be difficult to adequately address all aspects of this fear in vivo and as noted by others (Craske & Rachman, 1987), failure to capture all facets of the fear could result in incomplete habituation and ultimately, a return of fear.

Exposure (flooding) treatment

Patients randomly assigned to this condition received individualized imaginal exposure as described above twice weekly for the first 8 weeks. During weeks 9-12, patients were seen once per week for imaginal exposure, and once weekly for programmed practice/in vivo exposure (total of 24 sessions over 12 weeks; 20 imaginal and 4 programmed practice/in vivo exposure sessions). Programmed practice/in vivo exposure consisted of therapist directed exposure assignments developed in session and designed to transfer the gains made in individual flooding sessions to the patient’s every day environment. Thus, participants in both groups had equivalent therapeutic contact (2x/week for 12 weeks).

Wait list control

Patients were told that they would be treated after a 12-week wait period, at which point they would be able to select either SET or EXP. During the 12 week wait period, participants were telephoned weekly to assess their clinical status. No patient was removed from the wait list control condition because of worsening clinical status.

Therapists

Therapists were doctoral level psychologists or doctoral students in clinical psychology with previous experience conducting social skills training and exposure therapy, supervised by the first author. Clinicians provided treatment in both conditions, following standardized treatment manuals.

Treatment integrity

All treatment sessions were recorded and 20% were randomly selected and rated for adherence to the treatment manual. Blinded raters used a checklist of all elements of all treatment conditions (e.g., modeling of social skill, instruction to close eyes and imagine a scene, request to report rating using the Subjective Units of Distress Scale (SUDS), inquiry as to general well-being over the last week), with no crossover of treatment effects (100% integrity).

Results

Planned analyses

Outcome data were analyzed with a series of χ² comparisons and Mixed-model ANOVAs to examine change as a function of group (WL, SET, EXP). Follow-up analyses examined maintenance of treatment gains for SET and EXP groups. Multiple imputation methods using SPSS 19 Missing Data Module were used across all study waves to address attrition and missing data (Rubin, 1987; Schafer & Graham, 2002). Multiple imputation (MI) uses all available data, including non-completers’ pretreatment data, and all the completers data, to estimate likely values for each outcome for each participant. For these analyses presented, the computer ran the imputation 10 times, creating 10 unique datasets, meaning that each dataset had different values imputed for each missing data point based on the estimation procedures. The data analytic procedures are then run for each dataset, and then combined across the 10 sets of results using Rubin’s (1987) rules. Multiple imputation methods lead to less biased/more accurate results relative to single imputation, last observation carried forward, and complete case analysis (Schafer & Graham, 2002; Sterne et al., 2009). For each imputation, participant age, gender, and clinician-rated pre-treatment ADIS symptom severity ratings were used as additional predictors to improve imputation precision (Sterne et al., 2009). In the final step, results from these 10 analyses were combined using Rubin’s rules for combining estimates obtained from multiple imputed datasets (Rubin, 1987).

Post-treatment Comparison of the WL, SET, and EXP Groups

Attrition

It is important to note that even when data are missing not at random, MI approaches are valid as long as any systematic differences between completers and noncompleters are included in MI models (e.g., different attrition rates for men and women). For this analysis, all available data (see below) were included in the multiple imputation analysis in order to get the best estimate of how dropouts would have fared in treatment.

Based on the number of participants randomized to a treatment condition (N = 106), 71 individuals completed treatment and the post-treatment assessment (66%). Completers did not differ from noncompleters in age, sex (proportion of females), number of children, race/ethnicity, or any of the 10 pre-treatment clinical indicators (all p > .23). In addition, attrition was not significantly related to treatment condition, χ2 [2] = 2.85, p = .24. Collectively, these findings were interpreted as evidence that a missing at random (MAR) assumption was defensible. Nevertheless, it is impossible to know whether non-completers differed from completers during waves at which the former were not assessed. Therefore, all 106 participants who were initially randomized to the three conditions were included in an intent to treat analysis using multiple imputation to handle missing data according to the above methodological rationale.

Primary Outcome Variables

Treatment Responders (Presence/absence of social phobia)

There were significant group differences at post-treatment (χ² [2] = 21.30, p < .001), with significantly higher percentages of the SET (67%; d = 1.26) and EXP (54%; d = 0.95) groups no longer meeting diagnostic criteria for SAD compared to the WL group (10%; both χ² [1] ≥ 11.08, p < .001). The percentage of patients without a SAD diagnosis in the SET and EXP groups were not significantly different (χ² [1] = 3.68, p = .12; d = 0.42).

Alternative Treatment Responder Criteria (CGI ratings)

Dropouts were categorized as treatment responders or non-responders using their imputed scores. Based on CGI Improvement ratings, patients treated with SET or EXP were significantly more likely to be considered treatment responders when compared to the WL control group (χ2 [2] = 45.37, p < .0005); 92% of patients treated with SET (d = 2.68) and 75% for EXP (d = 1.61) compared to 6% for WL (6%; both χ2 [1] ≥ 23.55, both p < .001) met treatment responder criteria. There were significantly more treatment responders for SET relative to EXP (χ2 [1] = 4.30, p <.05; d = 0.46).

Secondary Outcome Variables

A series of 3 (group: WL, SET, EXP) × 2 (time: Pre, Post) Mixed-Model ANOVAs further examined group differences on secondary outcome variables: (a) self-reported anxiety, (b) BAT objective ratings of anxiety and skill, and (c) clinician rated symptoms. For planned comparisons, pre-treatment scores were not significantly different across groups unless noted. For the few variables in which pre-treatment differences were apparent, pre-treatment score was used as a covariate to examine post-treatment group differences.

Self-reported anxiety

For the SPAI, BSPS, and BAT Self-reported Anxiety Factor, all omnibus between-group and interaction terms were significant (F [1-2, 103] range: 3.68 to 13.34, all p < .05; see Table 2). SET and EXP groups had lower scores than the WL group on all three measures (all p < .0005; SET vs. WL: d_range = 1.26 to 1.92; EXP vs. WL: d_range = 0.99 to 1.29). In addition, participants treated with SET had significantly lower scores than the EXP group on the BSPS (p = .01; d = 0.56) and BAT self-reported anxiety (p = .02; d = 0.46) but were not significantly different on the SPAI (p = .46; d = 0.16; see Table 2). As illustrated in Figure 2, both treatment groups had post treatment scores that fell below the cut-off score for clinical significance. Furthermore, the SET group had scores that fell below the normative sample (at post-treatment and six month follow-up).

Table 2.

Pre-post data for primary outcomes variables

	Pre-treatment				Post-treatment				Between- group F (1,103)	Omnibus Tests1 Within- group F (1, 103)	Pre-post Interaction F (2,103)
Outcome	WL	SET	EXP	Contrasts (Cohen’s d Effect Size)	WL	SET	EXP	Contrasts (Cohen’s d Effect Size)
Self-reported
BAT Self-reported Anxiety2,3	0.55 (0.20)	−0.18 (0.16)	−0.04 (0.16)	SET < WL (0.72) EXP < WL (0.60)	1.02 (0.25)	−0.50 (0.11)	0.10 (0.16)	SET < WL (1.79) EXP < WL (1.08)	19.28***	--	3.68*
				SET = EXP (0.13)				SET < EXP (0.46)
BSPS	45.26 (2.56)	42.79 (1.53)	43.46 (1.78)	WL = SET (0.23) WL = EXP (0.16)	37.52 (3.26)	17.07 (1.30)	22.44 (1.61)	SET < WL (1.92) EXP < WL (1.29)	12.74***	183.51***	13.34***
				SET = EXP (0.06)				SET < EXP (0.56)
SPAI	104.67 (5.82)	107.85 (3.13)	104.46 (4.4)	WL = SET (0.14) WL = EXP (0.01)	93.65 (5.92)	67.60 (2.68)	70.77 (3.38)	SET < WL (1.26) EXP < WL (0.99)	3.19*	91.83***	7.29***
				SET = EXP (0.14)				SET = EXP (0.16)
Clinician-rated
CGI Avoidance	4.32 (0.13)	4.39 (0.11)	4.53 (0.15)	WL = SET (0.11) WL = EXP (0.25)	3.84 (0.30)	1.01 (0.16)	1.78 (0.23)	SET < WL (2.50) EXP < WL (1.44)	25.13***	236.78***	25.13***
				EXP = SET (0.16)				SET < EXP (0.61)
CGI Severity	5.21 (0.21)	5.14 (0.16)	5.15 (0.11)	WL = SET (0.07) WL = EXP (0.08)	4.78 (0.29)	2.08 (0.15)	2.83 (0.21)	SET < WL (2.45) EXP < WL (1.49)	22.34***	200.84***	27.34***
				EXP = SET (0.01)				SET < EXP (0.63)
HAMA	18.42 (1.26)	15.53 (1.05)	16.27 (1.20)	WL = SET (0.43) WL = EXP (0.30)	12.92 (1.49)	5.06 (0.60)	6.89 (0.74)	SET < WL (1.62) EXP < WL (1.13)	8.67***	121.92***	3.17*
				EXP = SET (0.10)				SET < EXP (0.42)
HAMD3	11.16 (1.18)	8.19 (0.68)	9.32 (0.79)	WL < SET (0.62) WL = EXP (0.36)	8.98 (1.34)	2.10 (0.41)	4.96 (0.69)	SET < WL (1.77) EXP < WL (0.82)	11.35***	81.15***	5.41**
				EXP = SET (0.23)				SET < EXP (0.79)
Observer-rated
BAT Anxiety3	0.22 (0.20)	−0.38 (0.14)	0.31 (0.16)	SET < WL (0.64) WL = EXP (0.09)	0.07 (0.22)	−0.50 (0.16)	0.54 (0.21)	SET < WL (0.36) WL = EXP (0.40)	15.49***	--	1.60, ns
				SET < EXP (0.69)				SET < EXP (0.74)
BAT Skill3	−0.48 (0.20)	0.23 (0.15)	−0.06 (0.16)	WL < SET (0.73) WL = EXP (0.42)	−0.12 (0.26)	0.47 (0.15)	−0.44 (0.16)	WL < SET (0.38) WL = EXP (0.47)	8.65***	--	4.55*
				EXP = SET (0.29)				EXP < SET (0.87)
BAT Total Time2	3.56 (0.51)	4.81 (0.32)	4.10 (0.28)	WL < SET (0.57) WL = EXP (0.26)	3.45 (0.55)	5.89 (0.28)	5.33 (0.32)	WL < SET (1.04) WL < EXP (0.87)	9.83***	9.42**	2.60, ns
				EXP = SET (0.37)				EXP = SET (0.18)

Note: Values reflect X̄ (SE)

¹F values combined across 10 multiple imputation datasets according to Rubin’s rules (Schafer & Graham, 2002)

²Significant pre-treatment scores were entered as covariates for post-treatment comparisons

³Values reflect latent factors interpreted as z-scores, and reflect each group’s relative standing in relation to the other groups. Significant interaction effects indicate changes in the relative standing of groups across assessments. Within-subjects effects are not interpreted because scores are standardized at each assessment point.

^*,p < .05;

^**p < .01;

^***p < .00l

Figure 2.

Self-report ratings of anxiety by group during behavioral task

Objective measures of anxiety and skill

For the three objective anxiety and skill measures (BAT Observer Anxiety Factor, BAT Observer Skill Factor, and BAT Total Speaking Time during the IST), all omnibus between-group effects were significant (F[1, 103] range: 8.65 to 15.49, all p ≤ .001). Within-subjects effects were not interpreted because factor scores are standardized at each wave (i.e., mean = 0 and SD = 1 at each wave). The interaction term was significant for BAT Observer Skill (p = .02), reflecting changes in the relative standing of groups across pre- and post-treatment. Planned comparisons were conducted separately for each variable with pre-treatment scores as a covariate due to pre-treatment group differences. At post-treatment, the SET group demonstrated lower anxiety (d = 0.36), higher skill (d = 0.38), and longer speech duration (d = 1.04) than the WL group (all p ≤ .005; Figures 3, 4, and 5 present the Z scores for each group for each of these variables, allowing illustration of group changes across time). In addition, the SET group demonstrated lower anxiety (d = 0.74), higher skill (d = 0.87), and longer speech duration (d = 0.18) relative to the EXP group (all p ≤ .017). Only patients treated with SET had post-treatment scores that exceeded the clinical significance cut-off scores and attained ratings consistent with a normative sample. In contrast, the EXP group demonstrated higher anxiety (d = 0.40) and lower skill (d = 0.47), but longer speech duration (d = 0.87), relative to the WL group (all p < .049).

Figure 3.

Blinded observer ratings of anxiety by group during behavioral assessment tasks

Figure 4.

Blinded observer ratings of skill by group during behavioral assessment tasks

Figure 5.

Total speaking time by group during the impromptu speech task

Clinician-rated symptoms

For the Behavioral Avoidance, CGI Severity, HAMA, and HAMD (see Table 2), all omnibus between-group and interaction terms were significant (F [1-2, 103] range: 3.17 to 30.75, all p < .05). Planned comparisons revealed that the SET and EXP groups were rated as less avoidant, less anxious, less depressed, and having less severe symptomatology relative to the WL group (all p ≤ .003; SET vs. WL: d_range = 1.62 to 2.50; EXP vs. WL d_range = 0.82 to 1.49). In addition, the SET group was less avoidant (p = .005; d = 0.61), less symptomatic, (p = .005; d = 0.63), and less depressed (p ≤ .0005; d = 0.79) relative to the EXP group. SET-EXP group differences were non-significant for the HAMA (p = .05; d = 0.42).

Clinical significance

A final set of analyses examined the clinical significance of the treatment outcome (Jacobson & Truax, 1991), using a normative sample to calculate a cut-off score beyond which treatment-related change is considered clinically meaningful. Normative data was available for the BAT variables (Beidel et al., 2010) and the SPAI (Turner, Beidel, et al., 1989). At post-treatment, 97% of the SET and 85% of the EXP group reported clinically meaningful decreases in BAT self-reported anxiety, relative to 14% of the WL group (χ2 results: SET = EXP > WL, p < .0005). In addition, 57% of the SET group and 43% of the EXP group demonstrated clinically meaningful improvements in BAT total speech time, relative to 12% of the WL group (SET = EXP > WL, p ≤ .002). BAT observer ratings of anxiety and skill indicate that 67% and 79% of SET participants demonstrated clinically meaningful improvements on these variables respectively, relative to 27% and 43% of the EXP group, and 35% and 50% of the WL group (SET > EXP = WL, p ≤ .03). Finally, examination of SPAI scores revealed that 50% of SET participants, relative to 26% (EXP) and 11% (WL), demonstrated clinically meaningful improvements (SET > EXP = WL, p ≤ .05)(Figure 6).

Figure 6.

Clinical significance of treatment outcome using normative sample data

Three- and Six-month Follow-up of SET and EXP Groups

Attrition

Because the WL group was offered treatment after completion of the wait list, data were not available for follow-up. Of the 59 individuals who completed SET (n=32) or EXP (n=27) treatment, 63% provided data at 3-month follow-up, and 50% were available at 6 months. A series of χ2 and t-tests compared completers and noncompleters at each wave on demographic variables, treatment condition, and all outcome variables from preceding waves. Treatment condition (SET, EXP) was not related to attrition at any wave (all p ≥ .32). All other variables were nonsignificant at all waves (p ≥ .11) with the following exceptions. Pre-treatment BAT observer-rated anxiety was significantly related to attrition at the 6-month (p = .03), but not 3-month follow-up (p = .07). Post-treatment BAT observer-rated anxiety was related to attrition at 3-month follow-up (p = .02) but not at 6 months (p = .75). Finally, BAT speech duration at 3 months predicted attrition at 6 months (p = .04). All significant variables were included with the predictors described previously when creating multiple imputation models to correct for missing data at the 3-, and 6-month follow-up periods.

Primary Outcome Variables

Presence/absence of social phobia

Significant group differences were apparent at 3-month follow-up (χ2 [1] = 7.32, p = .025; d = 0.61). Seventy-four percent of participants treated with SET no longer meeting diagnostic criteria for SAD compared to 57% for EXP. The groups were not significantly different at the 6-month follow-up (63% vs. 57%; d = 0.36).

Secondary Outcome Variables

A series of 2 (group: SET, EXP) × 3 (time: Post-treatment, 3-month, 6-month follow-ups) Mixed-Model ANOVAs examined maintenance of treatment gains for both groups using the same strategy described for the post-treatment analysis (see Table 3).

Table 3.

Pretreatment, Posttreatment and Follow-up Means and Standard Errors for Self-Report and Clinician Measures

Measure	SET	EXP	p
Self-report
Brief Social Phobia Scale			< .0005
Pretreatment	42.79 (1.53)	43.46 (1.78)
Posttreatment	17.07 (1.3)	22.44 (1.61)
3 Month Follow-up	16.30 (1.08)	25.22 (1.09)
6 Month Follow-up	19.91 (1.33)	22.90 (1.26)
Social Phobia and Anxiety Inventory			.61
Pretreatment	107.85 (3.13)	104.46 (4.4)
Posttreatment	67.59 (2.68)	70.77 (3.38)
3 Month Follow-up	67.16 (1.94)	62.66 (2.41)
6 Month Follow-up	63.98 (2.31)	60.62 (2.36)
Clinician Ratings
Behavioral Avoidance Rating			< .0005
Pretreatment	4.39 (0.11)	4.53 (0.15)
Posttreatment	1.00 (0.15)	1.78 (0.23)
3 Month Follow-up	1.43 (0.17)	1.86 (0.19)
6 Month Follow-up	1.53 (0.31)	1.82 (0.3)
CGI Severity of Illness			< .0005
Pretreatment	5.14 (0.16)	5.15 (0.11)
Posttreatment	2.08 (0.15)	2.83 (0.21)
3 Month Follow-up	2.4 (0.18)	2.39 (0.26)
6 Month Follow-up	2.47 (0.29)	2.64 (0.39)
Hamilton Anxiety Scale			< .0005
Pretreatment	15.53 (1.05)	16.27 (1.2)
Posttreatment	5.05 (0.6)	6.89 (0.74)
3 Month Follow-up	6.6 (0.68)	7.82 (0.6)
6 Month Follow-up	6.94 (0.8)	7.61 (0.61)
Hamilton Depression Scale			< .0005
Pretreatment	8.19 (0.68)	9.32 (0.79)
Posttreatment	2.1 (0.41)	4.96 (0.69)
3 Month Follow-up	3.57 (0.44)	3.42 (0.47)
6 Month Follow-up	4.18 (0.54)	4.72 (0.5)

Self-reported anxiety

Both treatment groups maintained their treatment gains across the follow-up period for all self-report measures. Furthermore, the SET group continued to report lower anxiety than the EXP group on the BSPS and the BAT self-ratings (all p < .0005; See Figure 2), whereas there were no group differences on the SPAI (p = .61). Paired sample t-test revealed that the SET group maintained their treatment gains, whereas the EXP group continued to show improvement, with significant lower scores at follow-up compared to post-treatment (ps<.02).

BAT objective measures of anxiety and skill

For the BAT Observer Anxiety Factor, BAT Observer Skill Factor and BAT Total Speech Time (see Figures 3, 4, and 5), pre-treatment scores were used as covariates due to significant pre-treatment differences. All omnibus and planned comparison between-group effects were significant (all p < .001), and indicated that the SET group continued to demonstrate less anxiety (d = 0.64 to 1.26) and more skill (d = 0.58 to 0.92) than the EXP group at follow-up. The BAT Total Speech Time interaction effect was significant (p = .036). Planned comparison paired t-tests revealed that the SET group maintained their treatment gains at follow-up (both p ≥ .17). In contrast, speech duration for the EXP group was not significantly different than pre-treatment at 3-month follow-up (p = .13).

Clinician-rated symptoms

Across follow-up, the SET group was rated as less avoidant, less anxious, less depressed, and having less severe symptoms relative to the EXP group (all p ≤ .0005). Although some variability in scores was observed across assessment periods, treatment gains were maintained for all measures (see Table 3).

Discussion

Exposure is an established critical component for effective treatment of SAD. Whether delivered alone or in combination with other procedures (e.g., cognitive interventions, applied relaxation, social skills training), exposure-based interventions benefit a significant percentage of people with SAD (Butler et al., 2006; Hofmann, 2010; Jørstad-Stein & Heimberg, 2009; Ponniah & Hollon, 2008). However, not all individuals with SAD achieve optimum response with standard CBT packages (e.g. Compton et al., 2014), and one viable explanation is the presence of social skills deficits (e.g., Beidel et al., 2010). Consistent with the critical treatment role for exposure therapy, qualitative reviews conclude that social skills training alone is not efficacious for SAD (Ponniah & Hollon, 2008); however, the fundamental question of whether exposure therapy plus social skills training is superior to exposure alone has not been directly addressed.

The results of this investigation indicates that 67% of individuals treated with SET no longer met diagnostic criteria for SAD as did 54% of individuals treated with EXP; these percentages were not significantly different, but significantly higher than the 10% rate found for the WL control group. Furthermore, both active interventions were superior to a wait list control group on each of the secondary outcome measures including social anxiety, general psychological distress, and overt behaviors in social interactions. These results are consistent with numerous studies published to date (Hofmann & Smits, 2008; Rodebaugh, Holaway, & Heimberg, 2004) and demonstrate that both active interventions were provided in sufficient strength to effect positive treatment outcome.

At posttreatment, individuals treated with EXP were rated inferior to patients in the WL group on measures of anxiety and skill even though they spoke significantly longer and reported reduced distress. The most likely explanation is that as a result of treatment, the EXP group was able to participate in the impromptu speech task for a longer duration with less self-rated distress, but they did not deliver a skilled presentation and still appeared anxious to blinded observers. In other words, they tolerated the speech longer and felt less distressed but were still anxious and unskilled, adding to the data that simply decreasing arousal and avoidance behavior does not result in the emergence of social skill (Hopko et al., 2001). We hasten to add that the longer-term impact of reductions in social anxiety and accompanying improvement in social abilities remains unknown.

More directly related to the role of SST, most individuals with SAD have suffered this disorder from childhood (e.g., Stemberger, Turner, Beidel, & Calhoun, 1995) suggesting a persistent pattern of social avoidance and missed social opportunities learn appropriate social behavior. Results of the current investigation suggest social skills training to be an important component of treatment success for SAD. Specifically, although both groups improved significantly relative to the WL, SET was superior to EXP on 8 of the 10 outcome measures, including the BSPS, CGI Severity Rating, behavioral avoidance rating, HAMD, BAT Observer Ratings of Anxiety, BAT Observer Ratings of Skill, IST Speech Duration, and BAT Self-Report of Anxiety, with moderate-to-large magnitude differences between active treatments. At post treatment, the percentage of patients without a diagnosis in the SET and EXP groups were not significantly different (67% vs. 54%), but significantly more patients in the SET group were judged to be treatment responders when compared to EXP alone (92% vs 75%). Thus, even though both interventions significantly reduced anxious distress, SET produced superior outcomes. Although the 92% treatment responder criteria may, to some, appear to be an artifact of the multiple imputation procedures, analysis of the same variable using only completer data resulted in significant percentages (90% for SET and 72% for EXP).

Whereas both SET and EXP resulted in equally clinically meaningful decreases on BAT self-reported anxiety and in BAT total speech time, only SET resulted in clinically meaningful improvements in blinded observer ratings of anxiety and skill during social encounters as well as scores on the BSPS. Therefore, both interventions resulted in clinically significant reductions in self-reported distress, but only SET evidenced concomitant increases in observable skill. These group differences were maintained at 6 month follow-up, which rather importantly demonstrates that changes in social skill did not emerge for individuals treated with EXP only, despite maintained reductions in anxiety and distress.

In contrast to some other investigations, we chose to use imaginal exposure. As noted, efficacious exposure requires that the individual be in contact with all aspects of the fear. The diagnostic criteria for SAD include a fear of negative evaluation by others and thus, for exposure therapy to be optimally efficacious, part of the fear complex must include exposure to others negative evaluation. Craske et al. (2014) noted that maximizing exposure therapy requires designing exposures that maximally violate the expectancies regarding negative outcomes. These authors note that for patients with SAD, the expectancy that they will “get anxious” is an incomplete expectation and therefore represents an insufficient expectancy violation. In contrast, the expectancy that they would be ignored or rejected represents a sufficient expectancy violation and thereby provide for maximum treatment response. Most in vivo exposure assignments would fall under Craske et al. (2014)’s insufficient category as therapists instruct patients to “go out and eat in a restaurant” or “talk to a co-worker”. Rarely, if ever, will the interpersonal partner react in a way consistent with a maximum expectancy violation (respond by ignoring or rejecting). Thus, despite the importance of in vivo exposure, imaginal exposure may be needed in order to provide the maximally effective “dose” of exposure. In fact, the increasingly common use of virtual reality for the treatment of anxiety disorders, including SAD, rests partially on the limited ability of in vivo exposure to re-create specific exposure needs. Finally therapists unfamiliar with imaginal exposure may incorrectly assume that it represents a “weaker” form of exposure than in vivo, yet a perusal of the effect sizes for imaginal exposure in this study notably ranged from 0.82 to 1.61, equivalent to effect sizes for other forms of exposure/CBT reported in a review of meta-analytic findings for cognitive and behavioral treatments for SAD (Deacon & Abramowitz, 2004).

The results of this investigation are consistent with findings in pre-adolescent children (Beidel et al., 2007) and adults (Herbert et al., 2005) showing that social skills training teaches prosocial behaviors that appear to be lacking in patients with SAD. Furthermore, this study improves upon the previous investigations in several ways. First, the outcome variables included assessment of actual behavior, not simply changes in social anxiety. Second, this study is one of the first for adult SAD to assess outcome in terms of clinical significance, not simply statistical significance. The results indicate that both interventions decrease social skill but specific inclusion of SST provides additional benefit.

As in all studies, certain limitations should be considered. First, individuals assigned to both treatment conditions knew they were receiving an active intervention, and in the case of the SET group, a focus on social skill and behavior was maintained throughout the course of treatment. Thus, it is possible that superiority of SET in improving social competence in particular resulted from this knowledge. This explanation would not, however, account for superior independent evaluator and blinded rater evaluations of both anxiety and skill. We did not examine the extent to which self-perception of social performance corresponded with actual changes in social behavior from pre to posttreatment. A number of studies have reported effective treatment of SAD to produce a reduction in negative self-perceptions (Hofmann, Moscovitch, Kim, & Taylor, 2004; Woody, Chambless, & Glass, 1997) but no study has examined the validity of these self-reports (i.e., whether improved self-perception follows actual changes in social behavior). This is a particularly important question for further research and emphasizes the need for direct observation of social skill by blinded observers as opposed to reliance on self-report.

It is important to note that 33% of participants dropped out during active treatment, but this rate of drop-out is similar to another investigation that reported a 32% drop-out rate (Herbert et al., 2005). The use of multiple imputation, which is superior to simple last observation carried forward, allows the data to be used in the treatment outcome analysis, but caution remains. It may be that twice per week treatments, although scientifically justified, may have limited utility for the general population. The reasons for discontinuing the study were varied and included an inability to keep the twice per week commitment, the inability to commit to the same treatment day each week (necessary for participants who were in SET and had a once per week group meeting), moving out of the area, and the development of a serious medical condition, which required immediate and comprehensive treatment, leaving less time for therapy. Future trials should examine treatment delivery on a once per week schedule to determine if decreasing the number of weekly sessions, at the expense of increasing the length of the entire treatment program results in similar outcomes with fewer dropouts.

Also, since all patients in the current study had a primary diagnosis of SAD and certain secondary diagnoses were excluded (e.g., severe mood disorders), it is unknown whether results are generalizable to all individuals with SAD. In general, results require replication. Finally, the sample was primarily non-Hispanic Caucasian. Although to date, there are no data to suggest that generalized SAD presents differently across racial and ethnic groups, the exact extent to which these finding would generalize to other racial and ethnic groups remains unclear.

The current set of findings also potential implications for the recent proliferation of transdiagnostic CBT approaches (i.e., unified protocols) for affective disorders (Allen, McHugh & Barlow, 2008; Barlow, Allen, & Choate, 2004), which diverge from a single-disorder focus by targeting shared underlying processes (such as emotional distress and cognitive distortions). Although outcomes have been reported as similar to diagnosis-specific protocols (Norton & Philipp, 2008), the results do not imply optimal effectiveness across patients. For example, Erickson et al. (2007) found that only panic-disordered patients (i.e., not patients with SAD) treated with a transdiagnostic intervention showed improvement over a delayed treatment condition. Other transdiagnostic studies have found patients with SAD to improve similarly to those with other anxiety disorders (Norton, 2008; 2011) but comparisons are generally been based on a single self-report measure of general anxiety and presence of SAD anywhere in the diagnostic profile (i.e., not necessarily the primary disorder). In light of the current findings indicating that social skills training to occupies a critical role in determining optimizing treatment outcomes, direct comparisons between broad-based approaches and comprehensive disorder-specific treatment (such as SET) that incorporate measures of actual behavior change for SAD are needed.

Highlights.

• Social Effectiveness Therapy (SET) and Exposure Therapy are superior to a wait list control condition for the treatment of social anxiety disorder (SAD)

• At posttreatment, 67% of patients treated with SET and 54% treated with Exposure Therapy no longer met diagnostic criteria for SAD, a non-significant difference

• SET was superior to Exposure Therapy on measures of social skill and general clinical status

• Changes for both treatment groups on measure of social anxiety were clinically, as well as statistically, significant.