Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2006 Jun 8.
Published in final edited form as: Behav Ther. 2000;31(3):499–515. doi: 10.1016/s0005-7894(00)80027-1

An Instrument to Assess Self-Statements During Public Speaking: Scale Development and Preliminary Psychometric Properties

Stefan G Hofmann 1, Patricia Marten DiBartolo 2
PMCID: PMC1475776  NIHMSID: NIHMS10370  PMID: 16763666

Abstract

Public speaking is the most commonly reported fearful social situation. Although a number of contemporary theories emphasize the importance of cognitive processes in social anxiety, there is no instrument available to assess fearful thoughts experienced during public speaking. The Self-Statements During Public Speaking (SSPS) scale is a 10-item questionnaire consisting of two 5-item subscales, the “Positive Self-Statements” (SSPS-P) and the “Negative Self-Statements” subscale (SSPS-N). Four studies report on the development and the preliminary psychometric properties of this instrument.

An Instrument to Assess Self-Statements During Public Speaking: Scale Development and Preliminary Psychometric Properties

Contemporary theories on the psychopathology of social anxiety emphasize the role of cognitive processes (e.g., Clark & Wells, 1995; Heimberg, 1994; Leary & Kowalski, 1995; Rapee & Heimberg, 1997). A common element of these theories is the presumption that social anxiety results from negative self-perception or perceived negative evaluation by other people in social situations. Despite the theoretical importance of cognitions in social phobia, a review of the literature suggests that less than three-quarters of the studies on social phobia utilized any type of cognitive assessment (Heimberg, 1994). This might be due to the fact that many of the available cognitive measures for social anxiety are either difficult to administer and score (such as the thought listing procedure), or are of questionable validity, such as Fear of Negative Evaluation Scale (Watson & Friend, 1969; see Heimberg, 1994 for a critique of this scale).

In contrast, many depression researchers have successfully utilized a number of self-statement questionnaires, such as the Negative Affect Self-Statement Questionnaire (NASSQ; Ronan, Kendall, & Rowe, 1994), the Automatic Thoughts Questionnaire (ATQ; Hollon & Kendall, 1980), and its modifications (Kendall, Howard, & Hayes, 1989). These instruments have lead to further insight into the psychopathology of depression by providing a means of testing basic theory and assessing change in cognitions associated with experimental manipulation (Hollon & Kendall, 1980; Kendall et al., 1989; Kendall & Hollon, 1989; Lerner, Safren, Henin, Warman, Heimberg, & Kendall, 1999; Ronan et al., 1994). This argues for the need of a reliable and valid self-statement questionnaire to measure cognitions related to social anxiety.

Of all social situations, public speaking is the most prevalent fear in both the general population and among social phobic individuals (Mannuzza et al., 1995; Stein, Walker, & Forde, 1996). However, surprisingly few instruments are available to reliably assess the degree of public speaking anxiety, and no instrument exists to specifically assess the cognitive aspects of public speaking anxiety. The only measures that directly assess public speaking anxiety are the Personal Report of Confidence as a Speaker Questionnaire (Paul, 1966), a shortened version of an instrument originally developed by Gilkinson (1942) and the Personal Report of Communication Apprehension (McCroskey, 1997). In addition, previous studies have employed the Social Interaction and Self-Statement Test (Glass, Merluzzi, Biever, & Larsen, 1982) to specifically assess fearful cognitions related to public speaking.

The intention of the original version of the Personal Report of Confidence as a Speaker Questionnaire (PRCS) developed by Gilkinson (1942) was to assess the degree of confidence as a speaker. Paul (1966) later shortened this instrument. This shortened version shows adequate internal consistency (Klorman, Weerts, Hastings, Melamed, & Lang, 1974) and validity (Lombardo, 1988). However, the test-retest reliability of this 30-item instrument is unknown, and its utility is limited due to its true-false format (e.g., McNeil, Ries, & Turk, 1995).

The Personal Report of Communication Apprehension (PRCA; McCroskey, 1997) scale consists of 24 items that are rated on a 5-point Likert scale to assess apprehension during public speaking, public meetings, and group discussions. Examples of items include: “I dislike participating in group discussions,” ”I am afraid to express myself at meetings,” and “I feel relaxed while giving a speech.” Although this scale has been used in a number of earlier studies by McCroskey and colleagues (e.g., Daly & McCroskey, 1975; McCroskey, 1978), little is known about the psychometric properties of this scale.

The Social Interaction and Self-Statement Test (SISST; Glass et al., 1982) is the most frequently used structured self-statement test in social anxiety research (Arnkoff & Glass, 1989). This instrument contains 15 positive and 15 negative self-statements about social interactions. The items were originally drawn from thought listings of a large student sample in response to frequently occurring problematic heterosocial situations. The scale has satisfactory split-half reliability and high item-total correlations (Glass et al., 1982), as well as factor-analytic validity (Glass et al., 1982), external validity (Beidel, Turner, & Dancu, 1985; Glass et al., 1982; Myszka, Galassi, & Ware, 1986), and concurrent validity (Dodge, Hope, Heimberg, & Becker, 1988; Glass & Arnkoff, 1994). However, the utility of the SISST is limited because the original version can only be administered following a structured social interaction task. Previous studies have therefore modified the instructions of the SISST in order to measure dispositional responses to social interactions, independent of a social challenge test (Hofmann & Roth, 1996; Hofmann, Newman, Ehlers, & Roth, 1995). In these studies, participants were asked to fill out the SISST while imagining a typical fearful social situation. By using this “trait version,” the SISST reliably distinguished subgroups of individuals with various levels of social anxiety.

Another limitation of the SISST is that several of its items are not applicable to a public speaking situation, including “I can usually talk to girls/boys pretty well,” “She/he probably won't be interested in me,” “I'm not too comfortable meeting girls/boys so things are bound to go wrong,” and “What the heck, the worst that can happen is that she/he won't go for me.” Nevertheless, a number of previous studies have utilized this instrument (with modified instructions) for assessing cognitions during public speaking (Beidel, Turner, & Dancu, 1985; Beidel, Turner, Jacob, Cooley, 1989; Turner, Beidel, & Larkin, 1986). These studies suggest that the SISST can discriminate between public speaking anxious people and controls, possibly because many of the SISST items are also relevant for the speech situation (Arnkoff & Glass, 1989).

In summary, there is no instrument currently available to assess self-statements during speaking situations. The PRCA and the PRCS were developed to assess the degree of communication apprehension and confidence as a speaker, respectively, whereas the SISST is an instrument to assess cognitions when interacting with the opposite sex. Previous studies have primarily utilized modified versions of the SISST to measure self-statements during public speaking because the SISST shows good psychometric properties and contains items that also seem to apply to public speaking situations. However, the psychometric properties of such modifications remain uncertain. The following four studies will report on the development and psychometric characteristics of the Self-Statements During Public Speaking (SSPS) scale, an instrument that was developed based on the SISST. The following studies report on the development and preliminary psychometric properties of an instrument to measure self-statements (and distress) related to public speaking1.

Study 1

The purpose of the first study was to develop a brief scale with good psychometric properties that can be administered to participants as a self-report questionnaire to assess typical self-statements during public speaking.

Method

Subject Sample

A 30-item scale was administered to 100 undergraduate students from the State University of New York at Albany. Participants ranged in age from 17 to 23 years with a mean of 18.8 (SD = 1.1). Approximately half (53%) of the participants were female and most of them were Caucasian (78%). Eight individuals were African-American, 6 Asian-American, 5 Latino/-a, and 3 Native-American.

Item Construction

The construction of the SSPS scale was largely based on the items of the SISST, a psychometrically sound and frequently used state measure of cognitions during social interactions. Eight of the original SISST items were modified to be suitable to the public speaking situation, and five SISST items were omitted because they specifically relate to interactional social situations (item 2: “I can usually talk to girls/boys pretty well,” item 9: “Maybe I can put him/her at ease by starting things going,” item 13: “She/he may want to talk to me as much as I want to talk to her/him,” item 27: “We probably have a lot in common,” and item 28: “Maybe we'll hit it off well”). These five items were replaced with four negative statements and one positive statement that are consistent with the cognitive model of social phobia (Clark & Wells, 1995) and empirical data (Hofmann, 2000a). The four negative statements included: “I'm a loser,” “I expect the worst,” “A failure in this situation would be more proof of my incapacity,” and “Most people would master this situation much better than I do.” The positive statement read: “It's just a kind of a game. Nothing bad can happen.” These items are consistent with the cognitive model of social phobia (e.g., Clark & Wells, 1995; Rapee and Heimberg, 1997). This model suggests that socially anxious individuals believe that they are in danger of behaving in an inept and unacceptable fashion (“Most people would master the situation much better than I do”), and that such behavior would have disastrous consequences (“I expect the worst”), which results in negative self-perception (“A failure in this situation would be more proof of my incapacity,” and “I'm a loser”).

For the scale construction of the SSPS, the instructions were modified to measure a trait variable. The revised instructions read: “Please imagine what you have typically felt and thought to yourself during any kind of public speaking situation. Imagining these situations, how much do you agree with the statements given below? Please rate the degree of your agreement on a scale between 0 (if you do not agree at all) to 5 (if you agree extremely with the statement)”.

Item Selection

The goal of the item selection procedure was to generate a brief questionnaire with high internal consistency, high item validity, and good factor analytic validity. The item selection procedure was carried out in two steps. First, items with low item validity (item-total correlation) were eliminated in order to maximize the internal consistency of the scale. Second, items with low factor loadings were excluded in order to increase the factor analytic validity of the scale and to minimize the total number of items of the scale.

First, a correlation matrix of the 30 items was calculated. The item with the lowest item-total correlation was excluded. After an item was eliminated, a new correlation matrix was calculated and the internal consistency of the remaining items was estimated. This item elimination process was repeated until the internal consistency of the total scale could not be increased further by additional item reduction. This process excluded four of the 30 items.

Results

The internal consistency (Cronbach's alpha) of this preliminary 26 item scale was alpha = .94. All item-total correlation coefficients were rs >.5. The 26 items were subjected to a principal component analysis. Five factors with Eigenvalues greater than 1 were identified (the Eigenvalues of the five factors were 10.8, 2.4, 1.5, 1.3, and 1.1). The first two factors explained 51% of the total variance. In order to minimize the number of items of the questionnaire, only items that showed high factor loadings on either one of the first two factors were considered for further analyses. The correlation matrix was then subjected to a VARIMAX rotation. Five items had high loadings on the first factor (all factor loadings >.6), but low loadings on any of the remaining four factors (all factor loadings <.3). Similarly, five other items showed high loadings on the second factor (factor loadings >.6) but low loadings on any of the remaining four factors (all factor loadings < .3 with the exception of one item which showed a factor loading of .48 on factor 3 and another item that showed factor loadings of .34 and .31 on factor three and four, respectively). The remaining items were excluded from further analyses.

The final scale consisted of 10 items, with half of the items loading highly on either one of the first two factors. This 10 item scale was then again subjected to a principal component analysis with VARIMAX rotation. Factor loadings for the final set of items are presented in Table 1. The two factor solution was replicated using the 10 items only. The two factors explained together 61.1% of the variance. The Eigenvalues of Factor I and II were 4.44 and 1.67, respectively. The first factor was interpreted as the “Positive Self-Statements” subscale (SSPS-P), and the second factor as the “Negative Self-Statements” subscale (SSPS-N).

Table 1.

Rotated Factor Loadings (VARIMAX) for a 2-Factor Solution on the Self-Statement During Public Speaking Scale (Study 1).

Item Factor I Positive Self-Statements Subscale Factor II Negative Self-Statements Subscale
1. What do I have to lose it's worth a try .82 .15
3. This is an awkward situation but I can handle it .75 .12
5. Even if things don't go well, it's no catastrophe .81 .13
6. I can handle everything .70 .26
9. Instead of worrying I could concentrate on what I want to say .67 .21
2. I'm a loser .04 .75
4. A failure in this situation would be more proof of my incapacity .15 .77
7. What I say will probably sound stupid .20 .72
8. I'll probably “bomb out” anyway .37 .76
10. I feel awkward and dumb; they're bound to notice .20 .76
Open in a new tab

Note: Items 1, 3, 5, 6, and 9 comprise the “Positive Self-Statements,” and items 2, 4, 7, 8, and 10 the “Negative Self-Statements.”

Cronbach's alpha was high for both SSPS-P (alpha = .84) and SSPS-N (alpha = .83). The SSPS-P showed a mean of 15.4 (SD = 5.1) and a median of 16.0 (range: 1 - 25), and the SSPS-N showed a mean of 7.9 (SD = 5.2) and a median of 7.5 (range: 0 - 25). The two subscales showed a correlation of r = −.69, n = 100, p < .0001.

Discussion

The sample from Study 1 consisted of an unselected group of undergraduate students (N = 100) from a large public university. Approximately half of them were female. The goal of the item selection procedure was to minimize the total number of items of the scale and to maximize its internal consistency, item validity, and factor analytic validity. Ten items were identified that met these criteria. These items loaded highly on either one of two subscales which were interpreted as “Positive Self-Statements” (SSPS-P) and “Negative Self-Statements” (SSPS-N).

Study 2

The objective of the second study was to replicate the initial factor structure found in the previous sample, and to gather additional data on the reliability and the convergent and discriminant validity of the scale.

Method

Subject Sample

A total of 201 women from a Northeastern women's college (Smith College, Northampton, Massachusetts) participated in a questionnaire study. Participants were Caucasian (76.0%), Asian-American (16.7%), African-American (3.1%), and Latina (3.1%) and were between the ages of 17 and 46 (M = 18.6; SD = 3.52).

Procedure

Participants were asked to fill out a questionnaire battery consisting of the SSPS scale, the Personal Report of Confidence as a Speaker (PRCS; Paul, 1966), the “trait version” of the Social Interaction Self-Statement Test (SISST; Glass, Merluzzi, Biever, & Larsen, 1982), the Fear of Negative Evaluation Scale (FNE; Watson & Friend, 1969), the Social Avoidance and Distress Scale (SADS; Watson & Friend, 1969), the Social Phobia and Anxiety Inventory (SPAI; Turner, Beidel, Dancu, & Stanley, 1989), and the Beck Depression Inventory (BDI; Beck, Ward, Mendelson, Mock, & Erbaugh, 1961). All participants completed this questionnaire battery in groups of 10 to 20.

Results

Cross-Validation of the Factor Structure

A principal component analysis of the 10 items of the SSPS identified again 2 factors with Eigenvalues of greater than 1. The two factors explained 42.7% and 15.5% of the total variance with Eigenvalues of 4.27 and 1.55, respectively. The VARIMAX rotation replicated the previous factor structure (see Table 1). The two SSPS subscales showed a correlation of r = −.45, n = 201, p < .001.

Comparison With Previous Study Sample

The SSPS-P subscale showed a mean of 15.8 (SD = 4.5) and a median of 16.0 (range: 1 - 25), and the SSPS-N subscale showed a mean of 7.1 (SD = 5.3) and a median of 6.0 (range: 0 - 24). A comparison between the Albany sample from study 1 and the present sample on the SSPS subscale scores (two tailed t-tests for independent samples) showed no statistically significant differences (all ps > .2).

Reliability

Cronbach's alpha was satisfactory for the SSPS-P (alpha = .75) and the SSPS-N subscale (alpha = .86).

Convergent and Discriminant Validity

The correlations between the PRCS and the SSPS-P and the SSPS-N subscales showed the highest correlations (r = −.58 and r = .67, respectively, ps < .01). Furthermore, the SSPS-P and the SSPS-N subscale showed moderate correlations (all ps < .01) with the FNE (rs: −.29 and .49, respectively), the SADS (rs: −.30 and .37, respectively), and the SPAI, social phobia subscale (rs: −.34 and .48, respectively).

Moderate correlations were also found between the “trait version” of the two SISST subscales and the SSPS subscales. The SSPS-N subscale showed a correlation of r = .48, p < .01, with the “negative self-statements” of the SISST and a correlation of r = −.32, p < .01, with the “positive self-statements” of the SISST. The SSPS-P showed a correlation of r = −.18, p < .05 with the “negative self-statements” of the SISST and a correlation of r = .35, p < .01, with the “positive self-statements.”

The BDI and the SPAI (Agoraphobia Subscale) showed significant correlations with the SSPS-N (BDI: r = .21, p < .01; SPAI, Agoraphobia subscale: r = .14, p < .05) but not with the SSPS-P subscale (BDI: r = −.09, p > .2; SPAI, Agoraphobia subscale: r = −.01, p > .7). Follow-up analyses showed that the BDI is more highly correlated with the SSPS-N than with the SSPS-P subscale, t (198) = 3.07, p < .005. However, the correlations between the SPAI, Agoraphobia subscale, and the two SSPS subscales were not significantly different.

Discussion

The 10-item SSPS scale was administered to a sample of 201 female undergraduates from a liberal arts college in Northampton (Smith College). The factor structure of the SSPS that was found in the Albany sample from Study 1 was again replicated in the Smith College sample. Despite the homogeneity of the Smith College sample (which also limits the generalizability of the findings), the internal consistency measures of the two subscales were again high, and the test-retest reliability was good.

The two subscales of the SSPS showed moderate correlations with frequently used social phobia and anxiety scales, which do not specifically assess public speaking anxiety, including the “trait version” of the SISST (all rs < .5). Although derived from items of the SISST, the two SSPS subscales showed the highest correlation with the PRCS, the only other instrument from the test battery that was specifically tailored to the public speaking situation. Finally, the BDI was more highly correlated with the SSPS-N than with the SSPS-P, which may suggest that the SSPS-N subscale is more closely related to negative affect and depression than the SSPS-P subscale. However, although these correlations reached statistical significance, the coefficients are very low which questions the clinical significance of this association.

Study 3

The purpose of the third study was to gather psychometric information on the SSPS in a group of social phobic patients.

Methods

Subject Sample

Participants were 41 consecutively admitted outpatients at the Center for Anxiety and Related Disorders at Boston University with a principal diagnosis of social phobia. The participants were between the ages of 19 and 50 (M = 33.2; SD = 9.8). Participants were Caucasian (86%), Asian-American (9.3%), African-American (2.3%), and Latinos/-as (2.3%). Approximately half of the participants (56%) were female, and 59% of the participants met criteria for the “generalized subtype” of social phobia. The remaining participants (44%) were assigned a “nongeneralized” subtype of social phobia because they did not meet criteria for the “generalized” subtype. Furthermore, 22% of the total sample could be classified as having “circumscribed” social phobia (Boone, McNeil, Masia, Turk, Carter, Ries, & Lewin, 1999)2.

Procedure

All 41 Participants were asked to fill out the SSPS scale after the diagnosis of social phobia was confirmed by experienced clinicians using the Anxiety Disorders Interview Schedule for DSM-IV: Lifetime Version (ADIS-IV-L; DiNardo, Brown, & Barlow, 1994). The patients of the present investigation were also part of a larger study determining the reliability of DSM-IV diagnoses. As part of this study, all patients received the same diagnostic interview twice by two independent and trained Master's level clinicians within a 2-week time period. The Kappa coefficient for the principal DSM-IV diagnosis of social phobia was κ = .76 (Brown, Lehman, Campbell, & DiNardo, 2000). This is consistent with an earlier reliability study which found a Kappa coefficient of κ = .79 by using the ADIS-R, an earlier version of this diagnostic interview (DiNardo, Moras, Barlow, & Rapee, 1993). Disagreements were resolved by a consensus diagnosis in a meeting chaired by senior clinicians.

In order to estimate test-retest reliability of the SSPS, a random sample of social phobic patients (n = 26) were asked to fill out the SSPS again after a 3 month waiting period. A second group of participants (n = 15) was offered treatment immediately following the initial assessment. In order to determine changes in the SSPS scales as a result of treatment, these 15 individuals were asked to fill out the instrument a second time after completing the intervention. In addition, these individuals were asked to fill out the SPAI (Turner et al., 1989) before and after treatment in order to evaluate the efficacy of the intervention.

Treatment consisted of 8-12 weekly 2-hour exposure sessions with 4-6 patients and an experienced female and male therapist in each group. Sessions were conducted according to a detailed speech phobia treatment manual (Hofmann, 1999)3. The components of the intervention included didactic training in communication and speech-making skills, in vivo exposures as part of homework assignments, mirror exposure, and exposure to formal and informal speaking situations with video feedback. In a recent meta-analysis conducted by Feske and Chambless (1995), the efficacy of an earlier version of this treatment manual was comparable to other empirically supported treatments for social phobia. Further details about this procedure and its efficacy are reported elsewhere (Hofmann, Newman, Becker, Taylor, & Roth, 1995; Newman, Hofmann, Trabert, Roth, & Taylor, 1994).

Results

Cross-Validation of the Factor Structure

A principal component analysis of the 10 items of the SSPS identified again 2 factors with Eigenvalues of greater than 1. The two factors explained 49.2% and 14.2% of the total variance with Eigenvalues of 4.92 and 1.42, respectively. The VARIMAX rotation replicated again the original factor structure (see Table 1). The two SSPS subscales showed a correlation of r = −.42, n = 41, p < .006.

Comparison With Previous Study Samples

The SSPS-P subscale showed a mean of 13.4 (SD = 6.0) and a median of 13 (range: 1 - 24), and the SSPS-N subscale showed a mean of 12.3 (SD = 6.3) and a median of 13 (range: 2 - 24). Two tailed t -tests for independent samples showed that the present sample had higher scores in the SSPS-N, t (139) = 3.93, p < .001, and lower scores in the SSPS-P subscales than the Albany sample from Study 1, t (139) = 1.83, p < .04, and the Smith College sample from Study 2, t (240) = 4.92, p < .0001 and t (240) = 2.38, p < .01, respectively.

Reliability

Cronbach's alpha coefficients were high for both the SSPS-P (alpha = .80) and SSPS-N subscale (alpha = .86). The 3-month test-retest reliability was acceptable for both the SSPS-P subscale (r = .78) and the SSPS-N subscale (r = .80).

Pre-Post Treatment Differences

Paired t-tests (two tailed) showed a significant reduction in the social phobia subscale scores of the SPAI from pre-treatment (M = 113.3, SD = 23.3) to post-treatment assessment (M = 95.8, SD = 23.3), t (14) = 3.36, p < .005. Similarly, the total scores of the SPAI showed a significant decline from pre-treatment (M = 95.6, SD = 22.4) to post-treatment assessment (M = 78.1, SD = 16.2), t (14) = 3.5, p < .003. In contrast, the agoraphobia subscale of the SPAI showed no changes from pre-treatment (M = 17.7, SD = 11.0) to post-treatment assessment (M = 17.7, SD = 9.4), t (14) = 0.0, p = 1.0.

The analysis of the SSPS showed significant changes in the SSPS-N from pre-treatment (M = 11.2; SD = 6.3) to post-treatment assessment (M = 6.5; SD = 4.8), t (14) = 3.36, p < .005. However, the increase in the SSPS-P scores from pre-treatment (M = 15.1; SD = 6.2) to post-treatment assessment (M = 17.5; SD = 4.5) was not statistically significant, t (14) = 1.32, p > .2.

Subtype Analyses

There were no significant differences across the social phobia subtypes in the SSPS-N or the SSPS-P before or after treatment (all ps > .1).

Discussion

The results of Study 3 replicated the factor structure of the scale for a third time in a group of social phobics. Measures of internal consistency and test-retest reliability of both SSPS subscales were satisfactory. As expected, social phobic individuals scored higher on the SSPS-N, and lower on the SSPS-P subscale than the two undergraduate samples from Study 1 and 2. A pre-post treatment comparison showed that the group intervention primarily led to a reduction in SSPS-N scores which is in line with previous studies suggesting that negative self-statements have a stronger relationship to measures of psychopathology than positive self-statements (Arnkoff & Glass, 1989; Glass et al., 1982; Hofmann, 2000a; Kendall & Hollon, 1981; Treadwell & Kendall, 1996). No difference was found between the social phobia subtypes.

Study 4

The results from Study 3 indicate that the SSPS-N subscale is sensitive to treatment changes among social phobic individuals. The purpose of the fourth study is to further explore the utility of this subscale in a student sample. We examined the responses of participants who scored high and low on the SSPS-N subscale to a behavioral avoidance task (BAT) focused upon public speaking. We chose the SSPS-N subscale rather than the SSPS-P subscale score to define the two extreme groups because the literature suggests that it is the “non-negative thinking” rather than the “positive thinking” that is related to treatment gains. Furthermore, the results of Study 3 suggest that negative self-statements may be more closely associated with public speaking anxiety than positive self-statements. In order to determine whether this scale has predictive value for anxious responding during a behavioral test, we administered this scale to a group of normal individuals and conducted extreme group comparisons.

Method

Subject Sample

Participants were female undergraduates from Smith College recruited from a pool of 100 students who completed the SSPS as part of a larger battery of measures. A total of 19 participants were randomly chosen from the upper and lower thirds of scores on the SSPS-N subscale. Three individuals from the high SSPS-N group refused the assessment after the speech task was described to them, yielding a total of 16 participants in the high (n = 7) and low (n = 9) SSPS-N groups. The difference between the two groups in the number of refusals did not reach the conventional level of statistical significance, χ2 (1) = 3.21, p = .07.

Measures

Prior to the speech task, participants completed the state version of the Positive and Negative Affect Scale (PANAS; Watson, Clark & Tellegen, 1988). Participants were then asked to rate how well they expected themselves to perform on the impending speech task (on a 0 to 100 scale from not at all well to extremely well) in order to measure their perceived self-efficacy. Throughout the experimental session (before and after each phase and at one-minute intervals during the speech task), participants were asked to indicate their feelings of anxiety using SUDS ratings from 0 (not at all anxious) to 100 (extremely anxious). Following completion of the task, participants were asked to rate their anxiety during the speech and their satisfaction with their performance (on a 0 to 100 scale). Furthermore, they were asked to complete the PANAS.

Procedure

After participants arrived, the speech task was described to them. They chose 3 of 5 current events topics to discuss (abortion, mandatory seat belt laws, health care system, corporal punishment in the schools, and death penalty). Participants then completed all pre-task measures. A 3-minute baseline (BSL) period followed during which participants sat quietly alone in the assessment room. The next 3-minute phase was the preparation phase (PREP). Participants were allowed to write notes and organize their thinking for the speech, although the use of written notes was not allowed during the actual speech task. The audience was then brought into the room and the subsequent 3-minute anticipation phase (ANT) involved the participant sitting quietly in the room with two audience members. The 10-minute speech task (TASK) followed. When the task ended, participants completed the post-task measures and were debriefed.

Results

Pre-task measures

The analyses revealed a significant group effect for individuals' performance expectations, F (1, 14) = 5.91; p < .05, ω2 = .23. Individuals in the high SSPS-N group reported lower expectations for their own success (M = 45.7; SD = 11.3) in comparison to the low SSPS-N group (M = 68.3; SD = 22.4).

SUDS ratings

A repeated measures ANOVA showed a significant Group × BAT phase effect, F (2, 28) = 4.89; p < .05, η2 = .26. Follow-up comparisons were conducted using the general linear model approach for multivariate comparisons (SPSS, release 9.0.0) with the two groups as a fixed factor and the three experimental tasks as dependent variables. The omnibus test was statistically significant, F (3, 12) = 7.03, p < .006. The between subject effects showed that individuals in the high SSPS-N group reported significantly higher SUDS ratings during the speech (TASK) than individuals in the low SSPS-N group, F (1, 14) = 11.07, p < .005. The difference between the two groups did not reach the conventional level of statistical significance for PREP, F (1, 14) = 4.49, p < .06 and ANT, F (1, 14) = 3.99, p < .07.

When including the baseline SUDS scores as a covariate into the model, the omnibus test remained statistically significant, F (3, 11) = 6.00, p < .011. The between group analyses further revealed a significant group effect in SUDS ratings for PREP, F (1, 13) = 5.22, p < .04, and TASK, F (1, 13) = 18.31, p < .001. However, no group difference was found for ANT, F (1, 13) = 2.05, p > .17. These results suggest that subjects with high SSPS-N scores reported greater distress during an impromptu speech than those with low SSPS-N scores.

Post-task measures

Analyses indicated significant group differences on participants' self-ratings of their anxiety during the speech, F (1, 14) = 6.40; p < .05, ω2=.25, and their satisfaction with the completed speech task, F (1, 14) = 5.09; p < .05, ω2=.20. High SSPS-N participants reported both more anxiety and less satisfaction with their performance (M = 66.4; SD = 21.0 and M = 41.4; SD = 21.2, respectively) than the low SSPS-N group (M = 41.1; SD = 19.0 and M = 65.6; SD = 21.3, respectively).

PANAS scores

Both negative affect (NA) and positive affect (PA) scores from the PANAS were analyzed using a 2 × 2 ANOVA with Group (high vs. low SSPS-N) as the between-subjects factor and Time (pre- vs. post-task) as the within-subjects factor. Analysis of NA scores revealed a significant main effect of Group, F (1, 14) = 13.95; p < .01, ω2=.45, with high SSPS-N participants reporting higher levels of NA (M = 28.7; SD = 17.1) than low SSPS-N participants (M = 6.7; SD = 4.6). No other significant effects were noted for NA or PA scores.

Discussion

The results of this study showed that female undergraduates with high SSPS-N scores reported higher subjective anxiety ratings and greater negative affect during a speech, lower expectations for successful performance, and less satisfaction with their speech performance than individuals with low SSPS-N scores. These data provide further evidence for the utility of the SSPS by distinguishing high and low public speaking anxious students.

General Discussion

Our objective was to develop a brief and psychometrically sound scale to assess self-statements during public speaking. Rather than drawing items from social phobic individuals' self-talk, we chose to derive the items from an already established instrument, the Social Interaction and Self-Statement Test (SISST), which is a reliable and valid instrument to assess self-statements during social interactions. Other items were added that are consistent with the cognitive model of social phobia. The goal of the item selection procedure was to generate a brief instrument with high internal consistency, good item validity, and good factor analytic validity. The result was the SSPS, a trait measure to assess fearful thoughts associated with public speaking. This brief 10-item questionnaire consists of two 5-item subscales, the “Positive Self-Statements” (SSPS-P) and the “Negative Self-Statements” (SSPS-N) subscale. The factor structure of the instrument was replicated in three different samples, including two undergraduate samples and a group of social phobic individuals. The two SSPS subscales showed good internal consistency and good test-retest reliability.

Although the development of the SSPS scale was largely based on the SISST, the two SSPS subscales showed the highest correlation with the PRCS, the only other instrument of our questionnaire battery to specifically assess public speaking anxiety whereas moderate correlations were found with the modified (trait) version of the SISST. Moderate correlations were also observed with other measures of social anxiety, including the FNE, the SADS, and the SPAI (social phobia subscale), whereas the two SSPS subscales showed low correlations with the BDI and the SPAI agoraphobia subscale. These findings indicate that the two SSPS subscales have good convergent and discriminant validity.

Additional analyses showed that the SSPS-N was more highly correlated with the BDI than was the SSPS-P in an undergraduate student sample. Furthermore, only the SSPS-N subscale but not the SSPS-P subscale scores changed significantly from pre-treatment to post-treatment assessment in a group of social phobic individuals. These findings, together with the observation that SSPS-N scores can differentiate anxious from non-anxious students during a public speaking task provide additional evidence for the utility of this instrument. High SSPS-N participants reported greater anxiety and a more negative cognitive style than low SSPS-N scorers in that they reported significantly lower expectations for successful performance prior to the task and less satisfaction with their actual performance following the speech. Furthermore, the SSPS seems to be clinically useful because the SSPS-N subscale is sensitive to change as a function of short-term treatment.

This scale could therefore be a potentially useful measure of treatment outcome for individuals with public speaking anxiety, especially when used as part of a multimodal assessment strategy (McNeil et al., 1995). The brevity of the scale and the simple scoring procedure further adds to the clinical utility of the instrument. Finally, in conjunction with other questionnaires, the SSPS could provide valuable data to test and further refine the cognitive model of social phobia by assessing cognitive change associated with experimental manipulation and treatment. Self-statement questionnaires have been successfully used for this purpose in the research of depression (Hollon & Kendall, 1980; Kendall et al., 1989; Kendall & Hollon, 1999; Lerner et al., 1999; Ronan et al., 1994), and we believe that research on social anxiety could similarly benefit from such an instrument. In fact, our preliminary data indicate that, consistent with the studies on depression, negative self-statements have a stronger relationship to psychopathology and treatment change than positive self-statements, which is consistent with the notion of the “power of non-negative thinking” (Kendall & Hollon, 1981).

However, additional studies will be necessary to address a number of weaknesses with the SSPS. Most importantly, it will be necessary to gather data on the external validity of the instrument. Specifically, it remains uncertain whether the SSPS measures affect, cognitions, or both. The aforementioned studies suggest that the SSPS measures an aspect of public speaking anxiety that is separate from overall social anxiety, fear of negative evaluation by others, and depressed mood. However, the SSPS-N subscale is closely associated with negative affect as measured by the PANAS. This might suggest that the SSPS measures negative affect in addition to anxious thoughts because the wording of the instructions of the SSPS might have encouraged participants to answer some of the items based on their affective experience regardless of whether they remember having a particular thought. Future studies could clarify this issue by comparing different versions of the SSPS that vary in the wording of the instructions. Furthermore, valuable data could come from studies comparing the SSPS with thought listings of individuals giving a public speech. Finally, most of our participants were Caucasian, and some of the studies were conducted with all-women samples, which limits the generalizability of our findings. Therefore, additional studies with sufficiently large sample sizes that also include a variety of participant subgroups are needed. Despite these and other limitations, our findings indicate that the SSPS is a potentially useful instrument to assess cognitions related to public speaking.

Figure 1.

Figure 1

Open in a new tab

Subjective distress during baseline (BSL), preparation (PREP), anticipation (ANT), and public speaking (TASK) in participants scoring high (N = 7) and low (N = 9) on the SSPS-N subscale.

Acknowledgments

We thank Alison Bateman, Carey Shaffer, and David Hershberger for their help with the data collection, and four anonymous reviewers for their valuable comments.

Footnotes

1

It is uncertain whether the SSPS is an instrument to only measure thoughts or a combination of thoughts and emotions related to public speaking. Future research will therefore have to investigate to what extent this instrument measures distress in addition to cognitions related to public speaking.

2

The social phobia subtype was determined on the basis of the patient's subjective anxiety ratings of the 24 social situations from the Liebowitz Social Anxiety Scale (LSAS, Liebowitz, 1987). In an attempt to examine the prevalence and overlap of social anxiety across different classes of situations, Holt, Heimberg, Hope, and Liebowitz (1992) classified the items of the LSAS into four different domains - formal speaking/interaction, informal speaking/interaction, assertive interaction, and observation by others. Patients of the present study were classified into the “generalized subtype” if they rated one or more social situations from each domain as at least moderately fear provoking (rating of 2 or greater on a 0 - 3 point scale). Otherwise, patients were classified as “nongeneralized.” A “circumscribed” subtype was assigned if participants endorsed at least moderate anxiety in only the “formal speaking/interaction” domain, which included the LSAS items 2 (participating in small groups), 6 (acting, performing or giving a talk in front of an audience), 16 (speaking up at a meeting), and 20 (giving a report to a group). No reliability information of this procedure is available. However, previous studies indicate that a similar procedure can reliably distinguish individuals with high and low social anxiety and other psychopathology (Hofmann & Roth, 1996; Hofmann, Albano, Heimberg, Tracey, Chorpita, & Barlow, 1999).

3

This intervention was part of a larger study on mediators and moderators of treatment change in social phobia (Hofmann, 2000b). As part of this study, individuals with social phobia and significant public speaking anxiety received exposure therapy for their speech anxiety. Most of these individuals also exhibited significant social anxiety in other social phobia domains (such as informal interaction, assertiveness situations, etc.). One of the goals of this study was to identify the mediators that are responsible for the generalization effect of exposure treatment for public speaking anxiety onto other social fears.

Supported in part by NIMH grant MH-57326 to Dr. Hofmann.

Contributor Information

Stefan G. Hofmann, Department of Psychology, Boston University

Patricia Marten DiBartolo, Department of Psychology, Clark Science Center, Smith College.

References

  1. Arnkoff DB, Glass CR. Cognitive assessment in social anxiety and social phobia. Clinical Psychology Review. 1989;9:61–74. [Google Scholar]
  2. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Archives of General Psychiatry. 1961;4:561–571. doi: 10.1001/archpsyc.1961.01710120031004. [DOI] [PubMed] [Google Scholar]
  3. Beidel DC, Turner SM, Dancu CV. Physiological, cognitive and behavioral aspects of social anxiety. Behaviour Research and Therapy. 1985;23:109–117. doi: 10.1016/0005-7967(85)90019-1. [DOI] [PubMed] [Google Scholar]
  4. Beidel DC, Turner SM, Jacob RG, Cooley MR. Assessment of social phobia: Reliability of an impromptu speech task. Journal of Anxiety Disorders. 1989;3:149–158. [Google Scholar]
  5. Boone ML, McNeil DW, Masia CL, Turk CL, Carter LE, Ries BJ, Lewin MR. Multimodal comparisons of social phobia subtypes and avoidant personality disorder. Journal of Anxiety Disorders. 1999;13:271–292. doi: 10.1016/s0887-6185(99)00004-3. [DOI] [PubMed] [Google Scholar]
  6. Brown TA, Lehman CL, Campbell L, DiNardo PA. Reliability and structure of DSM-IV diagnoses and symptom dimensions using the Anxiety Disorders Interview Schedule for DSM-IV: Lifetime version (ADIS-IV-L) 2000 Manuscript in preparation. [Google Scholar]
  7. Clark DM, Wells A. A cognitive model of social phobia. In: Heimberg RG, Liebowitz MR, Hope DA, Schneier FR, editors. Social Phobia: Diagnosis, assessment and treatment. Guilford; New York: 1995. pp. 69–93. [Google Scholar]
  8. Daly JA, McCrokey JC. Occupational desirability and choice as a function of communication apprehension. Journal of Counseling Psychology. 1975;22:309–313. [Google Scholar]
  9. DiNardo PA, Brown TA, Barlow DH. The Anxiety Disorders Interview Schedule for DSM-IV: Lifetime Version (ADIS-IV-L) Graywind Publications; Albany, NY: 1994. [Google Scholar]
  10. DiNardo PA, Moras K, Barlow DH, Rapee RM. Reliability of DSM-III-R anxiety disorder categories using the Anxiety Disorders Interview Schedule-Revised (ADIS-R) Archives of General Psychiatry. 1993;50:251–256. doi: 10.1001/archpsyc.1993.01820160009001. [DOI] [PubMed] [Google Scholar]
  11. Dodge CS, Hope DA, Heimberg RG, Becker RE. Evaluation of the Social Interaction Self-Statement Test with a social phobic population. Cognitive Therapy and Research. 1988;12:211–222. [Google Scholar]
  12. Feske U, Chambless DL. Cognitive behavioral versus exposure only treatment for social phobia: A meta-analysis. Behavior Therapy. 1995;26:695–720. [Google Scholar]
  13. Gilkinson H. Social fears as reported by students in college speech classes. Speech Monography. 1942;9:141–160. [Google Scholar]
  14. Glass CR, Arnkoff DB. Validity issues in self-statement measures of social phobia and social anxiety. Behaviour Research and Therapy. 1994;32:255–267. doi: 10.1016/0005-7967(94)90120-1. [DOI] [PubMed] [Google Scholar]
  15. Glass CR, Merluzzi TV, Biever JI, Larsen KH. Cognitive assessment of social anxiety: Development and validation of a self-statement questionnaire. Cognitive Therapy and Research. 1982;6:37–55. [Google Scholar]
  16. Heimberg RG. Cognitive assessment strategies and the measurement of outcome of treatment for social phobia. Behaviour Research and Therapy. 1994;32:269–280. doi: 10.1016/0005-7967(94)90121-x. [DOI] [PubMed] [Google Scholar]
  17. Hofmann SG. Self-focused attention before and after exposure treatment of social phobia. Behaviour Research and Therapy. 2000a;38:717–725. doi: 10.1016/s0005-7967(99)00105-9. [DOI] [PubMed] [Google Scholar]
  18. Hofmann SG. Treatment of social phobia: Potential mediators and moderators. Clinical Psychology: Science and Practice. 2000b;7:3–16. doi: 10.1093/clipsy/7.1.3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hofmann SG. Self-focused exposure treatment for social phobia. Boston University; 1999. Unpublished treatment manual. [Google Scholar]
  20. Hofmann SG, Albano AM, Heimberg RG, Tracey S, Chorpita BF, Barlow DH. Subtypes of social phobia in adolescents. Depression and Anxiety. 1999;9:15–18. [PubMed] [Google Scholar]
  21. Hofmann SG, Newman MG, Ehlers WT, Roth WT. Psychophysiological differences between subtypes of social phobia. Journal of Abnormal Psychology. 1995;104:224–231. doi: 10.1037//0021-843x.104.1.224. [DOI] [PubMed] [Google Scholar]
  22. Hofmann SG, Newman MG, Becker E, Taylor CB, Roth WT. Social phobia with and without avoidant personality disorder: Preliminary behavior therapy outcome findings. Journal of Anxiety Disorders. 1995;9:427–438. [Google Scholar]
  23. Hofmann SG, Roth WT. Issues related to social anxiety among controls in social phobia research. Behavior Therapy. 1996;27:79–91. [Google Scholar]
  24. Hollon SD, Kendall PC. Cognitive self-statements in depression: Development of an automatic thoughts questionnaire. Cognitive Therapy and Research. 1980;4:383–395. [Google Scholar]
  25. Holt CS, Heimberg RG, Hope DA, Liebowitz MR. Situational domains of social phobia. Journal of Anxiety Disorders. 1992;6:63–77. [Google Scholar]
  26. Kendall PC, Hollon SD. Anxious self-talk: Development of the Anxious Self-Statements Questionnaire (ASSQ) Cognitive Therapy and Research. 1989;13:81–93. [Google Scholar]
  27. Kendall PC, Hollon SD. Assessing self-referent speech: Methods in the measurement of self-statements. In: Kendall PC, Hollon SD, editors. Assessment strategies for cognitive-behavioral interventions. Academic Press; New York: 1981. pp. 85–118. [Google Scholar]
  28. Kendall PC, Howard BL, Hayes RC. Self-referent speech and psychopathology: The balance of positive and negative thinking. Cognitive Therapy and Research. 1989;13:583–598. [Google Scholar]
  29. Klorman R, Weerts TC, Hastings JE, Melamed BG, Lang PJ. Psychometric description of some specific-fear questionnaires. Behavior Therapy. 1974;5:401–409. [Google Scholar]
  30. Leary MR, Kowalski RM. Social anxiety. Guilford Press; New York: 1995. [Google Scholar]
  31. Lerner J, Safren SA, Henin A, Warman M, Heimberg RG, Kendall PC. Differentiating anxious and depressive self-statements in youth: Factor structure of the Negative Affect Self-Statement Questionnaire among youth referred to an anxiety disorders clinic. Journal of Clinical Child Psychology. 1999;28:82–93. doi: 10.1207/s15374424jccp2801_7. [DOI] [PubMed] [Google Scholar]
  32. Liebowitz MR. Social phobia. Modern Problems of Pharmacopsychiatry. 1987;22:141–173. doi: 10.1159/000414022. [DOI] [PubMed] [Google Scholar]
  33. Lombardo TW. Personal Report of Confidence as a Speaker. In: Hersen M, Bellack AS, editors. Dictionary of behavioral assessment techniques. Pergamon Press; New York: 1988. pp. 347–348. [Google Scholar]
  34. Mannuzza S, Schneier FR, Chapman TF, Liebowitz MR, Klein DF, Fyer AJ. Generalized social phobia: Reliability and validity. Archives of General Psychiatry. 1995;52:230–237. doi: 10.1001/archpsyc.1995.03950150062011. [DOI] [PubMed] [Google Scholar]
  35. McCroskey JC. An introduction to rhetorical communication. Allyn & Bacon; Boston: 1997. [Google Scholar]
  36. McCroskey JC. Validity of the PRCA as an index of oral communication apprehension. Communication Monographs. 1978;45:192–203. [Google Scholar]
  37. McNeil DW, Ries BJ, Turk CL. Behavioral assessment: Self- and other-report, physiology and overt behavior. In: Heimberg RG, Liebowitz MR, Hope DA, Schneier FR, editors. Social Phobia: Diagnosis, assessment and treatment. Guilford Press; New York: 1995. pp. 202–231. [Google Scholar]
  38. Myszka MT, Galassi JP, Ware WB. Comparison of cognitive assessment methods with heterosocially anxious college women. Journal of Counseling Psychology. 1986;33:401–407. [Google Scholar]
  39. Newman MG, Hofmann SG, Trabert W, Roth WT, Taylor CB. Does behavioral treatment of social phobia lead to cognitive changes? Behavior Therapy. 1994;25:503–517. [Google Scholar]
  40. Paul GL. Insight and desensitization in psychotherapy: An experiment in anxiety reduction. Stanford University Press; Stanford, CA: 1966. [Google Scholar]
  41. Rapee RM, Heimberg RG. A cognitive-behavioral model of social anxiety in social phobia. Behaviour Research and Therapy. 1997;35:741–756. doi: 10.1016/s0005-7967(97)00022-3. [DOI] [PubMed] [Google Scholar]
  42. Ronan KR, Kendall PC, Rowe M. Negative affectivity in children: Development and validation of a self-statement questionnaire. Cognitive Therapy and Research. 1994;18:509–528. [Google Scholar]
  43. Spielberger CD, Gorsuch RL, Lushene RE. State-Trait Anxiety Inventory. Consulting Psychologists Press; Palo Alto, CA: 1970. [Google Scholar]
  44. Stein MB, Walker JR, Forde DR. Public speaking fears in a community sample: Prevalence, impact on functioning, and diagnostic classification. Archives of General Psychiatry. 1996;53:169–174. doi: 10.1001/archpsyc.1996.01830020087010. [DOI] [PubMed] [Google Scholar]
  45. Treadwell KRH, Kendall PC. Self-talk in youth with anxiety disorders: States of mind, content specificity, and treatment outcome. Journal of Consulting and Clinical Psychology. 1996;64:941–950. doi: 10.1037//0022-006x.64.5.941. [DOI] [PubMed] [Google Scholar]
  46. Turner SM, Beidel DC, Dancu CV, Stanley MA. An empirically derived inventory to measure social fears and anxiety: The Social Phobia and Anxiety Inventory. Journal of Consulting and Clinical Psychology. 1989;1:35–40. [Google Scholar]
  47. Turner SM, Beidel DC, Larkin KT. Situational determinants of social anxiety in clinic and non-clinic samples: Physiological and cognitive correlates. Journal of Consulting and Clinical Psychology. 1986;54:523–527. doi: 10.1037//0022-006x.54.4.523. [DOI] [PubMed] [Google Scholar]
  48. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology. 1988;54:1063–1070. doi: 10.1037//0022-3514.54.6.1063. [DOI] [PubMed] [Google Scholar]
  49. Watson D, Friend R. Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology. 1969;33:448–457. doi: 10.1037/h0027806. [DOI] [PubMed] [Google Scholar]

RESOURCES