Psychometric properties of an innovative self-report measure: The Social Anxiety Questionnaire for Adults

Abstract

This paper presents the psychometric properties of a new measure of social anxiety, the Social Anxiety Questionnaire for adults (SAQ), composed of 30 items that were developed based on participants from 16 Latin American countries, Spain, and Portugal. Two groups of participants were included in the study: a non-clinical group involving 18,133 persons and a clinical group comprising 334 patients with a diagnosis of social anxiety disorder (social phobia). Exploratory and confirmatory factor analyses supported a 5-factor structure of the questionnaire. The factors were labeled: 1) Interactions with strangers, 2) Speaking in public/talking with people in authority, 3) Interactions with the opposite sex, 4) Criticism and embarrassment, and 5) Assertive expression of annoyance, disgust or displeasure. Psychometric evidence supported the internal consistency, convergent validity, and measurement invariance of the SAQ. To facilitate clinical applications, a ROC analysis identified cut scores for men and women for each factor and for the global score.

Social anxiety disorder (social phobia) is one of the most frequent anxiety disorders, with twelve-month prevalence rates ranging between 6.8% in the USA (Kessler et al., 2005; 2008) to 0.6% in Spain (Haro et al., 2008). Although once a neglected disorder (Liebowitz, Gorman, Fyer, & Klein, 1985), it has attracted considerable attention from clinicians and researchers in recent decades. Numerous measures have been developed to assess this condition, from semi-structured interviews, such as the Anxiety Disorders Interview Schedule (ADIS; Di Nardo, Brown, & Barlow, 1994) to self-report inventories, such as the Liebowitz Social Anxiety Scale (LSAS; Liebowitz, 1987), the Social Phobia and Anxiety Inventory (SPAI; Turner, Beidel, Dancu, & Stanley, 1989), the Social Phobia Scale (SPS; Mattick & Clarke, 1998), the Social Interaction Anxiety Scale (SIAS; Mattick & Clarke, 1998), and the Social Phobia Inventory (SPIN; Connor et al., 2000) as the most representative on an international level. Social anxiety has also been the focus of several recently developed intervention strategies (e.g., Heimberg & Becker, 2002; Hofmann & Otto, 2008; Hope, Heimberg, & Turk, 2010). While many intervention procedures have been empirically validated (e.g., Heimberg et al., 1990) and their use widely accepted (e.g., Caballo, Salazar, Garrido, & Irurtia, 2012; Heimberg, Liebowitz, Hope, & Schneier, 1995), the psychometric properties of social anxiety measures are less well established. Many self-report measures of social anxiety have been questioned on their development, content validity, validation strategies and applicability. Questions have been particularly raised about their item selection and refinement procedures (see Haynes, Richard, & Kubany, 1995). For instance, because the LSAS was originally developed as a clinician-administered measure, based on a small sample (Liebowitz, 1987), its items assess a limited range of social situations. Moreover, some of the items are male gender-biased (“urinating in a pubic bathroom” and “trying to pick up someone”). The items on the Social Phobia Scale (SPS) and the Social Interaction Anxiety Scale (SIAS) were subjectively derived mainly from an initial pool of 164 items that were derived from other fear survey schedules and social anxiety inventories (Mattick & Clarke, 1998). For example, the Social Phobia and Anxiety Inventory's (SPAI) initial item pool was generated by the authors after reviewing available inventories and DSM-III criteria for social phobia (APA, 1980), and by compiling a list of complaints from a patient population (Turner et al., 1989).

In addition, the number and type of factors informing the most popular instruments for the assessment of social anxiety vary considerably (e.g., Oakman, van Ameringen, Mancini, & Farvolden, 2003; Osman, Barrios, Aukes, & Osman, 1995; Peters, Sunderland, Andrews, Rapee, & Mattick, 2012; Romm et al., 2011). For example, studies reported very different number of factors of the LSAS, with authors reporting three (Romm et al., 2011), four (Oakman et al., 2003), five (Baker, Heinrichs, Kim, & Hofmann, 2002), or even eight (Heeren et al., 2012) factors. Similar inconsistent findings in factor solutions have been reported for other social anxiety/phobia measures, such as the SPIN (e. g., Connor et al., 2000; Osório, Crippa, & Loureiro, 2010; Radomsky et al., 2006), the SPAI (e. g., Osman et al., 1995; Turner et al., 1989), or the SPS and SIAS (see Caballo, Salazar, Irurtia, Arias, & Nobre [2013] for a review of these findings).

Another limitation of many self-report instruments is the difficulty in differentiating generalized from the “performance only” subtype of social anxiety disorder (Bhogal & Baldwin, 2007; Rytwinski, et al., 2009). Although the DSM-5 (American Psychiatric Association, 2013) no longer includes the “generalized” subtype specifier and instructs assessors to assign a “performance only” subtype specifier, only the LSAS included items on specific social situations. Additionally, most self-report instruments on social anxiety have been developed within English-speaking cultures, primarily North America, the United Kingdom, and Australia. Often, these instruments are applied in other countries or cultures without examining their culturally relevant psychometric properties (e.g., García-López, Olivares, Hidalgo, Beidel, & Turner, 2001; Osório et al., 2010; Radomsky et al., 2006; Terra et al., 2006; Van Dam-Baggen, Kraaimaat, & Elal, 2003). Finally, the samples used in these studies have often been small (e.g., van Dam-Baggen et al., 2003) and drawn from very few countries (e.g., Cox, Clara, Sareen, & Stein, 2008; Marques, Robinaugh, LeBlanc, & Hinton, 2011).

Considering the limitations of previous research, the ultimate goal of this project was to develop a psychometrically sound self-report measure of social anxiety in a broad range of social situations that would be valid and useful with persons from Spain, Portugal, and Latin America. The initial pool of more than ten thousand situations, gathered over six years by the snowball method, was reduced with subsequent data analysis and experts judgments. Large samples of participants from many countries were used, and different methods of statistical analysis were applied (see Caballo, Salazar, Irurtia, et al., 2010; Caballo, Salazar, et al., 2012 for a detailed description of the procedure). The final version of the Social Anxiety Questionnaire for adults (SAQ) consisted of 30 items with five subscales. Preliminary support for the psychometric properties of this scale was found with a sample of non-clinical university students from one country (Caballo, Salazar, Arias, et al., 2010).

The present research examined the psychometric characteristics of the final version of the SAQ with clinical and non-clinical samples from most Latin American countries, Spain, and Portugal. The current study examined the factor structure, internal consistency, convergent validity, and measurement invariance across countries and clinical status, education, age, and gender of participants. Finally, we examined the cut scores of the measure and its dimensions for their use in research and clinical settings.

Method

Participants

The first group of participants consisted of 18,133 non-clinical individuals (M: 25.38 years, SD: 9.98; range: 16-87 years) from 18 countries (22.98% Mexico, 16.38% Colombia, 14.29% Spain, 10.12% Peru, 7.88% Brazil, 4.30% Argentina, 3.51% Uruguay, 3.06% Venezuela, 3.02% Puerto Rico, 2.80% Portugal, 2.28% Chile, 2.19% Paraguay, 1.42% Costa Rica, 1.38% Honduras, 1.20% Bolivia, 1.10% El Salvador, 1.09% Dominican Republic, and 1.00% Guatemala). The sample included 10,300 women (M: 25.14 years, SD: 9.87) and 7,793 men (M: 25.70 years, SD: 10.11), with 40 participants not reporting their gender. The participants had different levels of education and types of occupations at the time of the assessment: 20.46% were university psychology students, 38.26% were university students from other majors, 12.21% were workers with a university degree, 8.78% were workers with no university degree, 10.13% were high school students, 2.17% were psychologists, and 7.20% could not be included in any of the former categories (e.g., retired or unemployed). No data on occupation were available for the remaining 0.78% of participants. The present sample is not in any way related to the samples of former studies.

The second group of participants consisted of 334 patients (M: 31.94 years, SD: 12.15; range= 16-72) from 7 countries (105 from Mexico, 98 from Spain, 41 from Argentina, 39 from Brazil, 29 from Colombia, 13 from Chile, and 9 from Peru). The sample consisted of 208 women (M: 32.90 years; SD: 12.09) and 126 men (M: 30.36 years; SD: 12.13). For inclusion in this group, patients had to meet a primary diagnosis of social phobia (social anxiety disorder) according to the criteria of the DSM-IV-TR (American Psychiatric Association, 2000) or ICD-10 (World Health Organization, 1992). Each center conducted its own diagnostic assessment of individual patients based on one of these two nosological systems. These patients were included even if they had other disorders in addition to social anxiety disorder (see Table 1), and invalid cases were removed for several reasons (e.g., incomplete data, presence of psychotic disorders, social anxiety disorder not the primary or one of the primary diagnoses). Furthermore, to be included in the study, patients needed to have a score equal to or above 60 on the LSAS-SR (Liebowitz, 1987; Mennin, et al., 2002). From a pool of 527 patients diagnosed with social anxiety disorder, 334 satisfied all the former criteria. Regarding occupation, 20.36% were workers with a university degree, 21.56% were workers with no university degree, 14.37% were university students from different majors, 10.78% were high school students, 0.60% were university psychology students, 0.90% were psychologists, and 30.54% could not be included in any of the former categories (e.g., retired or unemployed). No data on occupational status were obtained for the remaining 0.90% of participants. The clinical sample used here is not in any way related to the clinical samples of former studies.

Table 1.

Distribution of patients by psychiatric disorders

Psychiatric Disorder	Women	Men	Total
Social anxiety disorder	66	60	126
Social anxiety disorder + other anxiety disorder	25	21	46
Social anxiety disorder + mood disorder	69	25	94
Social anxiety disorder + other anxiety disorder + mood disorder	5	3	8
Social anxiety disorder + avoidant personality disorder	1	4	5
Social anxiety disorder + personality disorder (except avoidant)	9	4	13
Social anxiety disorder + other anxiety disorder + personality disorder (except avoidant)	4	1	5
Social anxiety disorder + eating disorder	13	0	13
Social anxiety disorder + eating disorder + personality disorder (except avoidant)	4	0	4
Social anxiety disorder + substance use disorder	0	1	1
Social anxiety disorder + mood disorder + substance use disorder	2	0	2
Social anxiety disorder + one other disorder (not included above)	6	6	12
Social anxiety disorder + two other disorders (not included above)	2	1	3
Social anxiety disorder + three other disorders (not included above)	2	0	2
Total	208	126	334

Measures

Social Anxiety Questionnaire for adults (SAQ)

The SAQ (Caballo, Salazar, Arias, et al., 2010; Caballo, Salazar, Irurtia, et al., 2010; Caballo, Salazar, et al., 2012) is the final version of the SAQ-A and its 30-item format has been reached through several recent studies (see the Introduction). Manuscripts already published addressing the development of this instrument have used a provisional version (i.e., 512-item, 118-item, or 82-item versions). Caballo, Salazar, et al. (2012) applied an 82-item version of the SAQ to most Latin American countries, Portugal and Spain and arrived at the five dimensions and 30-item final version (SAQ) used in this study (and known formerly as SAQ-A30). This final version has only been applied to a non-clinical sample of Spanish university students (Caballo, Salazar, Arias, et al., 2010) and the SAQ psychometric characteristics included here refers to that study. Each item of the SAQ could be answered on a five-point Likert scale to indicate the level of unease, stress or nervousness in response to each social situation: 1 = Not at all or very slight, 2 = Slight, 3 = Moderate, 4 = High, and 5 = Very high or extremely high. It has five dimensions (factors): 1) Speaking in public/ talking with people in authority, 2) interactions with the opposite sex, 3) assertive expression of annoyance, disgust or displeasure, 4) criticism and embarrassment, and 5) interactions with strangers. Each dimension consists of six items distributed randomly throughout the questionnaire. There is a score for each dimension (focus of the measure) and a global score for the whole questionnaire. Caballo, Salazar, Arias, et al. (2010) found that Cronbach's alpha for the whole questionnaire was .91 and split-halves reliability coefficient (Guttman) was .93. Pearson correlations found with the LSAS-SR (N=15,504) was .70 with the LSAS-Anxiety subscale and .66 with the whole LSAS-SR. More information on the development of the questionnaire and how the final SAQ was reached can be found in the references provided earlier.

Liebowitz Social Anxiety Scale (LSAS)

The LSAS (Liebowitz, 1987) is a 24-item interviewer-rated instrument that assesses fear/anxiety and avoidance of specific social situations. Respondents are asked to rate their fear/anxiety (LSAS-Anxiety subscale) on a four-point scale ranging from 0 (none) to 3 (severe) (first column), and avoidance (LSAS-Avoidance subscale) on a four-point scale ranging from 0 (never) to 3 (usually) (second column). However, the LSAS has also been used as a self-report instrument (LSAS-SR) in the literature (e.g., Fresco et al., 2001; Baker et al., 2002). Mennin et al. (2002) report a cut score for the LSAS-SR of between 30 and 60 for nongeneralized social anxiety disorder and of higher than 60 for generalized social anxiety disorder. The same cut points were found by Rytwinski et al. (2009). The authors reported that 30 and 60 on the LSAS-SR provided the best balance of sensitivity and specificity for classifying participants with social anxiety and generalized social anxiety disorder, respectively. With regards to the psychometric characteristics of the Spanish version, González et al. (1998) found a 4-factor structure of the LSAS-SR explaining 48.9% of the variance; the internal consistency reliability coefficient (Cronbach's α) for the LSAS-SR Anxiety subscale was 0.87, and 0.88 for the LSAS-SR Avoidance subscale. The former authors and Bobes et al. (1999) concluded that the LSAS-SR is suitable for use in the clinical research and assessment of patients with social anxiety disorder in Spain. Recently, Caballo et al. (2013) found a 5-factor structure of the LSAS-SR explaining 52.32% of the variance and with an internal consistency reliability coefficient (Cronbach's α) of .93. Given that the avoidance and anxiety subscales are of questionable discriminant evidence (Heimberg et al., 1999; Oakman et al., 2003), these authors considered only the LSAS-SR Anxiety subscale. Regarding the Portuguese version of the LSAS-SR, Terra et al. (2006) found a 5-factor structure for the LSAS-SR explaining 52.9% of the variance, and the Cronbach's α for the LSAS-SR total was 0.95.

Procedure

Our CISO-A Research Team is composed of researchers and psychologists from most Latin American countries, Portugal and Spain. The SAQ was developed and administered in collaboration with this team. More than one hundred of collaborators participated in this study applying the two self-report social anxiety measures usually in their work place (see acknowledgments).

The present study employed the final 30-item version for the first time in clinical and non-clinical samples in most Latin American countries. More specifically, the SAQ was administered to 342 patients with social anxiety disorder and 18,133 non-clinical controls. For the assessment of the clinical group, our collaborators administered the two questionnaires of the study (SAQ and LSAS-SR) -individually to patients- at the early stage of the assessment and who had been diagnosed with social anxiety disorder (social phobia) according to the diagnostic criteria of the DSM-IV-TR (American Psychiatric Association, 2000) or ICD-10 (WHO, 1992). Each clinical center based their diagnostic evaluation on clinical interviews to determine whether patients met the diagnostic criteria. In the non-clinical sample, the application of the questionnaires was done in groups. Collaborators working in high schools, colleges or universities administered the questionnaires to people in classes and meetings of teachers or professors. Those working in companies convened voluntary meetings for workers. Some of those working in schools distributed the questionnaires in parents’ meetings. Finally, a few collaborators also administered the questionnaires to people in family reunions. All non-clinical participants voluntarily completed the questionnaires without receiving any monetary compensation. No specific compensation was also offered to patients other than they would have more knowledge about their social anxiety problems.

The questionnaires were administered in Spanish and Portuguese. A back translation procedure was done from the beginning of the development of the questionnaire, with native Brazilian professors and doctoral students living in Spain serving as translators and back translators. The translation to English followed the same pattern. Participation was voluntary and no monetary reward was offered.

Both questionnaires were administered jointly to the clinical and non-clinical samples, but the order of the questionnaires was not fixed. In the non-clinical samples, the questionnaires were always filled out anonymously. The time of application ranged from 10 to15 minutes.

For analyses of results, statistical analyses were performed using SPSS, v. 20, MPlus, v. 6.0 (Muthén & Muthén, 2010), LISREL, v. 8.8. (Scientific Software International, 2006), SAS, v. 9.2 (SAS Institute, 2009), and MedCalc (MedCalc Software, 2010).

Results

The maximum percentage of missing data did not exceed 0.2% in any of the variables analyzed. We assumed that the structure of the missing data was Missing Completely At Random.

Extraction of two random subsamples

Two random subsamples were taken from the 18,133 participants in the total sample (N1 = 9,066; N2 = 9,067), with the aim of implementing the different analyses on the factorial structure of the questionnaire. Polychoric correlations among the items were computed in both subsamples.

The two matrices of correlations (polychoric among the items and Pearson among the total scores in the dimensions) were then calculated through the following procedures:

• 1)Comparison of the correlation structures using structural equations (Jöreskog & Sörbom, 2008; Kline, 2010; Von Eye & Mun, 2005). The goal was to test the hypothesis that the two correlations matrices were invariant, i.e., Φ⁽¹⁾ = Φ ⁽²⁾. The results in the case of the 30 items (Table 2) showed a satisfactory fit between the model and the data. The differences in the contributions to χ² from both subsamples were very small (49.81% by the first one versus 50.19% by the second). The Root Mean Square Error of Approximation (RMSEA) value was lower than the value usually taken as the cut point (.05), and all the other indices revealed a perfect fit (TLI = 1.00, IFC = 1.00). These results support the equivalence of both correlation matrices. Similar results were observed when comparing the correlations of the scores obtained in the five factors of the scale by both subsamples. The chi-square test was not significant (χ²₍₁₅₎ = 9.81, p = .830) and the contribution was similar for both subsamples (50.34% in the first one and 49.66% in the second). All indices of partial fit were highly satisfactory.
• 2)The second procedure involved the individual comparison of the pairs of correlations (435 in the case of the items and 10 in the case of the factors). Table 3 shows z values (lower half of the matrix) and the associated significance levels (upper half) for the factors. In the analysis of item correlations, only 10 of the 435 comparisons reached a z value equal to or higher than the critical value of 2.58, thereby confirming the results obtained by means of the structural equation methodology. The same was true for the analysis of the correlations across the scores obtained in the five factors: of the 10 comparisons, none of them showed significant differences with p < .01 (see Table 3). Consequently, it can be assumed that the subjects from the two subsamples randomly extracted from the global sample have statistically identical correlation structures when considering the questionnaire's 30 individual items and five factors.

Table 2.

Fit of the models (items and factors)

	30 items		5 factors
	Sub-sample 1 (N = 9,066)	Sub-sample 2 (N = 9,067)	Sub-sample 1 (N = 9,066)	Sub-sample 2 (N = 9,067)
Contribution to χ2	388.10	391.03	4.94	4.87
% Contribution to χ2	49.81	50.19	50.34	49.66
SRMR	.0071	.0071	.0040	.0040
GFI	1.00	1.00	1.00	1.00
DF	465		15
χ 2	779.13 (p = .000)		9.81 (p = .830)
RMSEA (90% CI)	.0086 (.0075; .0097)		.0000 (.0000; .0060)
P-close	1.00		1.00
TLI	1.00		1.00
CFI	1.00		1.00

Note. GFI = Goodness of Fit Index; χ² = Minimum Fit Function Chi-Square; RMSEA = Root Mean Square Error of Approximation; 90% CI = 90 Percent Confidence Interval for RMSEA; P-close = P-Value for Test of Close Fit (RMSEA < 0.05); TLI = Tucker-Lewis Index; CFI = Comparative Fit Index.

Table 3.

Pearson correlation coefficients among the 5 factors of the questionnaire, z values and significance levels of the differences

Correlations							z values and significance levels
		F1	F2	F3	F4	F5			F1	F2	F3	F4	F5
		Subsample 2							Significance levels
F1	Subsample 1		.58	.56	.42	.48	F1	z values		.180	.200	.332	.729
F2		.60		.56	.49	.45	F2		1.34		.196	.430	.396
F3		.57	.58		.47	.47	F3		1.28	1.29		.796	.388
F4		.40	.48	.47		.55	F4		−.97	−.79	.26		.923
F5		.47	.46	.48	.55		F5		−.35	.85	.86	.10

Note. F1. Interactions with strangers; F2. Speaking in public/Talking with people in authority; F3. Interactions with the opposite sex; F4. Criticism and embarrassment; F5. Assertive expression of annoyance, disgust or displeasure.

Exploratory factor analysis

In order to confirm the optimal number of factors, a parallel analysis (Horn, 1965) with the data from the first subsample was conducted using the Monte Carlo procedure with 200 replications. Parallel analyses compared the observed Eigenvalues extracted from the observed correlation matrix to be analyzed with those obtained from uncorrelated normal variables (parallel components derived from random data). The results showed that the 5-factor solution was the best fit to our data, given that only the Eigenvalues of these five factors were greater than the randomly generated Eigenvalues.

An exploratory factor analysis (EFA) by principal components with Promax rotation was computed on the two random subsamples of non-clinical participants. This EFA identified five factors with Eigenvalues higher than 1.00 in both subsamples, explaining 53.60% and 56.79% of the cumulative variance, respectively (see Table 4 for a detailed description). In all cases, the items in both samples loaded on the same factors as the original studies (Caballo, Salazar, Arias, et al., 2010; Caballo, Salazar, et al., 2012). The congruence coefficient computed for each factor (higher than .95 in all cases) suggests that the factors of the two subsamples were virtually identical (Lorenzo-Seva & ten Berge, 2006).

Table 4.

Factor loadings for both non-clinical subsamples (N1 = 9,066; N2 = 9,067) and congruence coefficients

	Subsample l					Subsample 2
Factors and items of the SAQ	F1	F2	F3	F4	F5	F1	F2	F3	F4	F5
F1. Interactions with strangers	Eigenvalue = 10.38					Eigenvalue = 10.34
Congruence coefficient = .999	Variance % = 34.61					Variance % = 34.47
13. Maintaining a conversation with someone I've just met	.79	.07	−.01	−.06	.03	.78	.08	−.01	−.06	.05
10. Making new friends	.79	−.01	.04	−.25	.11	.78	.01	.05	−.24	.10
17. Talking to people I don't know at a party or a meeting	.76	.04	.02	.08	−.05	.78	.00	.00	.14	−.07
19. Looking into the eyes of someone I have just met while we are talking	.70	.05	.04	−.04	.01	.69	.04	.07	−.06	.04
15. Greeting each person at a social meeting when I don't know most of them	.61	.12	−.07	.16	−.01	.62	.11	−.08	.18	.02
22. Attending a social event where I know only one person	.42	.05	.09	.36	−.07	.45	.06	.07	.40	−.09
F2. Speaking in public/Talking with people in authority	Eigenvalue = 2.27					Eigenvalue = 2.26
Congruence coefficient = .998	Variance % = 7.58					Variance % = 7.45
3. Speaking in public	−.08	.89	.04	−.11	.01	−.09	.9l	.02	−.10	−.01
12. Having to speak in class, at work, or in a meeting	.11	.86	−.06	−.11	.00	.09	.89	−.08	−.10	.00
18. Being asked a question in class by the teacher or by a superior in a meeting	.12	.70	−.01	.11	−.02	.13	.72	−.02	.08	−.02
7. Participating in a meeting with people in authority	.07	.62	.01	.05	.09	.06	.6l	.04	.03	.08
29. Talking to a superior or a person in authority	.19	.60	−.01	.09	.06	.17	.6l	.05	.05	.05
25. While having dinner with colleagues, classmates or workmates, being asked to speak on behalf of the entire group	−.01	.59	.07	.20	−.01	.00	.57	.02	.26	−.01
F3. Interactions with the opposite sex	Eigenvalue = 1.60					Eigenvalue = 1.67
Congruence coefficient = .996	Variance % = 5.34					Variance % = 5.57
4. Asking someone attractive of the opposite sex for a date	−.22	.07	.86	.01	.02	−.22	.07	.85	.02	.02
30. Telling someone I am attracted to that I would like to get to know them better	.03	−.02	.84	.01	.01	.00	.00	.84	.03	.02
27. Asking someone I find attractive to dance	.02	−.02	.79	.06	−.03	.02	−.06	.78	.09	−.03
23. Starting a conversation with someone of the opposite sex that I like	.30	−.05	.66	.05	−.02	.31	−.05	.67	.02	−.03
20. Being asked out by a person I am attracted to	.25	−.03	.65	−.06	.03	.25	−.01	.64	−.07	.01
6. Feeling watched by people of the opposite sex	.23	.13	.36	−.05	.16	.23	.10	.42	−.08	.17
F4. Criticism and embarrassment	Eigenvalue = 1.51					Eigenvalue = 1.57
Congruence coefficient = .995	Variance % = 5.05					Variance % = 5.22
24. Being reprimanded about something I have done wrong	−.05	−.05	.06	.73	.10	−.05	−.04	.06	.7l	.11
16. Being teased in public	.09	.07	.01	.7l	−.15	.10	.02	.05	.7l	−.14
8. Talking to someone who isn't paying attention to what I am saying	−.11	−.10	−.08	.69	.18	−.07	−.13	−.07	.70	.18
28. Being criticized	−.07	.03	.01	.70	.07	−.10	.07	.01	.67	.12
21. Making a mistake in front of other people	.04	.18	.14	.60	−.05	.04	.21	.11	.60	−.07
1. Greeting someone and being ignored	−.05	−.01	.00	.4l	.24	−.01	−.01	−.04	.46	.21
F5. Assertive expression of annoyance, disgust or displeasure	Eigenvalue = 1.21					Eigenvalue = 1.22
Congruence coefficient = .997	Variance % = 4.02					Variance % = 4.08
2. Having to ask a neighbor to stop making noise	−.24	.12	.09	.03	.7l	−.23	.11	.07	.07	.67
26. Telling someone that their behavior bothers me and asking them to stop	.08	.04	.03	.07	.67	.06	.04	.00	.08	.68
14. Expressing my annoyance to someone that is picking on me	.27	−.12	−.14	.10	.65	.25	−.10	−.12	.06	.67
5. Complaining to the waiter about my food	−.09	.17	.08	−.08	.64	−.10	.14	.15	−.07	.6l
9. Refusing when asked to do something I don't like doing	.15	−.05	−.08	.22	.52	.15	−.06	−.05	.14	.58
11. Telling someone that they have hurt my feelings	.20	−.09	.09	.04	.52	.20	−.06	.05	.05	.54

We also conducted an EFA with the clinical sample. The best solution based on the scree-test was again a 5-factor structure with Eigenvalues higher than 1.00, explaining 47.77% of the cumulative variance. The first factor Speaking in public/Talking with people in authority (Eigenvalue: 6.75) explained 22.51% of the variance. The second factor Criticism and embarrassment (Eigenvalue : 2.39) explained 7.98% of the total variance. Factor 3 Interactions with the opposite sex (Eigenvalue: 1.97) explained 6.57% of the variance. Factor 4 Interactions with strangers (Eigenvalue: 1.65) explained 5.50% of the variance. Finally, Factor 5 Assertive expression of annoyance, disgust, or displeasure (Eigenvalue: 1.56) explained 5.21% of the variance. All the items loaded on the same factors as the original studies (see above), except for two items of F2 (“Criticism and embarrassment”), which loaded on F5 (“Assertive expression of annoyance, disgust, or displeasure”), and two items of F5, which loaded on F2.

Evidence based on the internal structure of the SAQ

We tested the evidence based on the internal structure of the SAQ by means of confirmatory factor analysis (CFA; LISREL, v. 8.8.; Scientific Software International, 2006) and exploratory structural equation modeling (ESEM; MPlus, v. 6.0; Muthén & Muthén, 2010). The ESEM models have recently been developed for solving the problems usually found in CFA models. In these latter models the necessity to fix to zero the saturations frequently leads to a major modification of the model in order to obtain a better fit. The ESEM models avoid this problem as they do not impose such restrictions (Asparouhov & Muthén, 2009; Marsh, 2007; Marsh, Hau, & Grayson, 2005). The CFA and ESEM were completed with the second subsample (see above) of Latin American, Spanish, and Portuguese participants (n₂=9,067) using Weighted Least Squares Mean and Variance-adjusted (WLSMV) estimation. Four models were tested: M1 = Unifactorial model; M2 = Five correlated factors model; M3 = Five first-order factors and one second-order factor model; and M4 = ESEM. Model 3 is similar to Model 2, but we hypothesized that there could be a general factor, named “social anxiety” (second layer), linking the other five group factors (first layer). The analyses were conducted on the covariance matrix of the items of the second subsample (N= 9,067). Table 5 shows the results obtained with these four models. Best fit models seem to support the five correlated factors model and the ESEM.

Table 5.

Results of the models tested with the second non-clinical sample (CFA) (N2= 9,067)

	M1	M2	M3	M4
χ 2	10091.40	6664.95	7009.88	7303.94
df	405	395	400	295
p	.000	.000	.000	.000
RMSEA	.098	.052	.054	.052
(CI 90%)	(.097-.098)	(.052-.053)	(.053-.055)	(.051-.053)
p	.000	.000	.000	.004
CFI	.92	.98	.98	.96
TLI	.91	.98	.97	.95
SRMR	.080	.047	.051	.024

Note: M1 = Unifactorial model; M2 = Five correlated factors model; M3 = Five first-order factors and one second-order factor model; M4 = ESEM (exploratory structural equation modeling).

Given that the EFA with the clinical sample has four items that did not load in the same factors that the non-clinical sample, those same CFA and ESEM analyses were undertaken with the clinical sample, testing the same four models. Table 6 shows the results obtained with these four models. The best fitting models seem to support again the five correlated factors model and the ESEM. Model 3 reached a satisfactory fit, but it was not as good as Model 2 (in non-clinical and, particularly, in clinical samples). Therefore, we chose M2 as the best fitting model.

Table 6.

Results of the models tested with the clinical sample (CFA) (N= 334)

	M1	M2	M3	M4
χ 2	1562.17	812.52	859.26	375.30
df	405	395	400	295
p	.000	.000	.000	.000
RMSEA	.093	.056	.059	.040
(CI 90%)	(.088-.098)	(.051-.062)	(.053-.064)	(.027-.052)
p	.000	.030	.000	.907
CFI	.87	.95	.95	.96
TLI	.86	.95	.94	.94
SRMR	.100	.075	.079	.045

Note: M1 = Unifactorial model; M2 = Five correlated factors model; M3 = Five first-order factors and one second-order factor model; M4 = ESEM (exploratory structural equation modeling).

Convergent evidence for the SAQ

The convergent evidence for the SAQ was assessed via correlations with the LSAS-SR, which was administered (together with the SAQ) to the clinical and nonclinical samples. The correlations between the total score on the SAQ and the LSAS-SR Anxiety subscale and LSAS-SR total score were moderate in both samples: patients (.56 and .55) and non-patient (.65 and .67). The correlations between factors on the SAQ and the LSAS-SR scores were lower (Table 7).

Table 7.

Correlations (Pearson) between the SAQ and its factors and the LSAS-SR in clinical and non-clinical samples

SAQ	Clinical sample (N = 334)			Non-Clinical sample (N = 18,133)
	LSAS Anx.	LSAS Av.	LSAS Total	LSAS Anx.	LSAS Av.	LSAS Total
F1. Interactions with strangers	.46	.39	.46	.56	.49	.56
F2. Speaking in public/Talking with people in authority	.40	.26	.36	.58	.46	.56
F3. Interactions with the opposite sex	.31	.28	.32	.52	.41	.50
F4. Criticism and embarrassment	.39	.34	.40	.45	.37	.44
F5. Assertive expression of annoyance, disgust or displeasure	.37	.30	.37	.46	.38	.45
Total	.55	.44	.54	.67	.54	.65

Note. All correlations significant at p < .0001; LSAS-SR= Liebowitz Social Anxiety Scale-Self Report; LSAS Anx.= Liebowitz Social Anxiety, Anxiety subscale; LSAS Av.= Liebowitz Social Anxiety, Avoidance subscale.

Reliability of the SAQ and the LSAS-SR

The internal consistency reliability coefficient estimates (Cronbach's α) of the SAQ total score and five factors were calculated for the two non-clinical subsamples (N1 and N2) and for the clinical one. Cronbach's α was .922 and .811 for non-clinical subsamples N1 and N2, respectively, for the first factor, Interactions with strangers, and .800 for the clinical sample; .804 and .848 for the second factor, Speaking in public/Talking with people in authority, in the non-clinical subsamples, and .792 in the clinical sample; .845 and .836 for the third factor, Interactions with the opposite sex, in the non-clinical subsamples, and .752 in the clinical sample; .842 and .762 for the fourth factor, Criticism and embarrassment, in the non-clinical subsamples, and .662 in the clinical sample; .760 and .751 for the fifth factor, Assertive expression of annoyance, disgust or displeasure, in the non-clinical subsamples, and .660 in the clinical sample; and .922 and .921 for the global score of the SAQ for the non-clinical subsamples N1 and N2, respectively, and .877 for the clinical sample. The reliability of the questionnaire obtained for the whole non-clinical sample through the Guttman split-halves reliability coefficient was .931, and .900 for the clinical sample.

The internal consistency reliability coefficient estimates (Cronbach's α) of the LSAS-SR total score and the two subscales were calculated on the whole non-clinical sample. Cronbach's α was .895 and .873 for the Anxiety and Avoidance subscales, respectively, and .930 for the whole scale. Cronbach's α of the LSAS-SR for the clinical sample was .821 and .837 for the Anxiety and Avoidance subscales, respectively, and .895 for the whole scale.

Analysis of invariance

To test for equivalency of the measure (i.e., to examine the measurement invariance), we conducted confirmatory factor analyses across different groups in this study. There were four common models to test this invariance: (1) Configural invariance examines whether the groups have the same (invariant) factor structure; (2) weak or metric invariance examines whether the groups have the same facto loadings; strong or scalar invariance tests whether the observed scores are related to the latent scores (i.e., whether the groups have the same item intercepts); and strict or residual invariance examines whether the groups have the same item residual variances.

Based on the five correlated factors model, the invariance regarding sex was calculated for the second non-clinical subsample (N2) and for the clinical sample. In the non-clinical sample, RMSEA values fell between .052 and .057, whereas in the clinical sample these values fell between .075 and .105. Using ΔCFI> .01 as criterion (Wu, Li, & Zumbo, 2007), the configural, weak, strong, and strict invariance was supported for the non-clinical sample but only the configural and weak invariance was supported for the clinical sample (see Table 8), suggesting that the social anxiety construct is similar in patients and non-patients (configural invariance) and that both groups show similar factor loadings (weak invariance).

Table 8.

Fit indices for invariance models

Non-Clinical Sample	DF	N(M)	N(F)	χ2 (p)	Δχ2 (p)	RMSEA	TLI	CFI	ACFI	SRMR
Configural	800	3928	5119	10945.46 (.000)	---	.053 (.052; .054)	.974	.976	--	.061
Weak	830	3928	5119	11391.67 (.000)	446.21 (.000)	.053 (.052; .054)	.974	.975	−.001	.065
Strong	860	3928	5119	12686.24 (.000)	1294.57 (.000)	.055 (.054; .056)	.972	.972	−.003	.064
Strict	890	3928	5119	13375.30 (.000)	689.06 (.000)	.056 (.055; .057)	.971	.971	−.001	.067

Clinical Sample	DF	N(M)	N(F)	χ2 (p)	Δχ2 (p)	RMSEA	TLI	CFI	ACFI	SRMR
Configural	800	126	208	2110.19 (.000)	---	.079 (.075 ; .083)	.902	.910	--	.123
Weak	830	126	208	2258.80 (.000)	148.61 (.000)	.081 (.077; .085)	.897	.902	−.008	.120
Strong	860	126	208	2795.03 (.000)	536.23 (.568)	.093 (.089; .096)	.865	.867	−.035	.116
Strict	890	126	208	3277.57 (.000)	482.54 (.000)	.101 (.097; .105)	.840	.836	−.031	.118

Differences and similarities regarding several variables of the study

We examined differences between countries, level of education and age in the scores on the five dimensions using effect sizes r and ω. All effect sizes were very small for all the dimensions in education (from .073 to .149), country (from .134 to .210), and age (from .051 to .101). Therefore, we did not take these differences into account in any of the subsequent analyses. However, given that we have found significant differences between men and women in social anxiety in our former studies (Caballo et al., 2008; 2010; 2012), we employed Student's t-tests to examine sex differences in the clinical and non-clinical samples. Table 9 shows the results suggesting that there are statistically significant differences between men and women in all dimensions, as well as the global score of the SAQ in the non-clinical sample. However, these differences were minor (Cohen's d< 0.20) in two of these dimensions (Interactions with strangers, and Assertive expression of annoyance, disgust or displeasure). A similar pattern of results emerged for the clinical sample, where the differences in three of the dimensions did not reach the level of statistical significance (probably due to the small size of the sample). However, Cohen's d was equal or above 0.20 in four of the five dimensions (only one dimension, “Interactions with strangers”, clearly did not show differences between male and female patients.) The differences between men and women were small to mediumsized (0.20 ≤d≤ 0.50)

Table 9.

Gender differences in clinical and non-clinical samples in the five dimensions of social anxiety assessed by the SAQ

	M (SD)					M (SD)
Dimensions of the SAQ	Non-clinical males (N= 7,793)	Non-clinical females (N= 10,300)	t	p	d	Clinical males (N= 126)	Clinical females (N= 208)	t	p	d
1. Interactions with strangers	13.09 (4.67)	13.32 (4.83)	3.16	.001	0.04	21.42 (4.44)	20.71 (5.29)	1.27	.203	0.14
2. Speaking in public/Talking with people in authority	14.66 (5.03)	16.54 (5.54)	23.37	.000	0.35	22.47 (4.64)	23.40 (4.64)	1.78	.076	0.20
3. Interactions with the opposite sex	15.14 (5.27)	17.55 (5.45)	29.86	.000	0.45	22.97 (4.22)	23.92 (4.22)	2.00	.046	0.22
4. Criticism and embarrassment	17.21 (4.81)	18.53 (4.82)	18.21	.000	0.27	22.33 (3.95)	23.92 (3.81)	3.65	.000	0.41
5. Assertive expression of annoyance, disgust or displeasure	15.18 (4.62)	15.89 (4.77)	10.10	.000	0.15	21.46 (3.92)	22.27 (4.17)	1.77	.078	0.20
SAQ total	75.24 (18.95)	81.83 (19.66)	22.46	.000	0.34	110.66 (14.80)	114.23 (15.91)	2.04	.042	0.23

Establishing cut scores for the dimensions and the global questionnaire

Receiver operating characteristics (ROC) analyses allow researchers to determine the accuracy of the measure to discriminate individuals with social anxiety disorder (SAD) from individuals without this diagnosis. ROC analysis is based on logistic regression with a continuous predictor variable and a dichotomous criterion variable. Once the logistic regression equation is estimated, the probability of each value of the predictor and its associated sensitivity (the likelihood of correctly identifying a disordered individual as meeting criteria for SAD) and specificity (the likelihood of correctly identifying a healthy individuals as a person not meeting criteria for SAD) are derived (Mennin et al., 2002). The score that maximizes both sensitivity and specificity is usually considered the best cut value for the scale. ROC analysis was used to determine the optimal cut scores for the SAQ for the diagnosis of SAD. We used the entire clinical sample (126 males and 208 females) and those non-clinical participants who scored lower than 60 in the LSAS-SR¹, totaling 6,134 men (M: 25.74 years; SD: 10.05) and 7,457 women (M: 25.25 years; SD: 9.89). The goal was to examine the cut values of the SAQ that corresponded to a diagnosis of SAD by sex, distinguishing between patients with social anxiety disorder and comparing non-clinical subjects (Cohen's d's of these differences, for patients/non-clinical subjects, on the factors and total score of the SAQ were between 1.17 and 2.08). The differences between the cut points according to sex are due to the mean for women being significantly higher than for men (p < .05) in two of the factors and in the total score of the SAQ in the clinical sample, and in the five factors and the total score in the non-clinical sample.

In men, the ROC analysis produced a robust area under the curve (AUC: .959, SE: .006) with a 95% confidence interval between .954 and .964 (z: 76.603, p < .0001) for their classification into social anxiety disorder /non-social anxiety disorder groups. The SAQ total score of 89 provided he best balance between sensitivity (.937) and specificity (.853) and correctly classified 93.7% of the men diagnosed with social anxiety disorder (118 out of 126) and 85.3% (5,153 out of 6,040) without social anxiety disorder (some participants were overlooked because they did not have all the data). Table 10 shows the results with ROC curves in men using the five factors (dimensions) and the total score of the SAQ.

Table 10.

Results for the ROC analysis of the five-factors and total score of the SAQ by gender

											True positive		True negative		False positive		False negative
Gender	Sensitivity	Specificity	Cut	AUC	SE	95% CI			z	p	N	%	N	%	N	%	N	%
Males
F1	.841	.882	17	.924	.013	.917	-	.931	33.340	.000	106	84.1	5398	88.2	724	11.8	20	15.9
F2	.810	.839	18	.898	.015	.890	-	.905	27.507	.000	102	81.0	5151	83.9	986	16.1	24	19.1
F3	.865	.797	18	.901	.013	.894	-	.909	29.966	.000	109	86.5	4882	79.7	1245	20.3	17	13.5
F4	.794	.725	19	.825	.018	.816	-	.836	17.937	.000	100	79.4	4444	72.5	1682	27.5	26	20.6
F5	.730	.865	19	.875	.015	.867	-	.883	25.513	.000	92	73.0	5307	86.6	825	13.5	34	27.0
Total	.937	.853	89	.959	.006	.954	-	.964	76.603	.000	118	93.7	5153	85.3	887	14.7	8	6.3
Females
F1	.812	.792	15	.888	.012	.881	-	.895	32.765	.000	169	81.3	5918	79.2	1555	20.8	39	18.8
F2	.750	.837	20	.874	.012	.867	-	.882	31.348	.000	156	75.0	6265	83.7	1221	16.3	52	25.0
F3	.755	.825	21	..867	.012	.858	-	.875	30.046	.000	157	75.5	6182	82.5	1311	17.5	51	24.5
F4	.769	.787	21	.848	.013	.840	-	.856	27.528	.000	160	76.9	5889	78.7	1597	21.3	48	23.1
F5	.740	.832	19	.874	.012	.867	-	.882	32.365	.000	154	74.0	6222	83.2	1260	16.8	54	26.0
Total	.841	.895	98	.945	.006	.939	-	.950	70.976	.000	175	84.1	6598	89.5	772	10.5	33	15.9
Total sample
F1	.799	.841	16	.901	.009	.896	-	.906	45.077	.000	267	79.9	11431	84.1	2164	15.9	67	20.1
F2	.832	.780	18	.885	.009	.880	-	.890	42.466	.000	278	83.2	10620	78.0	3003	22.0	56	16.8
F3	.895	.718	18	.882	.009	.876	-	.887	42.772	.000	299	89.5	9772	71.8	3848	28.3	35	10.5
F4	.784	.750	20	.840	.011	.834	-	.846	32.496	.000	252	78.4	10208	75.0	3404	25.0	72	21.6
F5	.737	.847	19	.875	.009	.870	-	.881	41.588	.000	246	73.7	11528	84.7	2086	15.3	88	26.4
Total	.925	.821	90	.951	.004	.947	-	.954	103.178	.000	309	92.5	11010	82.1	2400	17.9	25	7.5

Note 1. F1. Interactions with strangers; F2. Speaking in public/Talking with people in authority; F3. Interactions with the opposite sex; F4. Criticism and embarrassment; F5. Assertive expression of annoyance, disgust or displeasure.

Note 2. Non-clinical sample for the ROC analysis was participants with a score on the LSAS-SR lower than 60.

In the female sample, the ROC analysis produced a robust AUC (.945, SE: .006) with a 95% confidence interval between .939 and .950 (z: 70.976, p < .0001) for their classification into social anxiety disorder /non-social anxiety disorder groups. The SAQ total score of 98 provided the best balance between sensitivity (.841) and specificity (.895) and correctly classified 84.1% of the women diagnosed with social anxiety disorder (175 out of 208) and 89.5% (6,598 out of 7,370) without social anxiety disorder (Table 10). Figure 1 shows the data on efficiency, specificity and sensitivity regarding ROC, albeit selecting at random 334 non-clinical subjects to compare with the 334 patients (some data are slightly different because of this random selection).

Figure 1.

ROC, efficiency, specificity and sensitivity curves with histogram of raw scores.

Discussion

The aim of his research was to discover the psychometric characteristics of the SAQ with most Latin American countries, Spain, and Portugal in clinical and non-clinical samples. Two former studies resulted in the final version of the SAQ (from an initial pool of more than 10,000 situations to 30) through a series of statistical and clinical analyses (Caballo, Salazar, Irurtia, et al., 2010; Caballo, Salazar, et al., 2012). The present study examined the psychometric properties of this final 30-item version of the instrument based on a large and representative sample. Our study is not unique to any particular cultural group. Other measures that were developed by North American or Australian groups have been translated and used in many other countries around the world. We took advantage of having access to a very large population. To our knowledge, this is by far the largest sample size of any study developing or examining an instrument to measure social anxiety. Given the changes in the DSM-5 (and in particular the changes in diagnostic subtypes), this measure, with its detailed assessment of the feared social situations, is likely to become a highly valuable tool. The Appendix includes the English translation of our questionnaire

This study addresses the confirmation of the 5-factor structure of the SAQ, the evidence based on its internal structure, its reliability (internal consistency and Guttman split-halves reliability coefficient), its invariance across sex, gender-related differences, and cut scores of the instrument. Although other research has been conducted to validate this self-report measure (Caballo, Salazar, Arias, et al., 2010; Caballo, Salazar, et al., 2012), this was the first time the final version of the SAQ was administered to a large number of subjects from different countries and cultures. We also included a clinical sample. This study again confirmed the excellent psychometric properties of the SAQ, replicating the 5-factor structure of the questionnaire with 16 Latin American countries, Spain, and Portugal in clinical and non-clinical samples. This factorial structure reveals that there are five distinct social dimensions that a person with a social anxiety might fear. Although a theoretical framework that relates to the multidimensionality of social anxiety is not known to date, it is worth noting that recently the DSM −5 (APA, 2013) included a major change in the definition of social anxiety disorder. An individual with a social anxiety disorder must experience intense fear or anxiety in situations of interaction, observation and / or performance. Our study provides empirical evidence on how individuals with social anxiety may fear not only three but up to five social dimensions and these results are consistent with others studies done with adults (see Caballo, Salazar, Irurtia , et al, 2010; Caballo, Salazar, Arias, et al, 2010; Caballo, Salazar, et al, 2012). The existence of five distinct dimensions that form the questionnaire of social anxiety, including the “Performance situations” subtype of the DSM-5 (“Speaking in public”), seems stable in our research, although further studies with other different cultures will be necessary to confirm this five dimensional structure.

One of the main changes from DSM-IV to DSM-5 (American Psychiatric Association, 2013) in the social anxiety disorder diagnosis is the replacement of the generalized subtype by the new Performance situations only subtype. All of our studies consistently point to the multidimensionality of social anxiety revolving around five factors, including Speaking in public. Although this dimension is clearly represented, we did not find it to be more “prevalent or unique” than the other four dimensions. However it accounts for most of the explained variance in the patients sample (Factor 1 in this sample, 22.66%) although not in the two community samples (Factor 2 in these samples, 7.58/7.45%). All five dimensions found in our research could be used as specifiers for a diagnostic system, thereby giving much more information about the kind of situations individuals fear than the usual self-report measures of social anxiety disorder (social phobia) (e. g., LSAS-SR, SPAI, SPIN, SPS, SIAS). Moreover, it is questionable whether there are, in fact, individuals who only fear public speaking (Kerns, Comer, Pincus, & Hofmann, 2013).

Regardless of whether or not social anxiety is generalized (a differentiation that has disappeared in the DSM-5), the identification of those specific situationally-defined dimensions can inform the range of feared social situations and could significantly aid treatment, tailoring its content according to the types of situations feared. For instance, intervention programs for social anxiety could be composed by five modules (corresponding to the five dimensions), based on the dimension(s) feared by the patient, the therapist would apply the corresponding module(s) (implying role-playing and exposure exercises for the specific dimension). And this five-factor solution of social anxiety is consistent with the cognitive-behavioral model of social anxiety disorder, because the situations describing the factors are commonly used exposure situations during cognitive behavioral therapy (e.g., Hofmann & Otto, 2008). Moreover, treatment outcomes that might differ across the situations feared by persons with social anxiety would be more accurately reflected by data from individual factors or scale scores than by data from a total score. In the same way, data from individual factors or scale scores might facilitate the identification of variables that trigger or perpetuate social anxiety in different situations (Caballo, Salazar, et al., 2012). This could be done much better with the SAQ than with the traditional self-report assessment measures of social anxiety.

One clear contribution the SAQ makes is that social anxiety is consistently measured with five dimensions, and therefore provides five scores, one for each dimension. Accordingly, the questionnaire provides information about the kind of social situations individuals fear and whether the social anxiety is more or less generalized (depending on the number of dimensions with high scores). The SAQ has, of course, a global score, although it provides little information (the same could well be said of the other self-report measures). For instance, somebody could have a very high score in one dimension (e. g., interaction with strangers, with a score of 28) but a medium score (17) in the other four dimensions. This individual's global score (96 when adding the five dimensions) would exceed the questionnaire's overall cut score (90). If we only considered the global score little diagnostic information is provided about either the kind of social situations provoking anxiety. However, if we consider the five dimensions, we could say s/he has a significant fear of only one type of situation; that is, of only one dimension. Therefore, considering the score in each of the five dimensions, as the SAQ does, is much more informative and useful for clinical and research purposes than having just the questionnaire's global score, as the other self-report assessment measures do. In this way, the assessment method of the SAQ is different from classical self-report measures of social anxiety in the sense that the main focus of this new measure is on the score of each one of the five dimensions, giving the global score of the questionnaire a secondary role.

The cut scores for the SAQ and its dimensions that we identified in the present study were very similar to those obtained in an earlier one using a previous version of the SAQ (Caballo, Salazar, et al., 2012). As was the case in earlier research, the cut points differed slightly for men and women. We have constantly found significant differences in social anxiety between men and women, as usually reported in the literature (i.e., Baños, Botella, Quero, & Medina, 2007; Beidel & Turner, 1992; Caballo et al., 2008; Caballo et al., 2013; Esau, Muris, & Ederer, 2002; Gültekin & Dereboy, 2011; Schmidt & Richey, 2008; Vieira, Salvador, Matos, García-López, & Beidel, 2013). Surprisingly, however, not one self-report measure of social anxiety for adults differentiates between the cut points for men and women. We understand that considering different cut scores for both sexes is less practical in clinical settings, but it seems necessary because we found significant differences between men and women in all the dimensions of social anxiety. We also found significant sex differences in the clinical and non-clinical sample. Specifically, the magnitude size of these differences was equal or above 0.20 (Cohen's d) in four dimensions and in the global score in the clinical sample as well as in three out of five dimensions and the global score in the non-clinical sample. It should be noted, however, that these differences were always small (Cohen's d ranged from 0.20 to 0.50). These differences should be taken into account when assessing social anxiety in the community and in clinical settings, although the social anxiety in people seeking professional help could be just as high in men as in women. The SAQ established different cut points for men and women, something that is not done in the other assessment measures of social anxiety.

Another question deals with cross-cultural research regarding the type of situations feared by clinical and non-clinical individuals from different countries. Throughout our research on the assessment of social anxiety, we have not found significant differences between clinical and non-clinical individuals in situations like “drinking in public places,” “urinating in a public bathroom,” “writing while being observed,” “working while being observed,” and “blushing in front of others.” This is one of the reasons why they were not included in the final version of the SAQ (Caballo, Salazar, Irurtia, et al., 2010; Caballo, Salazar, et al., 2012). Given that these situations are always present in most of the traditional self-report measures of social anxiety, it could be a cross-cultural difference between Latino American countries, Spain, and Portugal, on one hand, and other countries, particularly those where traditional self-report measures of social anxiety were developed (e. g., USA, UK, Australia), on the other (see also Marques et al., 2011). Future studies could probe more into this potential difference.

We also would like to note that there are some basic dimensions of the SAQ, most notably “Assertive expression of annoyance, disgust or displeasure” and “Interactions with the opposite sex,” that are under-represented in most other self-report measures of social anxiety. For instance, only 1 item of the SIAS and SPS deals with the opposite sex, only one item with speaking in public, and only one item with talking with someone in authority. The remaining items are very general and not situation-specific. The LSAS-SR includes only 1 item dealing with the opposite sex and the SPIN includes none. This is surprising (particularly with this last dimension) if, as seen in the previous studies with the SAQ, “Interactions with the opposite sex” constitutes a basic dimension of the questionnaire. A possible explanation might be that referring to the “opposite sex” is not politically correct, causing researchers to avoid it. Another possible explanation might be that the construct of social anxiety has not been sufficiently explored, although in our studies this dimension appears so obvious when considering the structure of the questionnaire that it is difficult to understand why the factor “Interactions with the opposite sex” does not appear as a key dimension in the literature on the assessment of social anxiety (Caballo et al., 2013). The SAQ includes (and measures) it as a basic dimension of social anxiety. However we would like briefly to discuss about the term “opposite sex.” We know that it is a little misleading and ambiguous because the answer depends on the sexual orientation of the respondent. Therefore, we included items such as “a person I am attracted to” or “someone I find attractive” in the SAQ. For individuals who are attracted to people of the same sex, “preferred sex” instead of “opposite sex” could have been an alternative option, but it was not possible to include it in the questionnaire, which had been empirically developed, and most people do not use expressions of this nature, at least in the countries participating in the development of the SAQ.

In addition to the factorial evidence to which we have referred, we have obtained the convergent evidence of the SAQ-30 (in relation with the LSAS-SR). In this study we aimed to assess the relationship between this new self-report measure of social anxiety and other frequently used measure of social anxiety, the LSAS-SR. One would expect that if the correlation between the two instruments was high, we would have an empirically confirmation of a conceptual relationship between them. The results shows that this relationship was moderate, both in the clinical and non-clinical sample. This moderate level of relationship between the SAQ and LSAS-SR indicate that they do not assess exactly the same aspects of the construct. The coincidences seem greater in the evaluation of anxiety that is experienced in public speaking situations, interactions with people in authority and with strangers (dimensions most represented in the LSAS-SR), but do not coincide much (based on the low correlations) in situations where are required the use of assertiveness to express annoyance, disgust or displeasure, and in those regarding to experiencing criticism and embarrassment and interactions with the opposite sex (dimensions barely represented in the LSAS-SR). Another goal of this study was to examine the reliability of the SAQ-30. The results showed that the internal consistency reliability (Cronbach's alpha) and Guttman reliability coefficients were high for the total scores in both samples and moderate to high for the dimensions of the SAQ-30.

This research also supports the factorial invariance of the SAQ in both the clinical (configural and weak) and non-clinical (four types) samples. Regarding the clinical sample, we think it is sufficient that the subjects of the examined groups employ the same framework to respond to the items of the questionnaire, as demonstrated by the evidence of configural and weak invariance (Cheung & Rensvold, 2002, Little, 1997; Vandenberg & Lance, 2000). The absence of strong invariance (i.e., not only the saturation of the items, but the intercepts are equal in both groups) or strict invariance (i.e., the residual variances of the items are the same in both groups) may be due to the relatively small sample size and should be tested in future studies with larger sample size.

Finally, the cut scores obtained here are very similar to those found in a recent study with a previous version of the questionnaire (Caballo, Salazar, et al., 2012). These scores could be of special interest for the screening of individual with social anxiety, from people who have problems in one kind of situations (one dimension) to people who have problems in most types of situations (five dimensions).

In sum, these findings suggest that this self-report measure can be confidently applied in clinical and research settings. The primary advantages over other commonly used self-report measures of social anxiety is the multidimensional assessment of social anxiety, the comprehensive coverage, the stability of the five-factor structure, and the separate cut scores for men and women in the five dimensions and the total score of the questionnaire. These characteristics distinguish the SAQ from other contemporary measures. Furthermore, some of the empirically derived items of the SAQ are unique to this measure and appear to be critically important for assessing social anxiety across different cultures (Caballo, Salazar, et al., 2012; Caballo, Salazar, Irurtia, et al., 2010). For instance, items dealing with “interactions with the opposite sex” are rarely included in any of the other major measures of social anxiety (e.g., the LSAS-SR and the SIAS include only one item, and the SPIN and the SPS none); only the SPAI includes multiple items, but these items are averaged with items that measure other aspects of social anxiety (Caballo et al., 2013).

Inferences from this study are limited in several ways. First, it is likely that nonclinical samples used in this study included participants who would meet diagnostic criteria for social anxiety. Based on prior epidemiology research (e.g., Spain; Haro et al., 2008; Colombia; Posada-Villa et al., 2008), between .6 > 2.8% of the community-based samples would meet diagnostic criteria, The inclusion of these participants would diminish differences between clinical and non-clinical groups on measures of social anxiety.

Second, additional data are needed on the discriminant validity of the SAQ. Although overlap among self-report measures of distress on multiple dimensions (e.g., depressed mood, other anxiety disorders) have been frequently reported in the literature, the identification of the unique dimensions tapped by the SAQ, and other measures of social anxiety, could increase their clinical utility and predictive validity.

Third, although the SAQ has undergone extensive psychometric evaluation and revision, the construct validity and applicability of the measures derived from it could be further strengthened by: (a) revising several items to render them more gender/sex-neutral, (b) further review and refinement of items in two scales in which coefficients of internal consistency were less than optimal (i.e., Cronbach's alpha < .7), and (c) additional data on the factor structure of the SAQ with a variety of clinical samples.

Fourth, because of the multi-national and community-based nature of the study, and despite our efforts to insure standardization, differences across subsamples in the recruitment and diagnostic strategies were unavoidable. Although this aspect of the study increases the ecological validity of the psychometric evidence it could also mask important associations between subsample characteristics and that evidence.

Finally, as noted by one of the reviewers, we cannot rule out that some participants might have felt obligated, or at least motivated, to participate in the study due to the nature of the recruitment methods and study procedures. However, it is unlikely that this would have systematically biased the results and we have not indication to assume that this was indeed the case. Furthermore, we followed the local ethical guidelines.

In sum, and despite these limitations, the SAQ is a brief, empirically-derived, and psychometrically supported instrument for the comprehensive measurement of social anxiety with clinical and non-clinical populations on situationally-defined dimensions.

Appendix

Below are a series of social situations that may or may not cause you unease, stress or nervousness. Please place an “X” on the number next to each social situation that best reflects your reaction, where “1” represents no unease, stress or nervousness and “5” represents very high or extreme unease stress, or nervousness.

If you have never experienced the situation described, please imagine what your level of unease, stress, or nervousness might be if you were in that situation and rate how you imagine you would feel by placing an “X” on the corresponding number.

Appendix.

Social Anxiety Questionnaire for adults (SAQ)^* (Caballo, Salazar, Arias, and Irurtia, 2010)

	Level of unease, stress or nervousness
Not at all or very slight	Slight	Moderate	High	very high or extremely high
1	2	3	4	5

Please rate all the items and do so honestly; do not worry about your answer because there are no right or wrong ones. Thank you very much for your collaboration.
1. Greeting someone and being ignored	1	2	3	4	5
2. Having to ask a neighbor to stop making noise	1	2	3	4	5
3. Speaking in public	1	2	3	4	5
4. Asking someone attractive of the opposite sex for a date	1	2	3	4	5
5. Complaining to the waiter about my food	1	2	3	4	5
6. Feeling watched by people of the opposite sex	1	2	3	4	5
7. Participating in a meeting with people in authority	1	2	3	4	5
8. Talking to someone who isn't paying attention to what I am saying	1	2	3	4	5
9. Refusing when asked to do something I don't like doing	1	2	3	4	5
10. Making new friends	1	2	3	4	5
11. Telling someone that they have hurt my feelings	1	2	3	4	5
12. Having to speak in class, at work, or in a meeting	1	2	3	4	5
13. Maintaining a conversation with someone I've just met	1	2	3	4	5
14. Expressing my annoyance to someone that is picking on me	1	2	3	4	5
15. Greeting each person at a social meeting when I don't know most of them	1	2	3	4	5
16. Being teased in public	1	2	3	4	5
17. Talking to people I don't know at a party or a meeting	1	2	3	4	5
18. Being asked a question in class by the teacher or by a superior in a meeting	1	2	3	4	5
19. Looking into the eyes of someone I have just met while we are talking	1	2	3	4	5
20. Being asked out by a person I am attracted to	1	2	3	4	5
21. Making a mistake in front of other people	1	2	3	4	5
22. Attending a social event where I know only one person	1	2	3	4	5
23. Starting a conversation with someone of the opposite sex that I like	1	2	3	4	5
24. Being reprimanded about something I have done wrong	1	2	3	4	5
25. While having dinner with colleagues, classmates or workmates, being asked to speak on behalf of the entire group	1	2	3	4	5
26. Telling someone that their behavior bothers me and asking them to stop	1	2	3	4	5
27. Asking someone I find attractive to dance	1	2	3	4	5
28. Being criticized	1	2	3	4	5
29. Talking to a superior or a person in authority	1	2	3	4	5
30. Telling someone I am attracted to that I would like to get to know them better	1	2	3	4	5

^*Reproduced with permission from “Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students: Similarities and differences among degree subjects and regions” by Caballo, Salazar, Arias, et al., 2010, Behavioral Psychology/Psicologia Conductual, 18, pp. 33-34. Copyright 2010 by Fundacion VECA.

Note: The SAQ was known formerly (and published elsewhere) as SAQ-A30. People whose sexual preferences are for the same sex can change the term “opposite sex” for “same sex”

Scoring instructions for the SAQ and its dimensions:

Dimension 1: Interactions with strangers (sum of the items 10, 13, 15, 17, 19, & 22)

Dimension 2: Speaking in public/Talking with people in authority (sum of the items 3, 7, 12, 18, 25, & 29)

Dimension 3: Interactions with the opposite sex (sum of the items 4, 6, 20, 23, 27, & 30)

Dimension 4: Criticism and embarrassment (sum of the items 1, 8, 16, 21, 24, & 28)

Dimension 5: Assertive expression of annoyance, disgust or displeasure (sum of the items 2, 5, 9, 11, 14, & 26)

Total score: Sum of all items of the questionnaire

Cut scores for every dimension and the whole questionnaire are included in table 10.