Syndromes of Self-Reported Psychopathology for Ages 18–59 in 29 Societies

Abstract

This study tested the multi-society generalizability of an eight-syndrome assessment model derived from factor analyses of American adults’ self-ratings of 120 behavioral, emotional, and social problems. The Adult Self-Report (ASR; Achenbach and Rescorla 2003) was completed by 17,152 18–59-year-olds in 29 societies. Confirmatory factor analyses tested the fit of self-ratings in each sample to the eight-syndrome model. The primary model fit index (Root Mean Square Error of Approximation) showed good model fit for all samples, while secondary indices showed acceptable to good fit. Only 5 (0.06%) of the 8,598 estimated parameters were outside the admissible parameter space. Confidence intervals indicated that sampling fluctuations could account for the deviant parameters. Results thus supported the tested model in societies differing widely in social, political, and economic systems, languages, ethnicities, religions, and geographical regions. Although other items, societies, and analytic methods might yield different results, the findings indicate that adults in very diverse societies were willing and able to rate themselves on the same standardized set of 120 problem items. Moreover, their self-ratings fit an eight-syndrome model previously derived from self-ratings by American adults. The support for the statistically derived syndrome model is consistent with previous findings for parent, teacher, and self-ratings of 1½–18-year-olds in many societies. The ASR and its parallel collateral-report instrument, the Adult Behavior Checklist (ABCL), may offer mental health professionals practical tools for the multi-informant assessment of clinical constructs of adult psychopathology that appear to be meaningful across diverse societies.

It has been said that globalization “impacts psychology as a catalyst for developing international knowledge” (Dana and Allen 2008, p. 26). Because an important consequence of globalization is that mental health professionals must increasingly serve people from different societies, it is essential that clinical constructs and the instruments for operationalizing assessment of these constructs be tested in multiple societies. We cannot assume that clinical constructs derived in one society would be automatically generalizable to other societies. Different social groups may sanction or encourage different behaviors, leading to different clusters of behaviors across societies (Weisz et al. 2006). Genetic factors affecting the co-occurrence of behaviors may also vary across societies (Way and Lieberman 2010), and the same may be true for epigenetic factors.

Because most clinical constructs for psychopathology come from a few rather similar societies, their generalizability to other societies must be tested. If clinical constructs are empirically supported for people from particular societies, this would justify assessing individuals in these societies in terms of these constructs. Appropriate norms would also be needed to compare individuals’ scores on clinical constructs with scores for representative samples of peers from their society.

The testing and normative calibration of common clinical constructs of psychopathology across societies is consistent with the etic approach to international research. Stemming from the linguistic terms “phonetic” (representing universal sounds of human speech) and “phonemic” (representing the smallest sound units capable of conveying unique meaning in a particular language), “etic” research focuses on constructs that are common to many societies, whereas “emic” research focuses on constructs specific to particular societies (Berry 1999). Etic approaches thus test the cross-cultural generalizability of psychopathology constructs, while emic approaches pursue culture-specific aspects of psychopathology. Etic and emic approaches are best viewed as complementary and synergistic, overcoming each other’s limitations when used together (Cheung et al. 2011).

To test whether constructs derived from samples of people in one society are generalizable to people from other societies, it is necessary to assess people in the new societies with the procedures that were used in the original society. The generalizability of the constructs can be tested by applying methods such as confirmatory factor analysis (CFA) to data from the new societies (Miller and Sheu 2008). The greater the number of societies in which constructs are tested and the more diverse the societies, the stronger the tests of the constructs’ generalizability.

In the past decade, there has been a proliferation of bicultural studies of psychopathology. The many methodological differences and the comparisons of only two societies per study make it difficult to draw conclusions across these studies. In the following section, we review studies that have tested constructs of adult psychopathology in at least three societies. The three-society criterion was chosen in order to evaluate the generalizability of findings across more than two societies per study. Because so few studies met this selection criterion, we also review cross-cultural studies of personality instruments that included scales for psychopathology constructs such as neuroticism and psychoticism (as reviewed by Eysenck and Barrett 2013 and McCrae and Terraciano 2008). Specifically, we highlight large scale studies of personality instruments in ≥3 societies.

Cross-Cultural Studies of Psychopathology Instruments

Du Paul et al. (2001) asked university students from Italy (N= 197), New Zealand (N=213), and the United States (U.S.; N= 799) to complete the Young Adult Rating Scale (YARS) that was developed for the study. The YARS is a self-report questionnaire assessing 17 symptoms of the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders—Fourth Edition (DSM-IV; 1994) Attention Deficit/Hyperactivity Disorder (ADHD) construct, plus seven “potential difficulties (e.g., problem remembering what was just read) that university students could encounter in association with ADHD symptoms” (p. 372). Exploratory factor analyses (EFA) were used to derive the factor structure of the YARS separately in each sample. For the U.S. and New Zealand, EFA identified inattention and hyperactivity-impulsivity factors. While Italian results also offered some support for these two factors, they were less robust, with 50% of the items not clearly loading on any factor. The authors attributed the Italian findings mostly to the following two cultural factors: First, Italian participants may have had a harder time discriminating among YARS items than students from the U.S. and New Zealand, because they may have been less familiar with the constructs of inattention and hyperactivity. Second, Italian participants may also have been using different reference groups in rating the items, as Italian college students may have higher rates of learning problems than college students in other societies, as suggested by high college acceptance and drop-out rates in Italy. In addition, the relatively small size of the Italian sample (N=197) may have limited the value of the EFA.

Using the screening sample of the Outcome of Depression International Network (ODIN) study, Nuevo et al. (2009) tested the structure and measurement invariance of the 21-item Beck Depression Inventory (BDI; Beck et al. 1961) in samples of 18–64-year-olds from Spain (N=1,245), the United Kingdom (U.K.; N=1,287), Ireland (N=456), Norway (N= 3,007), and Finland (N=1,939). The authors used Item Response Theory (IRT) modeling to test the unidimensionality of BDI ratings. While IRT is not formally classified as a factor analytic technique, it can be conceptualized as a single-factor CFA because it essentially relates item responses to a single latent dimension. The authors also used Multiple Indicator Multiple Cause (MIMIC) modeling, a structural equation modeling technique, to test the influence of society on item parameters (i.e., item thresholds and loadings). IRT modeling supported the unidimensionality of the BDI in each society. However, item parameters produced by IRT and MIMIC models indicated that certain items performed differently across societies, and that these differences were not explained by differences in mean levels of depression. In other words, IRT and MIMIC results supported structural invariance but did not support invariant item functioning across societies, suggesting that culture-level influences (e.g., item meaning and translation differences) affected item performance across societies.

Cross-Cultural Studies of Personality Instruments

Paunonen et al. (1996) tested the factor structure of the 136-item Nonverbal Personality Questionnaire (NPQ; Paunonen et al. 1992) in data from Canada, Finland, Poland, Germany, Russia, and Hong Kong. The NPQ is a pictorial self-report questionnaire that was developed as a nonverbal measure of the big five personality traits of Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness to Experience (McCrae and John 1992). The NPQ was administered to samples of university students in each society, ranging from 90 in Germany to 100 in Poland and Hong Kong. Although the sample sizes would be considered too small for the EFAs that were performed on 136 items, Paunonen et al. (1996) interpreted the results as supporting the five-factor structure in each society.

With notable cross-cultural breadth, Barrett et al. (1998) tested the generalizability of the 90-item Eysenck Personality Questionnaire (EPQ; Eysenck and Eysenck 1975) in 34 diverse societies using large general population samples of 10–89-year-olds (Ns ranged from 654 for Australia to 2,378 for Portugal). The EPQ is a self-report questionnaire measuring Psychoticism, Extraversion, and Neuroticism, plus Social Desirability response tendencies. Congruence coefficients were used to test the similarity of factor structures across societies by gender. Results indicated an impressive degree of factor congruence across the 34 societies.

Also with impressive breadth, McCrae (2001) tested the generalizability of the 240-item Revised NEO Personality Inventory (NEO-PI-R; Costa and McRae 1992) across 26 societies. The NEO-PI-R is a self-report questionnaire assessing the big five personality dimensions. Data for the study came from 26 previously published studies of adults who were ≥18 years old. These 26 studies had tested the psychometric properties of the instrument in each of the 26 societies (Ns ranging from 122 for Japan to 3,730 for Germany). Samples were stratified by age (i.e., 18–21 vs. older) and gender into 84 subsamples. Raw item data were aggregated into 30 summary scores, which were then subjected to principal components analysis with varimax and procrustes rotations, using the 84 subsamples as cases. Congruence coefficients between factor loadings obtained from this “intercultural factor analysis” (p. 820) and the original NEO-PI-R factor structure obtained from item-level analyses supported the five-factor model. Using the same data analytic procedures, McCrae (2002) replicated the findings for 10 additional samples.

A somewhat different approach was taken by Schmitt et al. (2007), who tested the factor structure of the Big Five Inventory (BFI; Benet-Martínez and John 1998) in data provided by 17,837 18–95-year-old adults from 56 societies. The BFI is a 44-item questionnaire that was designed for efficient assessment of the big five personality dimensions. Raw data provided by all participants were aggregated into a single data set and subjected to principal axis factoring with varimax rotation. The derived factor structure was very similar to the U.S. factor structure. It was then procrustes rotated to the U.S. factor structure, yielding high item and total congruence coefficients. The factor structure derived from the aggregated, multi-society data set was thus found to be similar to the U.S. structure.

Several studies have thus tested psychopathology and personality dimensions derived from adults’ ratings of their own emotional, behavioral, and social problems and personality characteristics in three or more societies. Methodological differences among the studies (i.e., different assessment instruments, sampling procedures, analyses, domains of assessed functioning) make it difficult to integrate their conclusions. However, their results support the viability of factor analytic and related methodologies for testing the generalizability of constructs for assessing adult emotional and behavioral problems and personality across societies.

The Adult Self-Report (ASR)

The ASR is a self-report questionnaire for ages 18–59 that assesses behavioral, emotional, and social problems, plus adaptive functioning, personal strengths, and substance use (Achenbach and Rescorla 2003). It can be completed in 15–20 min on paper, online, or in interviews. The ASR and its predecessor, the Young Adult Self-Report (YASR; Achenbach 1997), have been used in over 100 published studies with foci such as prospective follow-ups (van der Ende et al. 2012); treatment outcomes (Saavedra et al. 2010); molecular genetics (Boomsma et al. 2008); quantitative genetics (Forsman et al. 2010); and special populations (Buysse et al. 2010).

Several studies have tested prediction of scores on the ASR syndrome constructs from pre-adult to adult developmental periods. As an example, Reef et al. (2009) computed predictive odds ratios (ORs) from syndrome scores on Child Behavior Checklists (CBCLs) completed by parents of 1,365 Dutch 4–16-year-olds to scores on ASRs completed by the participants themselves 24 years later, when they were 28 to 40 years old. Despite the differences between instruments (CBCL vs. ASR) and raters (parents vs. self), plus the 24-year interval, the ORs showed significant homotypic prediction from CBCL syndromes to the corresponding ASR Anxious/Depressed, Withdrawn, Somatic Complaints, Aggressive Behavior, and Rule-Breaking Behavior syndromes.

Supporting their utility in different societies, ASR and YASR studies of clinical and nonclinical populations have been done in Australia (Hayatbakhsh et al. 2007); Finland (Haavisto et al. 2005); Germany (Retz et al. 2004); the Netherlands (Reef et al. 2009); Norway (Halvorsen et al. 2005); Poland (Zasepa and Wolanczyk 2011); Sweden (Forsman et al. 2010); and Switzerland (Steinhausen and Winkler Metzke 2004). Examples of findings include child to adult continuities of psychopathology (Forsman et al. 2010; Hayatbakhsh et al. 2007; Reef et al. 2009; Steinhausen and Winkler Metzke 2004), child risk factors for adult suicidal ideation and behavior (Haavisto et al. 2005), and emotional and behavioral characterization of general and special populations (Halvorsen et al. 2005; Retz et al. 2004; Zasepa and Wolanczyk 2011).

Purpose of this Study

The non-ASR studies reviewed earlier used instruments containing from 17 to 240 items to assess dimensions of psychopathology or personality in multiple societies. The psychopathology instruments were designed to assess either a single a priori dimension of depression (on the BDI) or two a priori dimensions of ADHD (on the YARS). The personality instruments were designed to assess either three dimensions (on the EPQ) or five dimensions (on the NPQ, NEO-PI-R, and BFI) that had been derived from empirical analyses of associations among self-ratings of personality and psychopathology items.

The purpose of the present study was to test the multi-society generalizability of the eight-syndrome model of the ASR. Like the studies reviewed above, the present study tested the degree to which syndromes of items based on self-ratings in one society would be supported by self-ratings in other societies. Like the studies of personality instruments, the present study tested syndromes of items that had been statistically derived. However, the present study used CFAs to test an eight-syndrome model derived from 120 items, 99 of which loaded significantly on the syndromes. Moreover, the present study used samples from more societies (29) than did the previous studies of psychopathology instruments, although two studies of personality instruments included more societies (Barrett et al. 1998; Schmitt et al. 2007). CFAs of self-ratings by adolescents in 33 societies have supported a syndrome model derived factor-analytically from the Youth Self-Report (YSR), which includes adolescent versions of many ASR items (Ivanova et al. 2007c; Rescorla et al. 2012). Consequently, we hypothesized that the ASR syndrome model would be supported by our CFAs of self-ratings by adults in multiple societies.

Method

Samples

Indigenous researchers arranged to have ASRs completed by 17,152 18–59-year-olds from the 29 societies listed in Table 1. Samples averaged 42% male, and Ns ranged from 293 (Egypt) to 1,548 (Flanders). Table 1 describes the samples, including the mean age of participants and sampling procedures.

Table 1.

Sample information

Society	Reference	N	Age range	Mean age (SD)c	% Male	Sample
Albania	Sokoli (2012)b	750	18–59	37.2 (12.8)	50	Nationally representative.
Angola	Caldas (2012a)b	399	18–59	18–25: 43%	63	Community sample.
				26–39: 34%
				40–49: 12%
				50–59: 11%
Argentina	Samaniego and Vázquez (2012)	679	18–59	35.7 (12.0)	48	Community sample stratified by level of educational attainment to be representative of the greater Buenos Aires area.
Belgium (Flanders)	Decoster and Fontaine (2012)b	1,548	18–59	38.6 (12.2)	50	Community sample stratified by region, gender, age, and educational attainment to be representative of Flanders.
Brazil	Silvares (2012)b	813	18–59	34.5 (11.8)	41	Community sample stratified by region, age, gender, and socioeconomic status to be representative of the national population.
China	Liu (2012)b	578	18–59	33.3 (9.5)	38	Community sample drawn from regions of mainland China.
Czech Republic	Csemy (2012)b	588	18–59	37.8 (12.4)	51	Community sample stratified by region, age, gender, and educational attainment to be representative of the national population.
Egypt	Riad (2012)b	293	18–59	25.7 (8.2)	29	Community sample.
France	Mahr et al. (2013); Leynet et al. (2013)	1,238	18–59	24.5 (7.4)	29	University students.
Hong Kong	Au and Leung (2012)b	324	18–59	29.4 (12.7)	39	Community sample stratified by age and gender to be representative of the Hong Kong population.
Iceland	Guðmundsson and Árnadóttir (2012)b	353	18–59	37.5 (12.0)	44	Representative national sample randomly drawn from the national register.
Italy	Bellina (2012)b	504	18–59	38.1 (12.4)	46	Representative sample of the Lecco province randomly drawn from the electoral roll.
Japana	Funabiki (2012)b	1,000	18–59	38.2 (10.7)	47	Community sample recruited by a research company.
Kenya	Harder and Ndetei (2013)b	427	18–59	38.9 (8.5)	40	Regional sample of parents of school-aged children, with children’s names randomly drawn from class rosters.
Koreaa (South)	Kim et al. (2009)	1,000	18–59	37.9 (9.8)	51	Representative national sample, randomly drawn from the national registry, with stratification by age, gender, and educational attainment.
Kosovo	Shahini and Ahmeti-Pronaj (2012)b	571	18–59	30.6 (10.5)	40	Community sample.
Latvia	Sebre (2012)b	302	18–59	33.9 (12.7)	43	Community sample stratified by age, gender, educational attainment, and region to be representative of the national population.
Lithuania	Šimulionienė et al. (2010)	573	18–59	35.3 (11.1)	48	Representative national sample randomly drawn from the national registry, with stratification by gender, age, and educational attainment.
Mexico	Leiner and Avila Maese (2013)b	308	18–59	27.3 (9.8)	59	Community sample.
Pakistan	Mahr (2012)b	654	18–37	21.5 (2.8)	26	English-speaking university students in the Lahore area.
Poland	Zasepa (2012)b	310	18–59	36.7 (11.9)	37	Community sample stratified by age, gender, residence, and educational attainment to be representative of the national population.
Portugal	Caldas (2012b)b	397	18–59	35.4 (12.0)	49	Community sample stratified by age and gender to be representative of the national population.
Romania	Dobrean (2011)b	638	20–56	24.2 (6.1)	15	University students.
Russia	Malykh (2012)b	429	18–55	20.6 (4.3)	33	University students.
Serbia	Markovic (2012)b	314	18–59	35.7 (10.6)	42	Representative sample of the Novi Sad metropolitan area randomly drawn from the population registry, with stratification by age.
Spain	Ezpeleta et al. (2014)	1,136	18–58	37.6 (5.3)	48	Community sample of parents of preschoolers in the greater Barcelona metropolitan area randomly drawn from the registry of parents of preschoolers.
Taiwan	Chen (2012)b	300	18–59	37.0 (11.9)	50	Community sample stratified by region, gender, and age to be representative of the national population.
Turkey	Sakarya (2012)b	383	18–58	25.6 (8.2)	24	Community sample.
UK	Talcott, Nakubulwa, and Virk, (2012)b	343	18–59	34.0 (12.5)	35	Community sample.

^aThe identical sample sizes for Japan and Korea are coincidental, not errors

^bUnpublished data

^cOnly age ranges were available for Angola

Instrument and Tested Model

The ASR’s 120 problem items are rated 0=not true, 1= somewhat or sometimes true, or 2=very true or often true, based on the preceding 6 months. The problem item ratings discriminate significantly between adults referred for mental health or substance use services versus demographically similar nonreferred adults (Achenbach and Rescorla 2003).

The eight ASR syndromes were modeled as first-order correlated factors, with no hierarchical relations between factors assumed. Each of the 99 items was assigned to only one latent factor. For Japan, items assessing illegal behavior (6. I use drugs (other than alcohol and nicotine) for nonmedical purposes; 57. I physically attack people; 82. I steal; and 92. I do things that may cause me trouble with the law) were omitted from the ASR because their endorsement by study participants would have legally obligated the investigators to report them to authorities. Because item 37. I get in many fights was not endorsed by any participant in Taiwan, it was omitted for Taiwan.

Data Analyses

Because our goal was to test the fit of the U.S. factor model in other societies, we followed the factor analytic procedures reported by Achenbach and Rescorla (2003). We transformed item ratings to 0 versus 1 or 2, and computed tetrachoric correlations on these bivariate ratings. Following Achenbach and Rescorla’s procedures, ASRs missing ratings of ≥9 problem items were excluded from analyses (1.1% of all cases). Missing data were modeled as Missing at Random (MAR) with the Mplus default Full Information Maximum Likelihood (FIML) method. We used the robust WLSMV estimator (Muthén and Muthén 1998–2012) to account for the nonnormal distribution of the data. The model was tested separately for each society.

The Root Mean Square Error of Approximation (RMSEA) was selected as the primary index of model fit because it was identified as the best performing model fit index for the WLSMV (Yu and Muthén 2002).In a Monte Carlo simulation study, Yu and Muthén (2002) found that RMSEA values of less than .05–.06 reliably indicated good model fit for ordered categorical variables. We also computed the Comparative Fit Index (CFI) and Tucker Lewis Index (TLI), but considered their results to be secondary to the RMSEA. Hu and Bentler (1999) suggested that CFI and TLI values greater than .95 should be used to indicate good fit. However, Marsh et al. (2004) argued that this threshold was too stringent for complex factor models in applied research. Because our model was much more complex than the model comprising three five-item factors that Hu and Bentler tested, we used less stringent criteria of .80 to .90 to indicate acceptable model fit, and ≥.90 to indicate good model fit.

Results

The correlated eight-syndrome model converged for all 29 samples. As Table 2 shows, RMSEAs ranged from .018 (China) to .034 (Pakistan), indicating good model fit for all 29 societies. The RMSEA equaled .02, .023, and .026 at the 25th, 50th, and 75th percentiles, respectively. CFI and TLI values indicated acceptable to good model fit for all societies, and their values were similar within societies. CFIs ranged from .812 for Angola to .952 for Japan. TLI values ranged from .807 for Angola to .950 for Japan and Kenya.

Table 2.

CFA results

Society	RMSEA	CFI	TLI	Items with nonsignificant loadingsa	Empirically underidentified itemsa,b	Factor loadings		Factor correlations
						Median loading	Range	Median correlation	Range
1. Albania	.026	.914	.911			.69	.15–.91	.67	.22–.91
2. Angola	.027	.812	.807			.55	.22–.80	.78	.62–.98
3. Argentina	.024	.866	.862	22	40	.60	.16–1.05b	.59	.10–.75
4. Belgium (Flanders)	.027	.895	.892			.65	.25–.84	.60	.18–.78
5. Brazil	.023	.901	.898			.61	.18–.81	.65	.14–.80
6. China	.018	.937	.935			.66	.33–.84	.74	.51–.90
7. Czech Republic	.022	.905	.902			.64	.37–.84	.62	.13–.86
8. Egypt	.020	.918	.916	26, 69	18	.62	−.05–1.04b	.65	.21–.85
9. France	.028	.856	.852			.60	.26–.87	.56	.01–.72
10. Hong Kong	.020	.945	.944			.70	.41–.93	.70	.28–.88
11. Iceland	.019	.936	.934			.70	.32–.97	.66	.17–.88
12. Italy	.019	.912	.910			.62	.18–.86	.60	.05–.78
13. Japan	.024	.952	.950			.73	.40–.92	.78	.44–.89
14. Kenya	.020	.951	.950			.65	.40–.87	.85	.52–.94
15. Korea (South)	.024	.942	.940			.66	.28–.90	.74	.33–.90
16. Kosovo	.020	.927	.925			.62	.22–.82	.75	.48–.88
17. Latvia	.025	.853	.849		18	.59	.22–1.09b	.55	.05–.75
18. Lithuania	.025	.902	.899	17		.64	.07–.89	.60	.33–.83
19. Mexico	.025	.865	.861	22		.61	−.09–.87	.64	.24–.84
20. Pakistan	.034	.831	.826			.63	.25–.96	.72	.41–.88
21. Poland	.024	.882	.879	56e	54	.64	.12–1.01b	.61	.12–.85
22. Portugal	.026	.822	.817	7, 22, 26, 122		.60	−.04–.93b	.65	.27–.83
23. Romania	.023	.917	.914		18	.60	.22–1.06b	.61	.30–.83
24. Russia	.027	.881	.878	22, 26		.60	.04–.85	.63	−.01–.80
25. Serbia	.021	.925	.923			.68	.36–.95	.66	.32–.89
26. Spain	.019	.906	.904	22, 82		.63	.09–.89	.63	.26–.84
27. Taiwan	.020	.942	.941	26, 40, 70, 90		.65	.09–.96	.69	.30–.92
28. Turkey	.022	.925	.923			.64	.33–.88	.68	.37–.88
29. UK	.022	.871	.867	70		.63	.25–.90	.60	.20–.78

RMSEA Root Mean Square Error of Approximation, CFI Comparative Fit Index, TLI Tucker-Lewis Index

^aThe number is the item’s number on the ASR

^bThe 95% confidence intervals around out-of-range factor loadings included values that were in the admissible parameter space (0.00–1.00)

As Table 2 documents, all 99 items had statistically significant loadings on their respective factors for 19 societies. For Argentina, Lithuania, Mexico, Poland, and the UK, one item had a nonsignificant loading. For Egypt, Russia, and Spain, two items had nonsignificant loadings. Four items had nonsignificant loadings for Taiwan and Portugal. Only 19 (0.7%) of the 2,866 tested item loadings were thus nonsignificant. Of the 19 nonsignificant loadings, five were for item 22. I worry about my future, four for item 26. I don’t feel guilty after doing something I shouldn’t, and two for item 70. I see things that other people think aren’t there.

The medians and ranges of factor loadings for each society are presented in Table 2. The median factor loading ranged from .55 (Angola) to .73 (Japan), with an overall median of .63. This indicates that for each society, the tested items demonstrate robust loadings on their predicted factors. Table 2 also presents medians and ranges for correlations between latent factors across the societies. Median correlations between latent factors ranged from .55 in the Latvian sample to .85 in the Kenyan sample, with an overall median of .65. These correlations indicate that, on average, latent factors were related to each other (reflecting the general factor of psychopathology), but not redundant with each other.

Five societies (Argentina, Egypt, Latvia, Poland, and Romania) each had one negative residual item variance (item 40. I hear sounds or voices that other people think aren’t there for Argentina; item 18. I deliberately try to hurt or kill myself for Egypt, Latvia, and Romania; and item 54. I feel tired without good reason for Poland). Thus, only 5 (0.06%) of the 8,598 estimated parameters were outside the admissible parameter space. The estimated parameters included 99 thresholds, item loadings, and residual variances for 27 societies, plus 98 thresholds, item loadings, and residual variances for Taiwan, plus 95 thresholds, item loadings, and residual variances for Japan. We tested the five aberrant parameters by forming 95% confidence intervals around them and determining whether these confidence intervals included the admissible parameter space (Chen et al. 2001). Because the confidence intervals for all out-of-range parameters included the admissible parameter space, sampling fluctuations may explain the five aberrant parameters.

Table 3 presents the means, medians, standard deviations and ranges of the loadings for each item and for the items comprising each syndrome across the 29 societies. Across all syndromes, the median loadings of individual items ranged from .37 (item 26. I don’t feel guilty after doing something I shouldn’t) to .81 (item 54. I feel tired without a good reason), with an overall median of .64. Within syndromes, the median item loadings ranged from .59 for Attention Problems to .70 for Anxious/ Depressed. This indicates that the tested items demonstrate robust loadings on their predicted factors across societies.

Table 3.

Descriptive statistics for factor loadings across 29 societies by syndrome

Syndromes and itemsa	Mean Loading	SD	Median Loading	Range of median Loadings
Anxious/Depressed	(.67)	(.08)	(.70)	(.45–.80)
12. Lonely	.65	.08	.68	.44–.77
13. Confused	.74	.06	.75	.62–.82
14. Cries a lot	.57	.06	.57	.38–.68
22. Worries about future	.43	.21	.45	−.09–.74
31. Fears doing bad	.58	.12	.60	.23–.76
33. Feels unloved	.73	.05	.73	.61–.82
34. Others out to get him/her	.63	.08	.62	.48–.80
35. Feels worthless	.73	.07	.73	.56–.87
45. Nervous, tense	.70	.09	.73	.48–.83
47. Lacks self-confidence	.70	.07	.70	.51–.82
50. Fearful, anxious	.71	.07	.70	.49–.83
52. Feels too guilty	.69	.06	.70	.56–.81
71. Self-conscious	.58	.10	.59	.35–.81
91. Suicidal thoughts	.71	.12	.71	.33–.91
103. Unhappy, sad	.79	.05	.80	.65–.89
107. Can’t succeed	.68	.07	.69	.55–.82
112. Worries	.65	.13	.66	.37–.87
113. Worries about relations with opp. sex	.57	.11	.58	.26–.77
Withdrawn	(.64)	(.08)	(.67)	(.47–.72)
25. Doesn’t get along	.68	.08	.68	.48–.83
30. Poor relations with opp. sex	.58	.11	.62	.37–.77
42. Rather be alone	.57	.06	.58	.43–.69
48. Not liked	.72	.10	.72	.49–.89
60. Enjoys little	.71	.07	.70	.56–.87
65. Refuses to talk	.66	.08	.67	.41–.82
67. Trouble making friends	.69	.07	.70	.53–.87
69. Secretive	.46	.15	.47	−.05–.66
111. Withdrawn	.63	.12	.64	.39–.84
Somatic complaints	(.64)	(.10)	(.64)	(.49–.81)
51. Feels dizzy	.71	.11	.73	.35–.91
54. Tired without reason	.81	.09	.81	.66–1.01b
56a. Aches, pains	.60	.10	.62	.34–.78
56b. Headaches	.54	.09	.55	.32–.71
56c. Nausea, feels sick	.74	.09	.74	.51–.87
56d. Eye problems	.48	.13	.49	.23–.75
56e. Skin problems	.48	.11	.51	.12–.64
56f. Stomachaches	.60	.09	.62	.30–.77
56g. Vomiting	.68	.10	.70	.40–.85
56h. Heart pounding	.66	.09	.66	.38–.79
56i. Numbness	.67	.11	.68	.45–.89
100. Trouble sleeping	.55	.09	.57	.32–.77
Thought problems	(.60)	(.09)	(.62)	(.41–.72)
9. Can’t get mind off thoughts	.61	.11	.65	.35–.79
18. Harms self	.75	.16	.72	.43–1.09b
36. Accident-prone	.57	.10	.57	.36–.74
40. Hears sounds, voices	.64	.19	.65	.20–1.05b
46. Twitching	.63	.07	.65	.50–.75
63. Prefers older people	.43	.12	.41	.15–.69
66. Repeats acts	.57	.12	.57	.36–.76
70. Sees things	.54	.16	.52	.25–.82
84. Strange behavior	.62	.13	.60	.36–.96
85. Strange ideas	.62	.12	.64	.34–.90
Attention problems	(.59)	(.09)	(.59)	(.43–.71)
1. Forgetful	.46	.08	.48	.28–.60
8. Can’t concentrate	.61	.06	.61	.50–.71
11. Too dependent	.57	.05	.58	.48–.65
17. Daydreams	.48	.15	.51	.07–.67
53. Trouble planning	.65	.08	.65	.43–.80
59. Fails to finish	.68	.08	.69	.51–.84
61. Poor work performance	.69	.08	.67	.54–.86
64. Trouble setting priorities	.67	.07	.67	.52–.82
78. Trouble making decisions	.71	.07	.71	.56–.82
101. Skips job	.54	.11	.55	.36–.69
102. Lacks energy	.67	.09	.68	.46–.89
105. Disorganized	.61	.10	.59	.39–.76
108. Loses things	.56	.10	.55	.36–.73
119. Not good at details	.50	.12	.50	.24–.70
121. Late for appointments	.42	.10	.43	.28–.62
Aggressive behavior	(.64)	(.09)	(.63)	(.48–.79)
3. Argues	.46	.13	.48	.20–.71
5. Blames others	.53	.11	.54	.22–.71
16. Mean to others	.54	.11	.54	.31–.75
28. Gets along badly with family	.56	.10	.56	.38–.75
37. Gets in fights	.59	.14	.60	.34–.92
55. Mood swings	.77	.10	.78	.54–.96
57. Attacks people	.63	.17	.65	.25–.95
68. Screams a lot	.59	.09	.59	.44–.72
81. Changeable behavior	.72	.11	.74	.33–.89
86. Stubborn, sullen, irritable	.66	.11	.67	.31–.84
87. Mood changes	.76	.08	.78	.58–.84
95. Hot temper	.65	.08	.66	.43–.77
97. Threatens people	.62	.12	.60	.37–.87
116. Easily upset	.74	.08	.73	.62–.90
118. Impatient	.64	.07	.63	.53–.82
Rule–breaking behavior	(.60)	(.11)	(.61)	(.37–.77)
6. Uses drugs	.47	.11	.46	.24–.69
20. Damages own things	.69	.11	.72	.33–.83
23. Breaks rules	.60	.09	.61	.43–.77
26. Lacks guilt	.34	.16	.37	.02–.64
39. Bad friends	.57	.11	.58	.32–.77
41. Impulsive	.68	.08	.69	.50–.81
43. Lying, cheating	.66	.09	.67	.40–.82
76. Irresponsible	.75	.09	.77	.52–.90
82. Steals	.64	.18	.65	.16–.93
90. Gets drunk	.52	.16	.48	.24–.96
92. Trouble with the law	.60	.16	.59	.31–.83
114. Fails to pay debts	.59	.10	.60	.26–.75
117. Trouble managing money	.62	.07	.62	.48–.76
122. Trouble keeping jobs	.59	.17	.61	−.04–.83
Intrusive	(.65)	(.06)	(.65)	(.57–.74)
7. Brags	.53	.14	.57	.20–.77
19. Demands attention	.62	.11	.62	.38–.78
74. Showing off, clowning	.66	.10	.67	.30–.77
93. Talks too much	.61	.13	.62	.37–.83
94. Teases a lot	.71	.14	.74	.39–.90
104. Loud	.70	.10	.71	.51–.89

Values in parentheses and italics are descriptive statistics for syndromes

^aThe number is the item’s number on the ASR

^bThe 95% confidence intervals around out-of-range factor loadings included values that were in the admissible parameter space (0.00–1.00)

Discussion

This study tested the generalizability of the eight-syndrome ASR model for assessing adult psychopathology in 29 societies. The data came from societies differing widely in social, political, and economic systems, languages, ethnicities, religions, and geographical regions.

In all samples, the eight-syndrome model converged, RMSEAs indicated good model fit, and secondary indices (CFI and TLI) indicated acceptable to good fit. Of the 8,598 tested parameters, only 5 (0.06%) fell outside the admissible parameter space, indicating either negligible model misspecification or sampling fluctuations. Item loadings were robust across societies, with the median item loading being .63. The results thus supported the eight-syndrome model in all samples.

Our findings are consistent with findings for adolescents’ self-ratings on the YSR, for which an eight-syndrome model has been supported by CFAs of data from 33 societies (Ivanova et al. 2007c; Rescorla et al. 2012). Our findings are also consistent with CFA findings for parent ratings on the Child Behavior Checklist for Ages 6–18 in 41 societies and the Child Behavior Checklist for Ages 1½–5 in 23 societies, as well as for teacher ratings on the Teacher’s Report Form for Ages 6–18 in 27 societies and the Caregiver-Teacher Report Form for Ages 1½–5 in 14 societies (Ivanova et al. 2007a, 2007b, 2010, 2011; Rescorla et al. 2012). Taken together, our findings indicate that syndrome models of both child and adult psychopathology derived empirically on large normative samples can demonstrate remarkable generalizability across diverse societies.

The consistency of our findings for adults with previous CFA findings for children may seem surprising. One might hypothesize that a syndrome model derived from adults’ self-ratings in one society would not be supported by self-ratings in so many very different societies, because syndromes might be shaped more by adults’ longer exposure to society-specific influences than would be true for children. However, the great varieties of both genetic and environmental influences potentially affecting self-rated problems in each society may overlap sufficiently with those in other societies to converge on the syndrome structure that was supported by our CFAs. The CFA support for the eight-syndrome model indicates considerable commonality among diverse societies with respect to basic patterns of self-rated problems.

Limitations of the Study

The results should be interpreted in the framework of CFA methodology, which tests a single a-priori specified syndrome model. Tests of other syndrome models and use of other analytic methods might yield different results. Because the ASR does not include all the behavioral, emotional, and social problems that may be clinically relevant in every society, assessment of additional problems might reveal additional syndromes in some or all the participating societies.

Another limitation of our findings is that, because all ASR problem items are scored in one direction, we were unable to control for acquiescence response bias, as has been done in a test of personality constructs across societies (Schmitt et al. 2007). By reducing item variance, acquiescence response biases can reduce the power of CFA to establish a factor structure. Because acquiescence bias covaries with cultural characteristics, such as collectivism and uncertainty avoidance (Smith 2004), it can challenge the interpretability of cross-cultural CFA findings. Although the ASR 0–1–2 ratings may be less vulnerable to acquiescence bias than ratings of agreement versus disagreement, any remaining acquiescence bias or other response biases (e.g., negative or moderate response biases) did not prevent the ASR syndrome model from being supported in all samples.

Some might consider the present study’s etic methodology, namely use of the same standardized assessment instrument in all societies, to be another limitation. However, etic and emic methodologies can be viewed as complementary, rather than opposing approaches. Emic methodology employing instruments tailored to each society can be used for follow-up studies to identify reasons for differences that etic methods find between societies. Emic methods might also illuminate differences between societies in how particular items are interpreted, and may suggest additional items for assessment. Alternatively, etic methodology might follow emic methodology, as in testing the cross-society generalizability of items or clinical constructs derived within a single society.

Another limitation is that data from additional informants might yield different results (De Los Reyes 2011). To examine this possibility, we tested the generalizability of the eight-syndrome model in ratings of many of our study’s participants on the Adult Behavior Checklist, a collateral-report instrument paralleling the ASR (Ivanova et al. 2014). The findings supported the generalizability of the tested syndrome model to collateral ratings.

Implications of the Findings

Our findings that 17,152 adults in 29 societies were willing and able to rate themselves on the same standard set of problem items and that their ratings fit the eight-syndrome model support the generalizability of a “bottom-up” approach to assessment of psychopathology in terms of statistical aggregations of self-rated problems into syndrome constructs. This approach differs from the more “top-down” approach whereby experts construct diagnostic categories and then construct interviews for operationalizing assessment of the diagnostic categories. The bottom-up and top-down approaches are not necessarily incompatible, as experts’ judgments can be used to identify items for scoring bottom-up assessment instruments in terms of top-down diagnostic constructs (Achenbach et al. 2005). Conversely, responses to diagnostic interviews can be statistically analyzed to identify syndromes of problems that may be detectible in interviewees’ responses.

Although diagnostic interviews have been administered to adults in multiple societies to compare prevalence estimates for DSM-IV diagnoses (e.g., World Health Organization 2004), the generalizability of the diagnostic constructs assessed by the interviews has not been tested in similarly analyzed samples from multiple societies. Consequently, it is to be hoped that DSM-5 diagnostic constructs will be subjected to such tests. For example, standardized instruments for assessing symptoms that define the diagnostic constructs could be administered to large samples of individuals in multiple societies. The data from these societies can then be tested to determine whether the diagnostic constructs are supported.

After clinical constructs have been supported by data from multiple societies, scores on the constructs should be compared between those societies to determine whether different norms are required to evaluate individuals assessed in the different societies. Krueger et al. (2003) have illustrated cross-cultural comparisons of scores on factor-analytically derived syndromes. Although Krueger et al. did not report statistical tests of societal differences in syndrome scores, Rescorla et al. (2014) do report statistical tests of societal differences in ASR syndrome scores as a basis for constructing appropriate norms.

Results of the present study support clinical constructs of adult psychopathology for use in societies that differ in many ways. These constructs can be used to advance services, research, and training, as well as to facilitate international collaboration. Equally important, having been translated into dozens of languages, the ASR and ABCL offer clinicians working with adults of different backgrounds practical tools for assessment of a common core of clinical constructs from multi-informant perspectives.