Reliability and Validity of the Korean Young Schema Questionnaire-Short Form-3 in Medical Students

Abstract

Objective

The Young Schema Questionnaire (YSQ) is a self-report measure of early maladaptive schemas and is currently in its third revision; it is available in both long (YSQ-L3) and short (YSQ-S3) forms. The goal of this study was to develop a Korean version of the YSQ-S3 and establish its psychometric properties in a Korean sample.

Methods

A total of 542 graduate medical students completed the Korean version of the YSQ-S3 and several other psychological scales. A subsample of 308 subjects completed the Korean YSQ-S3 both before and after a 2-year test-retest interval. Correlation, regression, and confirmatory factor analyses were performed on the data.

Results

The internal consistency of the 90-item Korean YSQ-S3 was 0.97 and that of each schema was acceptable, with Cronbach's alphas ranging from 0.59 to 0.90. The test-retest reliability ranged from 0.46 to 0.65. Every schema showed robust positive correlations with most psychological measures. The confirmatory factor analysis for the 18-factor structure originally proposed by Young, Klosko, and Weishaar (2003) showed that most goodness-of-fit statistics were indicative of a satisfactory fit.

Conclusion

These findings support the reliability and validity of the Korean version of the YSQ-S3.

INTRODUCTION

Schema theory and therapy, developed by Jeffrey Young,¹ is an integrative model of psychotherapy for patients with severe, chronic psychological problems who have been considered difficult to treat. Central to the schema model is the notion of early maladaptive schemas (EMSs), defined as broad, pervasive emotional and cognitive patterns that begin early in development and repeat throughout life.² In the meanwhile, schema-related coping styles, as a response to the EMS, develop and often end up rather reinforcing and perpetuating the EMS.¹,² The goal of schema therapy is to help patients to understand the origin of their EMS, to identify and stop using maladaptive coping styles, and to build healthy schemas typically using limited parenting, imagery, and schema flashcard or diaries.¹,² There is a growing literature of outcome studies for patients with borderline personality disorder (BPD) and other chronic Axis I disorders.³,⁴,⁵,⁶ In a comprehensive review on its effectiveness for BPD patients, schema therapy showed beneficial effects on many facets of BPD psychopathology in both individual and group settings and compared to treatment as usual and other specialized BPD treatment.⁶

To date, Young et al.² have identified 18 different EMSs, each with its own proposed origin and long-term impact. The 18 EMSs are grouped into five more general categories, known as schema domains, which bring together the EMSs that tend to develop together. The EMSs are assessed using the self-report Young Schema Questionnaire (YSQ).⁷,⁸ Each item in the questionnaire is a statement based on a maladaptive belief as defined by schema theory. Respondents are asked to rate the degree to which they agree with the statements on a six-point Likert scale (1-6). A mean score is calculated for each EMS, with higher scores representing a stronger endorsement of the EMS in question. Since the original long from of the YSQ was introduced in 1990,⁷,⁸ the YSQ has evolved as schema theory has continued to develop, and it is currently in its third revision, which is available in both long (YSQ-L3)⁹ and short (YSQ-S3)¹⁰ forms. Previous psychometric studies reported that the instrument had high internal consistency¹¹,¹²,¹³ and adequate test-retest reliability.¹³,¹⁴ Studies that tested the concurrent validity of the instrument demonstrated significant correlations with psychological symptoms,¹⁵ cognitive structures,¹⁶ and attachment styles.¹⁷ Moreover, discriminant analysis also supported construct validity of the questionnaire.¹³,¹⁴,¹⁸ In sum, both the long and short version of the YSQ-S3 have demonstrated strong internal consistency, reliability, and validity and can be used with generally accurate results in both clinical and non-clinical setting.¹⁹

In terms of Korean translation, Cho initially developed the Korean version of the 205-item YSQ-L1 among 833 college students.²⁰ Sixteen factors were extracted from factor analysis, and 15 of these corresponded exactly with 15 of the 16 EMSs originally proposed by Young et al.⁷ Consistent with previous studies,¹²,¹³ only the social undesirability schema, which was finally deleted in a later version of the YSQ, was not identified by this Korean version. A higher-order factor analysis indicated that the 16 factors were reorganized into two higher-order factors, namely the vulnerable self and the inflated self.

The Korean version of the 75-item YSQ-S2 was also validated among 833 undergraduate students.¹¹ This study demonstrated that the 13-factor YSQ-S2 has good psychometric properties and reliability. That is, of the original 15 EMSs, the subjugation and the dependence/incompetence schemas were not identified as separate factors. It is of note that this study was conducted with Australian undergraduate students at the same time and showed a similar factor structure and internal reliability in the South Korean as well as Australian groups.

Given that the YSQ-S3 was developed in 2005, an updated Korean version of the YSQ-S2 is needed. The necessity of updating the YSQ-S2 is mainly because three schemas, the approval seeking/recognition seeking, the negativity/pessimism, and the punitiveness were added to the YSQ-S3. First, three additional schemas addressed by 15 items need to be validated because this addition could affect the psychometric properties of this instrument. Second, the use of the YSQ-S2 is also outdated in the context of contemporary schema-related research. As three schemas have been added, the structure of schema domains has also been changed. Thus, there is a limitation to compare data between two versions directly. Third, the use of the YSQ-S3 would let clinicians understand patients better with a new 18 set of schemas in a clinical setting. Therefore, the present study aimed to investigate the psychometric properties of a Korean version of the YSQ-S3, which we translated independently from the earlier version.

METHODS

Participants

The graduate medical students at the Kyungpook National University School of Medicine participated in this study and completed assessments during their first and third year of medical school from 2009 to 2013. Subjects with previous or current history of psychiatric or neurological diagnoses, or severe medical illness, were excluded. Students who scored above 23 on the Beck Depression Inventory (BDI) in their initial assessment were also excluded. In accordance with the exclusion criteria and/or the data with any missing values, 15 were ruled out. In the final analysis, the sample consisted of 542 graduate students [307 men and 235 women; mean age (SD)=25.2 (2.3), range=21-36 years]. A subsample of 308 subjects completed the Korean YSQ-S3 both before and after a 2-year test-retest interval. All participants provided written informed consent after the study had been fully explained. We used data drawn from the mental health assessment for graduate school students, an annual study conducted with the approval of the Ethics Committee of the Kyungpook National University School of Medicine.

Measures

Young Schema Questionnaire-Short form version 3 (YSQ-S3)

EMSs were assessed with the short form of the Young Schema Questionnaire, version 3 (YSQ-S3), which contains 90 items and assesses 18 EMSs.¹⁰ Each item is rated on a six-point Likert scale. Higher scores are indicative of stronger endorsements of dysfunctional beliefs. The 18 subscales are grouped into five broad categories, referred to as schema domains: the disconnection and rejection domain consists of abandonment/instability, mistrust/abuse, emotional deprivation, defectiveness/incompetence, and social isolation/alienation; the impaired perfor-mance and autonomy domain consists of dependence/incom-petence, vulnerability to harm or illness, enmeshment/undeveloped self, and failure; the impaired limits domain consists of entitlement/grandiosity and insufficient self-control/self-discipline; the other-directedness domain consists of subjugation, self-sacrifice, and approval-seeking/recognition-seeking; and the overvigilance and inhibition domain consists of negativity/pessimism, emotional inhibition, and unrelenting standards/hypercriticalness.

The previous version of the YSQ-S2 contained 75 items and was composed of the five highest-loading items for each schema, as determined by factor analysis.¹³ The Korean version of the YSQ-S2 has good psychometric properties and internal reliability (α=0.94).¹¹ An additional 15 items addressing three new schemas (approval seeking/recognition seeking, negativity/pessimism, and punitiveness) were included in the current YSQ-S3.

With the permission of the authors, the YSQ-S3 was translated into Korean by an experienced psychiatrist and a clinical psychologist. It was then back-translated by a bilingual individual, and modifications were made. The final version was approved by the two original translators.

Symptom Checklist-90, revised version

The Symptom Checklist-90 (SCL-90-R) is a well-validated self-report questionnaire assessing a broad range of psychological problems and symptoms of psychopathology.²¹ Respondents rate 90 items on a five-point scale (1=no problem to 5=very serious) measuring the extent to which they have experienced particular symptoms in the past 7 days. The SCL-90-R consists of nine symptom scales (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, psychoticism) and three global indices [the Global Severity Index (GSI), the Positive Symptom Distress Index (PSDI), and the Positive Symptom Total (PST)]. Higher scores on the SCL-90-R indicate greater psychological distress. The internal consistency coefficient alphas for the nine symptom dimensions ranged from 0.77 for psychoticism, to 0.90 for depression.

This scale was included to study the validity of the YSQ-S3 based on their relationships to symptoms of Axis I disorders such as depression and anxiety. The cognitive content-specificity theory states that each disorder could be characterized by a cognitive content, possibly an EMS in this study, that is specific to that disorder. We used the Korean version of the SCL-90-R,²² and only four general symptoms scales (somatization, interpersonal sensitivity, depression, and anxiety) were included in the analysis considering the fact that our subjects are healthy graduate students and multiple correlations may cause confusion to interpret the result.

Modified Experiences in Close Relationships

The Modified Experiences in Close Relationships (ECR-M36) scale, a modified version of the original 36-item ECR, was designed in 2008 to assess attachment to significant others among medically ill, older individuals.²³ Each question is scored on a seven-point Likert scale. Odd-numbered questions relate to the avoidance dimension (α=0.88) and even-numbered questions relate to the anxiety dimension (α=0.91). Higher scores indicate more attachment avoidance or attachment anxiety. We included this scale because the theoretical roots of schema theory firmly embedded in attachment theory. The Korean version of the ECR-M36 has good psychometric properties and internal reliability (α=0.87 for all items; α= 0.85 for anxiety; α=0.82 for avoidance).²⁴

Beck Depression Inventory

Current levels of depression were measured with the Korean version of the Beck Depression Inventory (BDI),²⁵ a 21-item scale that targets the cognitive, behavioral, affective, and somatic components of depression. The measure was initially designed to be administered by clinicians, but it is now used primarily as a self-report measure. Each of the 21 items asks respondents to select which of four statements most accurately reflects the intensity of a given symptom during the past week.²⁶

Statistical analyses

We used the Kolmogorov-Smirnov test as the goodness-of-fit index (GFI) to assess the normality of the distribution of individual schema scores. The internal consistency of all 90 items, as well as of the five items in each schema, was estimated using Cronbach's alpha. Test-retest reliability was assessed using Pearson's correlation coefficients. To assess the concurrent validity of the instrument, correlations between the YSQ-S3 subscales and four of the SCL-90-R subscales, two of the attachment domains of the ECR-M36, and BDI scores were analyzed. Additionally, multiple regression analysis was employed to investigate the extent to which each of the 18 schemas predicted the level of depression and anxiety. SPSS software (version 13; SPSS, Inc., Chicago, IL, USA) was used to perform all analyses. A Bonferroni correction was applied to control the family-wise error rate (0.05/18=0.0028).

To test whether our YSQ data fitted an 18-factor structure,¹⁵ we conducted confirmatory factor analysis (CFA) with the LISREL 8.80 software (Scientific Software International, Inc., Lincolnwood, IL, USA). The maximum likelihood (ML) estimation approach was used to estimate the parameters. Traditionally, chi-squared tests are used in two ways. First, a non-significant chi-square statistic suggests that the model does not deviate from the data. Second, when the chi-square statistic is significant but less than twice the degrees of freedom, the model is considered to be a good representation of the data. However, chi-square values are highly sensitive to sample size and tend to overestimate the badness of a model fit. Thus, the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) were calculated in addition to the more traditional chisquare and GFI values. As a general rule, GFI values >0.85, CFI values >0.90, and a RMSEA ≤0.08 are considered satisfactory, with CFI values >0.95 indicating excellent model fit. Among these fit indices, CFI seems to be the best and most valid because it has a minimal sampling variability and a negligible downward bias relative to the other indices.

RESULTS

Descriptive statistics

Mean scores for the 18 individual schemas are presented in Table 1. Mean schema scores were similar to (albeit lower than) those for the previous version of the YSQ-S2 with a South Korean sample.¹¹ The Kolmogorov-Smirnov GFI test for a normal distribution indicated a positively skewed distribution for all schemas (D values: 0.08-2.77, all df=542, all p<0.001).

Table 1. Descriptive statistics for the subscales of YSQ-S3 (N=542).

Standard error of skewness=0.11, standard error of kurtosis=0.21. SD: standard deviation, YSQ-S3: Young Schema Questionniare-Short form version 3

Reliability

The internal consistency of the 90-item Korean YSQ-S3, measured by the Cronbach's alpha coefficient, was 0.97. With the following exceptions, the internal consistency for each schema was higher than the generally accepted value of 0.70: 0.69 for insufficient self-control and punitiveness, 0.66 for self-sacrifice, and 0.59 for entitlement/grandiosity (Table 2). However, Kline noted that values below even 0.70 can be expected when dealing with psychological constructs due to the diversity of the constructs being measured.²⁷

Table 2. Split-half reliability for the subscales of YSQ-S3 (N=542).

YSQ-S3: Young Schema Questionniare-Short form version 3

In terms of the corrected item-total correlation within each schema, all 90 items showed correlations >0.25 with the total score for each schema. However, items 45 (enmeshment/undeveloped self), 46 (subjugation), 69 (insufficient self-control), 86 (entitlement), and 90 (punitiveness) showed relatively low item-total correlations (<0.30).

Among the 308 participants who completed the YSQ-S3 before and after a 2-year interval, the Pearson's r correlation coefficient for test-retest reliability ranged from 0.46 (entitlement/grandiosity) to 0.65 (abandonment) (all p<0.001) (Table 3).

Table 3. Pearson's correlation between the test and re-test for the subscales of YSQ-S3 (N=308).

All p<0.001. YSQ-S3: Young Schema Questionniare-Short form version 3

Concurrent validity

Correlations between the YSQ-S3 and the SCL-90-R, BDI, and ECR-M36 are presented in Table 4. In general, every schema was positively correlated with almost all psychological measures.

Table 4. Pearson's correlations between the YSQ-S3, SCL-90R, BDI, and ECR-M36.

^*p<0.001, ^**p<0.0001. N=542 for SCL-90-R and BDI; N=217 for ECR-M36. YSQ-S3: Young Schema Questionnaire-short form 3, SCL-90-R: Symptom checklist-90-R, BDI: Beck Depression Inventory, ECR-M36: Modified experiences in close relationships

In terms of the SCL-90-R subscales, somatization was significantly correlated with the schemas, albeit relatively more weakly than were the other symptom subscales. Interpersonal sensitivity showed the strongest correlation with social isolation/alienation (r=0.55, p<0.001), followed by mistrust/abuse (r=0.54, p<0.001). Depression showed the strongest correlation with social isolation/alienation (r=0.57, p<0.001), followed by failure (r=0.52, p<0.001), mistrust/abuse (r=0.52, p<0.001), and defectiveness/shame (r=0.50, p<0.001). Similar relationships were found with the BDI. Anxiety showed the highest correlation with mistrust/abuse (r=0.47, p<0.001), followed by social isolation/alienation (r=0.46, p<0.001) and vulnerability to harm or illness (r=0.440, p<0.001).

In the context of attachment to others, the anxiety domain showed the strongest correlation with the emotional deprivation schema (r=0.49, p<0.001), whereas the avoidance domain showed the strongest correlation with the abandonment schema (r=0.68, p<0.001).

Multiple regression analysis was performed using the simultaneous entry method (Table 5). The 18 schemas accounted for a total of 41% (39% adjusted) of the variance in BDI scores (F= 20.46, p<0.001) and 31% (29% adjusted) of the variance in anxiety scores (F=13.08, p<0.001) (Table 5).

Table 5. Multiple regression for depression and anxiety.

^*dependent variable: Beck depression score, ^†R²: 0.41, F: 20.46, p<0.001, ^‡constant: B (SE)=-3.41 (0.83), t=-4.12, ^§dependent variable: anxiety score from SCL-90-R, ^∥R²: 0.31, F : 13.08, p<0.001, ^¶constant: B (SE)=31.79 (0.97), t=32.71

Confirmatory factor analysis

The fit indices and ML estimates for an 18-factor model are presented in Table 6. The CFA for the 18-factor structure originally proposed by Young et al.⁷ showed that most, but not all, goodness-of-fit statistics were indicative of a satisfactory fit. Indeed, the chi-square value was significant (χ²₃₇₆₂=4454.46, p<0.001) but lower than the desired 2:1 chi-square:df ratio. The other fit indices reached conventional levels of adequacy: RMSEA=0.03, 90% confidence interval for RMSEA=(0.027, 0.034), p-value for test of close fit (RMSEA<0.05)=1.00, and CFI=1.00. However, the GFI value was 0.67, which is below the satisfactory value of 0.85.

Table 6. Maximum likelihood estimates for a 18-schema structure.

All parameters significant at p<0.05; Estimates over 0.4 are shown in bold. χ²(3762)=4454.46 (p<0.001), CFI=1.00, RMSEA=0.03, GFI=0.67. S1: Emotional deprivation, S2: Abandonment, S3: Mistrust/Abuse, S4: Social Isolation/Alienation, S5: Defectiveness/Shame, S6: Failure, S7: Dependence/Incompetence, S8: Vulnerability to Harm or Illness, S9: Enmeshment/Undeveloped Self, S10: Subjugation, S11: Self-Sacrifice, S12: Emotional Inhibition, S13: Unrelenting Standards/Hypercriticalness, S14: Entitlement/Grandiosity, S15: Insufficient Self-Control/Self-Discipline, S16: Approval Seeking/Recognition Seeking, S17: Negativity/Pessimism, S18: Punitiveness

DISCUSSION

The present study examined the reliability, validity, and factor structure of the Korean version of the YSQ-S3. The Korean version of the YSQ-S3 was found to be reliable and stable. The internal consistency of all 90 items was 0.97, and that of individual schemas was generally higher than the accepted value of 0.70. These findings are consistent with versions in other languages such as Arabic (0.71-0.88; 0.74-0.86),²⁸,²⁹ Romanian (0.68-0.96),³⁰ and Turkish (0.63-0.80)³¹ as well as with the previous Korean version of the YSQ-S2 (0.72-0.90 for 13 schemas).¹¹ Although the schemas with low internal consistency varied across studies, entitlement/grandiosity showed the lowest internal consistency (0.59) in this study. In fact, item-correlations for these five items were all <0.40, and that for item 86 was only 0.25. One reason may be that the positive connotation of "value" and "contribution," which appear in item 86 ("I feel that what I have to offer is of greater value than the contributions of others"), may make participants, especially medical school students, respond more positively to this item than to other items.

In terms of test-retest reliability, the correlation coefficients for 18 schemas before and after the 2-year interval were between 0.46 and 0.64. Considering the long interval, the correlations between the YSQ-S3 at baseline and follow-up were relatively strong. Consistent with the reports of Riso et al.³² (rho=0.55-0.85) and Wang et al.³³ (rho=0.42-0.73), these findings may rather support the long-term stability of EMSs.

The correlations between the YSQ-S3 and the SCL-90R, BDI, and ECR-M36 further demonstrate the concurrent validity of the YSQ-S3. Consistent with the previous version, the YSQ-S2,¹¹ and versions of the YSQ-S3 in other languages,²⁸,²⁹,³⁰,³¹ every schema measured by the Korean version of the YSQ-S3 showed statistically significant correlations with measures of the clinical symptoms of depression, anxiety, somatization, and social isolation. Moreover, the schemas accounted for 41% and 31% of the total variance in depression and anxiety symptoms, respectively, in this study. In fact, the schemas are well-known predictors of depression and anxiety, accounting for approximately 40-50% of the variance of depressive and anxiety symptoms.⁵,¹⁶ In this sense, some authors have already proposed that schema theory and therapy should be applied to treat mood and anxiety disorders.⁵

This study also revealed that the YSQ-S3 was closely related to two dimensions of adult attachment: attachment anxiety, defined as fear of interpersonal rejection or abandonment, and attachment avoidance, defined as fear of dependence and interpersonal intimacy.³⁴ In the schema model, secure attachment to others (which includes safety, stability, nurturance, and acceptance) is one of five core emotional needs of human beings. According to this model, childhood emotional development proceeds from attachment to autonomy and individuation.² Young et al. originally designed the first domain, the so-called disconnection and rejection domain, to describe patients with schemas that rendered them unable to form secure, satisfying attachments to others.² In fact, as originally conceived, emotional deprivation and abandonment, which showed the strongest correlations with attachment anxiety and attachment avoidance, respectively, were included in this domain. These findings may support the incorporation of attachment theory into the basic conceptual assumptions of schema theory. However, other schemas that do not belong to the rejection and disconnection domain also showed significant correlations with attachment domains, especially with attachment avoidance. A previous study reported that attachment avoidance and anxiety were significantly associated with all schemas within the disconnection domain, and were associated with a number of EMS in other domains.³⁵ We assume that attachment may have an impact on the development of schemas in general.

The CFAs using the ML procedure in LISREL indicated that the hypothesized 18-factor structure of the Korean version of the YSQ-S3 provided an acceptable fit to the data. Several previous studies have tested the factor structure of the YSQ-S3 with mixed results, finding 18-,³⁶ 17-,²⁸ and 14-factor structures.³¹ These minor discrepancies in the factor structures identified by different studies may be attributable to various causes, including cultural differences, translation problems, and the use of different subject groups. Although the previous Korean version of the YSQ-S2 showed a moderate fit with the data according to the 15-factor model, a 13-factor solution, which excluded the dependence/independence and subjugation schemas, provided a better fit for the data. Unlike the Korean version of the YSQ-S2, our version of the YSQ-S3 showed a good fit with the original factor structure. It is worth noting that our version was translated independently from the earlier version.

The current study has several limitations. First, caution is needed in generalizing our findings because our subjects were graduate medical students. In fact, their mean scores on schemas were generally lower, by 1-2 points, than were those of the undergraduate psychology students examined in previous research regarding the YSQ-S2. However, our subjects may be suitable for validation of this measure because they constituted a highly homogenous and reliable sample. Further research with more representative samples including various age groups is needed. Second, as the YSQ was developed to assess severe and long-lasting characterological problems in clinical subjects, such as patients with personality disorders, additional studies with clinical samples are needed to establish its discriminant validity. Third, though we included the SCL-90R and attachment scale for concurrent validity, the lack of measures regarding core beliefs or personality may limit the interpretation of concurrent validity of the Korean version of the YSQ-S3.

In conclusion, the present study confirmed the reliability, validity, and sound psychometric properties of the Korean version of the YSQ-S3 in healthy medical students. Future research with clinical samples is needed to determine its clinical utility.