Abstract
Objective: The aim of the study was to assess the initial psychometric properties of the Cognitive Distortions Questionnaire (CD-Quest) in its Brazilian Portuguese version tested in adult undergraduate students.
Methods: Brazilian undergraduate medical and psychology students comprising the sample (n=184) completed the following measures: Cognitive Distortions Questionnaire, Beck Depression Inventory and Beck Anxiety Inventory. These self-report instruments were administered collectively in classrooms.
Results: The Cognitive Distortions Questionnaire showed adequate internal consistency (Cronbach’s alpha=0.85) and concurrent validity with Beck Depression Inventory (r=0.65, p<0.001) and Beck Anxiety Inventory (r=0.52, p<0.001). Furthermore, it was able to discriminate between groups possessing depressive (Beck Depression Inventory composite score ≥12) and anxious (Beck Anxiety Inventory composite score ≥ 11) indicators from those not possessing them (p<0.001). Principal components showed the measure was unidimensional, and it explained about 29 percent of the data variance. A confirmatory factor analysis showed that all the regression coefficients are greater than or equal to 0.40
Conclusion: The original Brazilian version of the Cognitive Distortions Questionnaire is adequate for use in the context of national undergraduate students and is able to separate different cognitive distortions. However, further studies using clinical samples are needed.
Keywords: Validation, questionnaire, Cognitive Distortions Questionnaire, CD-Quest
INTRODUCTION
It has been shown that cognition influences emotion, behavior, and interpersonal relationships and facilitates information processing at implicit (nonconscious) and explicit (conscious) levels of awareness.1 Thus, when information is processed in a biased way, consequent emotional and behavioral responses may be dysfunctional and negatively impact interpersonal relationships and one’s well-being.2 The ability to identify and correct biases in thinking is essential in the practice of cognitive behavioral therapy (CBT).3
Cognitive behavioral therapists have identified three levels of cognition. At the most superficial level, automatic thoughts (ATs)—considered to be situation-specific—are the easiest to identify and modify. Negative ATs are thought to be mostly the consequence of an underlying error in reasoning. Such cognitions can be assessed with the Automatic Thoughts Questionnaire (ATQ-30), a 30-item questionnaire developed by Hollon and Kendall4 to measure the frequency of occurrence. Other measures to assess cognitive errors have been developed, including the Cognitive Bias Questionnaire (CBQ)5 and the Cognitive Errors Questionnaire.6 More recently, Covin et al3 developed the Cognitive Distortions Scale (CDS), an instrument that measures the tendency to make 10 cognitive distortions in interpersonal and achievement domains. Cognitions occurring at an intermediate level, characterized by underlying assumptions and rules, can be assessed by the Dysfunctional Attitude Scale, designed to assess negative attitudes of patients with depression toward self, outside world, and the future.7 The deepest level of cognition comprises schemas and core beliefs, and these constructs can be measured using the Personality Beliefs Questionnaire8 and the Young Schema Questionnaire.9
This study focuses on the first and more easily accessed, although not less important, level of cognition—ATs. According to cognitive theory, patients with mental health disorders demonstrate negatively biased information processing, which in turn influences the nature of the thoughts they experience in specific situations.2 In many instances, these ATs are characterized by one or more errors in thinking that results from biased information processing. These cognitive errors, also called cognitive distortions, are easily identified and labeled by patients as part of the treatment.10
The purpose of developing the Cognitive Distortions Questionnaire (CD-Quest) was to create an instrument to be used by patients during the course of CBT treatment in order to facilitate connections between cognitive errors and their consequent emotional states as well as dysfunctional behaviors. The instrument was also designed to be used by therapists in order to obtain quantitative estimates of the frequency and intensity of their patients’ cognitive errors and the manner these variables respond to CBT over time.
Although the English11,12 and the Brazilian Portuguese13 versions of the CD-Quest were constructed simultaneously by the first author (I.R.O.), the objective of this study was to analyze the psychometric properties of the Brazilian version regarding internal consistency, test-retest properties, concurrent validity, construct validity, and discriminant validity.
METHODS
This study was conducted in two phases, separately described in this section. The objective of Phase I was to construct the CD-Quest, and the objective of Phase II was to assess its properties in a sample of undergraduate medical and psychology students.
Phase I: Development of the CD-Quest. The first author (I.R.O.) reviewed cognitive therapy books2,10,14-17 that included a list of cognitive distortions and their definitions, organized known cognitive distortions in a preliminary list with clinical examples, put them together in a grid similar to that devised by Williams et al18 for the Hamilton Depression Rating Scale,19 organized them in two dimensions (frequency and intensity of the belief), and made it available to be examined by members of the Academy of Cognitive Therapy (www.academyofct.org) through its listserv. After review and suggestions given by several members, a final 15-item questionnaire was compiled. The Brazilian Portuguese version was designed at the same time as the English version by the first author (I.R.O.). Each item score ranged from 0 to 5, measured both dimensions simultaneously, and was presented grid (Figure 1).
FIGURE 1.
First page presentation of the CD-Quest
Phase II: Assessment of a sample. Undergraduate medical (58.0%) and psychology (42.0%) students comprised the sample of participants for the study. There was a preponderance of female subjects (67%), and the mean (± standard deviation [SD]) sample age was 21.85±3.37 years. All subjects volunteered to participate and signed an informed consent.
Instruments. In addition to the CD-Quest, the following questionnaires were administered:
The Beck Depression Inventory (BDI),20,21 the Brazilian version of which was translated and adapted to Brazilian Portuguese,22 is a 21-item, self-report questionnaire that measures the severity of depressive symptoms and of which total scores range from 0 to 63. It has high internal consistency and concurrent validity vis-à-vis other measures of depression.21 Each item is rated on a four-point scale ranging from 0 to 3. Its psychometric properties have been found to be satisfactory.23
The Beck Anxiety Inventory (BAI),24 of which a Brazilian version was also translated and adapted to Brazilian Portuguese,22 is a self-report instrument for the assessment of general aspects of anxiety consisting of 21 items scored on a five-point Likert scale ranging from 0 to 4. It showed high internal consistency and test-retest reliability over one week.
CD-Quest, BDI, and BAI were applied jointly and collectively in the classrooms. At least two members of the research team was available to clarify any student concerns and to answer questions.
Data collection. This study was approved by the Institutional Review Board of the University Hospital, at Federal University of Bahia. Undergraduate students from two courses, medicine and psychology, were contacted. The subjects who agreed to participate in the study responded to the self-report instruments CD-Quest, BDI, and BAI in their classrooms, after signing the informed consent. The questionnaires were applied again 2 to 3 weeks later for test-retest analysis.
Data coding and treatment. The were allocated to a data bank and submitted to statistical analysis using the R software. The demographic and clinical data of the sample under study were analyzed by applying descriptive statistics. The Student’s t test was used for group comparisons.
The following statistical procedures were applied for the study of validity/reliability of the CD-Quest:
Cronbach’s alpha for the evaluation of the internal consistency of the scales—The Cronbach’s alpha values considered to be acceptable were those exceeding 0.70.25,26
Pearson correlation coefficient (r) between the total scores of the CD-Quest, BDI, and BAI to assess the concurrent validity between scales—The interpretation of the magnitude of the correlations detected was defined as follows: 0 to 0.25, weak; 0.26 to 0.50, moderate; 0.51 to 0.70 strong; and above 0.71, very strong.27
Principal component analysis (PCA) with varimax rotation to assess the construct validity of the scales—The measure of sampling adequacy used was Kaiser-Meyer-Olkin index above 0.60; significant Bartlett’s sphericity test, eigenvalues greater than one; and factor loadings greater than or equal to 0.40.28
Intraclass correlation coefficients were used for the analysis of test-retest reliability.29
The level of significance was set at p value of 0.05 or less in all analyses.
RESULTS
Frequency of the items and internal consistency. Each CD-Quest item was assessed individually, and the mean scores, SDs, and percentage of the scores ranging from 0 to 5 were calculated. Data are shown in Table 1. Mean scores ranged from 1.03 to 2.31. Globally, the highest mean scores were found for Item 11 (“should” statements), Item 14 (“what if...”), and Item 8 (“mind reading”), all of which were also found to have the highest scores, as expected.
TABLE 1.
Distribution of CD-Quest items according to mean scores, SD, and percentage of scores 0-5 in a sample of university students (N = 84)
| CD-QUEST ITEMS | MEAN | SD | SCORES, % | |||||
|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 2 | 3 | 4 | 5 | |||
| 1. Dichotomous thinking | 1.55 | -1.3 | 26 | 27 | 22 | 17 | 6 | 2 |
| 2. Fortune telling | 1.49 | -1.32 | 32 | 24 | 22 | 15 | 8 | 1 |
| 3. Discounting positives | 1.06 | -1.29 | 49 | 24 | 13 | 11 | 4 | 2 |
| 4. Emotional reasoning | 1.52 | -1.37 | 30 | 24 | 23 | 14 | 6 | 3 |
| 5. Labeling | 1.51 | -1.35 | 32 | 26 | 21 | 16 | 5 | 3 |
| 6. Magnification/minimization | 1.19 | -1.3 | 51 | 25 | 16 | 13 | 3 | 2 |
| 7. Selective abstraction | 1.45 | -1.37 | 36 | 19 | 23 | 14 | 9 | 1 |
| 8. Mind reading | 2.11 | -1.37 | 13 | 24 | 26 | 21 | 11 | 5 |
| 9. Overgeneralization | 1.09 | -1.4 | 51 | 17 | 14 | 10 | 5 | 3 |
| 10. Personalizing | 1.03 | -1.25 | 47 | 29 | 12 | 9 | 3 | 2 |
| 11. Should statements | 2.31 | -1.5 | 15 | 16 | 23 | 22 | 16 | 8 |
| 12. Jump to conclusions | 1.44 | -1.36 | 32 | 26 | 20 | 12 | 6 | 3 |
| 13. Blaming | 1.07 | -1.38 | 51 | 19 | 15 | 9 | 4 | 3 |
| 14. What if... | 2.16 | -1.6 | 20 | 18 | 21 | 19 | 13 | 10 |
| 15. Unfair comparisons | 1.63 | -1.5 | 32 | 19 | 19 | 17 | 9 | 4 |
| Total score | 22.59 | -11.86 | - | - | - | - | - | - |
SD: standard-deviation
The internal consistency of the CD-Quest was calculated separately for both subscales, frequency, and intensity, and for the total scale. The Cronbach’s alpha values were 0.91, 0.89, and 0.85, respectively. These values are indicative of good internal consistency. The item-total correlation coefficients ranged from 0.47 to 0.76 for the frequency subscale, 0.46 to 0.71 for the intensity subscale, and 0.45 to 0.70 for the total scale. Regarding the total scale, the highest correlation was for Item 15 (“unfair comparisons”) and the lowest correlation was for Item 4 (“emotional reasoning”). For any of the subscales and the total scale, there are no items that could increase the Cronbach’s alpha if they were deleted.
Concurrent validity. Concurrent validity was studied by determining the correlation between the total scores and subscales scores of the CD-Quest, as well as the BAI, and BDI scales, values of which are presented in Table 2.
TABLE 2.
CD-Quest concurrent validity in a sample of university students (N=184)
| CD-QUEST | BAI | BDI | SS FREQUENCY | SS INTENSITY |
|---|---|---|---|---|
| Total scale | 0.52** | 0.65** | 0.95** | 0.96** |
| SS frequency | 0.50** | 0.61** | NA | 0.85** |
| SS intensity | 0.47** | 0.59** | 0.85** | NA |
p<0.01
SS: subscale (CD-Quest); BAI: Beck Anxiety Inventory; BDI: Beck Depression Inventory; NA: not applicable
All correlations between the CD-Quest, its subscales, and the clinical self-report inventories were significant (p<0.01). Correlations between the CD-Quest subscales and BDI ranged from 0.59 to 0.65, the highest correlation observed, indicating a convergence between cognitive distortions and depressive symptoms, as well as the validity of CD-Quest construct. Similarly, a convergence between the CD-Quest and BAI was observed, correlations of which were considered moderate.
Discriminant validity. The CD-Quest total scale mean score was 22.59 (SD=11.86). Frequency subscale mean score was 15.17 (SD)=7.43), and intensity subscale mean score was 17.44 (SD=8.04). Table 3 shows the CD-Quest total scores distribution according to percentiles, allowing classification of cognitive distortions as absent/minimal, slight, moderate and severe.
TABLE 3.
Classification of CD-Quest scores according to magnitude (severity) of distortions in a sample of university students (N=84)
| PERCENTILE | CD-QUEST SCORES | CLASSIFICATION |
|---|---|---|
| 1 | 3 | Absent/Minimal |
| 5 | 4 | |
| 10 | 8 | |
| 15 | 10 | |
| 20 | 11 | |
| 25 | 14 | |
| 30 | 15 | Slightly |
| 40 | 18 | |
| 50 | 22 | |
| 60 | 25 | Moderately |
| 70 | 29 | |
| 75 | 32 | |
| 80 | 35 | Severely |
| 85 | 37 | |
| 90 | 39 | |
| 95 | 42 | |
| 99 | 52 | |
| 100 | 60 |
Subjects were separated in two subgroups according to the BDI scores: those indicating depressive symptoms (BDI≥12) and those without depressive symptoms (BDI<12). For each subgroup, CD-Quest mean scores and SDs (total scale, frequency subscale, and intensity subscale) were calculated.
Data shown in Table 3 suggest that the CD-Quest was able to identify different groups, indicating that subjects with depressive symptoms had a mean score significantly higher than those not presenting depressive symptoms. Table 4 also shows that the CD-Quest mean score was significantly higher in anxious subjects (BAI≥11) than in nonanxious subjects (BAI < 11).
TABLE 4.
CD-Quest mean scores and SDs according to total scale and SS with and without indicators of depression and anxiety in a sample of university students (N = 84)
| CD-QUEST | INDICATORS OF DEPRESSION BDI | INDICATORS OF ANXIETY BAI | ||||
|---|---|---|---|---|---|---|
| YES n=52 | NO n=132 | STATISTICS | YES n=43 | NO n=141 | STATISTICS | |
| Total scale, mean (SD) | 31.57 (10.36) | 19.12 (10.47) | t= -7.27 p<0.0001 | 30.06 (11.16) | 20.34 (11.16) | t= -4.99 p<0.0001 |
| SS frequency, mean (SD) | 20.38 (6.78) | 13.16 (6.65) | t= -6.57 p<0.0001 | 19.72 (7.05) | 13.79 (7.02) | t= -4.83 p<0.0001 |
| SS intensity, mean (SD) | 23.13 (6.38) | 15.27 (7.5) | t=-7.13 p<0.0001 | 22.11 (6.93) | 16.04 (7.85) | t= -4.55 p<0.0001 |
SD: standard deviation; SS: subscale; BDI: Beck Depression Inventory; BAI: Beck Anxiety Inventory
Principal component analysis (PCA). The Kaiser-Meyer-Olkin (KMO) measure was 0.86, and the Bartlett’s sphericity test was highly significant (chi-squared=749,22; p<0.001), which suggests that the sample is adequate to perform the principal component analysis.
The factor structure that best explained the variance of the CD-Quest items was unidimensional (i.e., all of the items were loaded onto one single component). This conclusion was based upon comparison of different techniques: Kaiser’s criterion (number of factors equal to number of eigenvalues > 1), parallel analysis, and the distribution of factor loadings across different components. The explained variance for one single dimension was 29 percent, which is expected for a unidimensional measure with 15 items. In order to maximize the variance of the squared loadings on all the items, the varimax rotation was performed along with the PCA. Table 5 shows the rotated component matrix with loadings of the CD-Quest items (total scale), their respective communalities, and their corrected item-total correlations.
TABLE 5.
Psychometric properties of the CD-Quest items (total scale) in a sample of university students (N 84)
| CD-QUEST ITEMS | FACTOR 1 | h2 | ITC | ALPHA IF ITEM DELETED |
|---|---|---|---|---|
| 1. Dichotomous thinking | 0.43 | 0.18 | 0.39 | 0.85 |
| 2. Fortune telling (catastrophizing) | 0.45 | 0.21 | 0.42 | 0.85 |
| 3. Discounting positives | 0.43 | 0.19 | 0.4 | 0.85 |
| 4. Emotional reasoning | 0.4 | 0.16 | 0.36 | 0.86 |
| 5. Labeling | 0.65 | 0.42 | 0.59 | 0.84 |
| 6. Magnification/minimization | 0.52 | 0.27 | 0.49 | 0.85 |
| 7. Selective abstraction | 0.52 | 0.27 | 0.48 | 0.85 |
| 8. Mind reading | 0.58 | 0.33 | 0.53 | 0.85 |
| 9. Overgeneralization | 0.58 | 0.34 | 0.53 | 0.85 |
| 10. Personalizing | 0.62 | 0.38 | 0.56 | 0.84 |
| 11. Should statements | 0.55 | 0.31 | 0.51 | 0.85 |
| 12. Jump to conclusions | 0.54 | 0.29 | 0.5 | 0.85 |
| 13. Blaming | 0.53 | 0.28 | 0.48 | 0.85 |
| 14. What if... | 0.44 | 0.2 | 0.42 | 0.85 |
| 15. Unfair comparisons | 0.72 | 0.51 | 0.65 | 0.84 |
h2: communality; ITC: corrected item-total correlation
Confirmatory factor analysis. A confirmatory factor analysis was performed in order to check the unidimensionality of the CD-Quest. The fit measures for the overall model are close to the expected (RMSEA<0.075, CFI=0.87, GFT=0.89, and chi-squared (90) = 179,85, p<0.001) in the literature30 and all the regression coefficients are 0.40 or greater.Figure 2 shows the structural equation modeling with the regression coefficients for the 15 items of the CD-Quest.
FIGURE 2.
Confirmatory factor analysis of the CD-Quest
Test-retest reliability. Intraclass correlation (ICC) was calculated to investigate the test-retest reliability for the CD-Quest over a period of 3 to 4-weeks. The results of the ICC for the total scale (ICC=0.87+0.82-0.90 confidence interval [CI]), and for the frequency and intensity subscales (0.86+0.81-0.89 CI and 0.85+0.80-0.89 CI, respectively), indicated very satisfactory repeatability.
DISCUSSION
The aim of this study was to develop and validate the CD-Quest, a questionnaire designed to allow clinicians and researchers to assess commonly identified cognitive distortions in CBT clinical practice.
Other questionnaires have been developed with the aim of assessing cognitive errors, such as the CBQ,5 ATQ,4 CEQ,6 and CDS,3 and of these, the CDS is the only one that permits the examination of specific cognitive errors. Covin et al3 assessed the frequency with which responders experienced 10 cognitive errors listed by Burns15 (mindreading, catastrophizing, all-or-nothing thinking, emotional reasoning, labeling, mental filter, overgeneralization, personalization, should statements, and minimizing or disqualifying the positive) in two domains: interpersonal and achievement situations. The CD-Quest, however, is different in that it assesses 15 cognitive errors considering the dimensions frequency of occurrence and intensity of the credit given to them by the respondents.
The psychometric indicators of the Brazilian version of the CD-Quest reveal that it is an adequate and promising measure of common cognitive distortions. Regarding its internal consistency, the alpha values obtained were significant for the various groups both for the total scale and the subscales. All items were also significantly correlated with the total, demonstrating high conformity among them and the adequacy and coherence of the instrument for construct validity.
The concurrent validity correlation of the CD-Quest with the BAI and BDI was classified as strong (BAI=0.52, p<0.001; BDI=0.65, p<0.001). Compared to the study of concurrent validity of the CDS conducted by Covin et al,3 the values obtained in our study were superior in comparison with the scale used in Covin’s study regarding the BDI. In that study,3 the correlation between the CDS and the BDI-II was moderate (r=0.39).
The best factor solution for the CD-Quest was unidimensional and is consistent with the study by Covin et al.3 In their study of 10 cognitive errors, the authors performed a factor analysis that supported the use of a unidimensional model, although, according to them, further analyses suggested that some errors may have more clinical significance than others.3
The inter-rater reliability of the CD-Quest proved to be strong. This agreement is believed to be attributed to clear explanation of the definitions of cognitive distortions and examples given to the participants before filling in the questionnaires.
Limitations. As the CD-Quest was developed to be used in clinical contexts, an obvious and important limitation of this study is that we did not use a clinical sample. Rather, our study comprised a sample of undergraduate students from a single region of Brazil, which prevents the generalizability of the findings to other contexts, such as general and clinical populations. However, it is not uncommon for researchers to first test a clinically useful measure with nonclinical samples and then proceed to the next stage of validation (i.e., testing it in clinical samples). 3 With this in mind, future studies evaluating the psychometric aspects of this questionnaire in diversified samples, especially in the general population, are necessary.
In general, we conclude that the CD-Quest in its Brazilian Portuguese version is adequate in terms of its psychometric quality, with satisfactory validity and reliability indices.
Footnotes
FUNDING:No funding was received for the preparation of this manuscript.
FINANCIAL DISCLOSURES:The authors have no conflicts of interest relevant to the content of this article.
REFERENCES
- 1.Alford BA, Beck AT. The Integrative Power of Cognitive Therapy. New York (NY): Guilford Press; 1997. [Google Scholar]
- 2.Beck AT. Cognitive Therapy and the Emotional Disorders. New York (NY): Meridian; 1976. [Google Scholar]
- 3.Covin R, Dozois DJA, Ogniewicz A, Seeds PM. Measuring cognitive errors: initial development of the cognitive distortions scale (CDS) Int J Gognit Ther. 2012;4(3):297–322. [Google Scholar]
- 4.Hollon DH, Kendall PC. Cognitive self-statements in depression: development of an Automatic Thoughts Questionnaire. Cognit Ther Res. 1980;4(4):383–395. [Google Scholar]
- 5.Krantz S, Hammer C. Assessment of cognitive bias in depression. J Abnorm Psychol. 1979;88(6):611–619. doi: 10.1037//0021-843x.88.6.611. [DOI] [PubMed] [Google Scholar]
- 6.Lefebvre MF. Cognitive distortion and cognitive errors in depressed psychiatric and low back pain patients. J Consult Clin Psychol. 1981;49(4):517–525. doi: 10.1037//0022-006x.49.4.517. [DOI] [PubMed] [Google Scholar]
- 7.Weissman AN, Beck AT. Development and validation of the Dysfunctional Attitude Scale: a preliminary investigation. Presented at the Annual Meeting of the American Educational Research Association; 1978 Mar 27-31; Toronto.
- 8.Beck AT, Butler AC, Brown GK, et al. Dysfunctional beliefs discriminate personality disorders. Behav Res Ther. 2001;39(10):1213–1225. doi: 10.1016/s0005-7967(00)00099-1. [DOI] [PubMed] [Google Scholar]
- 9.Young JE. Cognitive Therapy for Personality Disorders: A Schema-focused Approach, Third Edition. Sarasota (FL): Professional Resources Press; 1999. [Google Scholar]
- 10.Beck JS. Cognitive Therapy: Basics and Beyond, Second Edition. New York (NY): Guilford Press; 2011. [Google Scholar]
- 11.de Oliveira IR. Trial-based cognitive therapy (TBCT): a new cognitive-behavior therapy approach. In: de Oliveira IR, Schwartz T, Stahl S, editors. Integrating Psychotherapy and Psychopharmacology: A Handbook for Clinicians. New York (NY): Routledge; 2014. pp. 24–65. [Google Scholar]
- 12.de Oliveira IR. Trial-Based Cognitive Therapy: A Manual for Clinicians. New York (NY): Routledge; 2015. [Google Scholar]
- 13.de Oliveira IR. Terapia cognitiva processual (TCP): uma nova abordagem de terapia cognitivo-comportamental. In: de Oliveira IR, Schwartz T, Stahl S, editors. Integrando Psicoterapia e Psicofarmacologia. Porto Alegre (RS): Artmed; 2015. pp. 26–65. [Google Scholar]
- 14.Beck AT, Rush A, Shaw B, Emery G. Cognitive Therapy of Depression. New York (NY): Guilford; 1979. [Google Scholar]
- 15. Burns DD. Feeling Good: The New Mood Therapy. New York (NY) Signet; 1980. [Google Scholar]
- 16.Dryden W, Ellis A. Rational emotive behavior therapy. In: Dobson KS, editor. Handbook of Cognitive-Behavioral Therapies, Second Edition. New York (NY): Guilford Press; 2001. pp. 295–348. [Google Scholar]
- 17.Leahy RL. New York (NY): Guilford Press; 2003. Cognitive Therapy Techniques A Practitioner’s Guide. [Google Scholar]
- 18.Williams JB, Kobak KA, Bech P, et al. The GRID-HAMD: standardization of the Hamilton Depression Rating Scale. Int Clin Psychopharmacol. 2008;23(3):120–129. doi: 10.1097/YIC.0b013e3282f948f5. [DOI] [PubMed] [Google Scholar]
- 19.Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960;23:56–62. doi: 10.1136/jnnp.23.1.56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Beck AT, Ward CH, Mendelsohn M, et al. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–571. doi: 10.1001/archpsyc.1961.01710120031004. [DOI] [PubMed] [Google Scholar]
- 21.Beck AT, Steer RA, Garbin MG. Psychometric properties of the Beck Depression Inventory: twenty-five years of evaluation. Clin Psychol Rev. 1988;8(1):77–100. [Google Scholar]
- 22.Cunha JA. Manual da Versão em Português das Escalas Beck. São Paulo (SP): Casa do Psicólogo; 2001. [Google Scholar]
- 23.Whisman MA, Perez JE, Ramel W. Factor structure of the Beck Depression Inventory-second edition (BDI-II) in a student sample. J Clin Psychol. 2000;56(4):545–551. doi: 10.1002/(sici)1097-4679(200004)56:4<545::aid-jclp7>3.0.co;2-u. [DOI] [PubMed] [Google Scholar]
- 24.Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56(6):893–897. doi: 10.1037//0022-006x.56.6.893. [DOI] [PubMed] [Google Scholar]
- 25.Cronbach LJ. Test reliability: its meaning and determination. Psychometnka. 1947;12(1):1–16. doi: 10.1007/BF02289289. [DOI] [PubMed] [Google Scholar]
- 26.Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993;78(1):98–104. [Google Scholar]
- 27.Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to their Development and Use, Third Edition. London: Oxford University Press; 2003. [Google Scholar]
- 28.Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess. 1995;7(3):286–299. [Google Scholar]
- 29.Fleiss J. Statistical Methods for Rates and Proportions. New York: John Wiley and Sons; 1981. [Google Scholar]
- 30.Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: a Multidisciplinary Journal. 1999;6(1):l–55. [Google Scholar]