Screening for Major Depression in Private Practice

Abstract

Background

Several studies have contrasted the 16-item self-report version of the Quick Inventory of Depressive Symptomatology (QIDS-SR₁₆) with other depression scales, but none has used patients in single, large, private psychiatric practice. This study compared 175 outpatients on the QIDS-SR₁₆, the 17-item Carroll Depression Rating Scale (CDRS-SR₁₇, a self-report modification of the Hamilton Rating Scale for Depression), and the thirteen depression items from the Symptom Check List-90 (SCL-D₁₃). The Mini version of the Structured Clinical Interview for DSM-IV (MiniSCID) served as a “gold standard” to assess depression.

Methods

Basic Item and scale statistics were obtained using classical test theory. Dimensionalities were obtained using factor analysis. Test information functions obtained from the Samejima item response theory model provided additional reliability-like results. This model was also used to compare patients classified as depressed vs. nondepressed on the basis of the MiniSCID. Additional validity information was assessed comparing: (a) ANOVA effect sizes, (b) receiver operating characteristic curves, (c) univariate logistic regression, (d) the MANOVA, and (e) multivariate logistic regression.

Results

The QIDS-SR₁₆ related most strongly to MiniSCID diagnoses. The SCL-D₁₃, however, was the most reliable of the three scales (α = .91). It was the most sensitive to differences in depression for all but the most depressed patients, for whom the CDRS-SR₁₇ was the most sensitive.

Conclusions

All three measures performed satisfactorily, but there are clearly defined advantages to using the QIDS-SR₁₆, as, by its very design, it assesses the core symptoms of depression and does not require a clinician.

Introduction

There is a clearly defined need to screen patients’ depressive symptoms in private practice settings. In previous research, the 16 item self-report and clinician versions of the Quick Inventory of Depressive Symptomatology (QIDS-SR₁₆ and QIDS-C₁₆)^1–3 were compared to several other inventories in their ability to screen using various populations primarily created for research purposes. For example, Rush et al.⁴ compared these scales to the 17-item Hamilton Rating Scale for Depression^5,6 in the course of the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study.^7,8

One problem with studies like Rush et al. is that the patients were all part of one or another research protocol so additional assumptions must be made if one wishes to generalize to a typical clinic populations. This report presents results obtained with an outpatient clinic population. In particular, the goal is to compare the QIDS-SR₁₆, the 17 item Carroll Depression Rating Scale⁹ (CDRS-SR₁₇), and the thirteen depression items from the Symptom Check List-90¹⁰ (SCL-D₁₃). The CDRS-SR₁₇⁹ is a self-report version of the Hamilton Rating Scale for Depression.^5,6 Evidence for the comparability of the two instruments may be found in Nasr et al.¹¹

We had available diagnoses in this sample as to whether the patient was in a current major depressive episode (MDE) based upon the MiniSCID,¹² which could be used as a “gold standard” against which the former three scales could be compared.

Method

Evaluable Sample

Patients were obtained through a chart review of 224 individuals who made an initial outpatient screening visit at the Nasr Psychiatric Services clinic between 09/2006 and 11/2007. The study was conducted in accordance with international guidelines for good clinical practice and the Declaration of Helsinki and approved by the Institutional Review Board of St. Anthony Memorial Health Centers in Michigan City, Indiana. All patients provided written, informed consent for inclusion of their data in this study. The final sample consisted of 175 who had completed the MiniSCID, QIDS-SR₁₆, CDRS-SR₁₇, and SCL-D₁₃. Their composition is described below under “Sample Characteristics”..

Assessments

Every patient was asked to complete the MiniSCID¹² at a computer terminal in the office followed by paper-and-pencil completion of the SCL-D₉₀, CDRS-SR₁₇, and QIDS-SR₁₆ in the same time frame. At a later date, a face-to-face clinical interview was conducted with a board-certified psychiatrist with extensive training and great experience in structured interviews for depression (coauthor SJN). A DSM IV-R diagnosis was generated at this session for clinical and billing purposes based upon clinical judgment derived from the above data. Patients were given the papers together and allowed to do them in whatever order they preferred. One limitation of the study is that it is not possible to determine whether or not there was an order effect since there is no way to identify the order in which the tests were taken. Another limitation of the study was that the clinician who evaluated the patient had access to the test data, but this, in itself, did not imply that any one was relied on most heavily in the diagnostic process.

Test Scoring

All scoring was done in the conventional manner. In particular, the QIDS-SR₁₆ was scored by selecting the maximum (most pathological) response to four sleep items, four weight/appetite items, and two restlessness/agitation items, thus converting the sixteen individual items into four domains with five additional domains defined by a single item each for a total of nine. In contrast, all item scores for the CDRS-SR₁₇ and SCL-D₁₃ were based upon responses to individual items.

Statistical Methods

Analysis proceeded in the following steps.

Classical test theory (CTT) analysis generated the mean and item/total correlation (r_it) for each item or domain, coefficient α, and scale means and standard deviations. Correlations among the three measures were also obtained.

Then, component analysis and parallel analyses^13–16 were used to define the number of dimensions on each scale. In a unidimensional scale, the first real principal component’s eigenvalue should exceed the first randomly generated principal component’s eigenvalue, but the reverse should be true of all subsequent eigenvalues.

The Samejima^17,18 item response theory (IRT) model for graded measures was applied. This involves estimating the parameters of a series of logistic ogives for each item or domain. The purpose here was to obtain the test information functions of each scale. This function describes the sensitivity to differences in the value of depression as a function of its magnitude. In effect, it is a reliability-like measure.

Three analyses examined the relation of the three measures to diagnose patients in an MDE vs. those not in an MDE based upon the MiniSCID. The first of these was simply the effect magnitudes or difference in mean score for currently depressed and currently not depressed groups divided by the pooled within-group standard deviation. This and the associated ANOVA main effect of group upon each of the scales employs linear criteria in assessing the ability of the scales to discriminate depressed vs. nondepressed groups in this study. They are also univariate evaluations since they ignore the covariances among the three measures. Second, univariate logistic regressions were conducted in which the ability of each scale to classify the two groups was determined separately for each scale. Associated with this was an receiver operating characteristic (ROC) analysis. ROC analyses are loglinear since they utilize the log odds ratios rather than probabilities themselves as criteria. Third, the contribution of each measure to the MANOVA discriminant axis was obtained. This is also linear, but it is multivariate since it assesses the ability of a given scale to increment the other two in classification, thus holding the latter constant. Finally, logistic regressions were conducted in which all three scales and pairs of scales were jointly entered as predictors, which assesses the ability of each scale to relate to the log odds of classification holding the two other scales constant. This is both loglinear and multivariate. The various methods are used because it is quite possible for one measure to be most successful by one criterion and other measures to be most successful by different criteria.

Finally, scores on the three scales were equated using methods previously described.^19–21 This allows users who are only familiar with one or two of the scales to place the unfamiliar one(s) in a familiar context.

Results

Sample Characteristics

The patients all came from a clinic located in a suburban part of the Metropolitan Chicago area. They were generally middle class. Most had private insurance (20% were on Medicare) and had some college education. Patients were accepted only upon referral from a primary care colleague or therapists who were well known to the practice. The database originally contained 224 records, of which 16 were excluded as they were younger than 18 years of age, and an additional 33 were excluded because they failed to answer one or more questions on the QIDS-SR₁₆, CDRS-SR₁₇, or SCL-D₁₃, the remaining 175 were included in the subsequent analyses. Those included obtained lower scores on the QIDS-SR₁₆ (9.7 vs. 11.9, p < .01), did not differ on the CDRS-SR₁₇, (14.4 vs. 14.4), and obtained lower scores on the SCL-D₁₃ (20.2 vs. 22.6, p < .05) from those 18 or over who were excluded. The patients’ mean age of those included was 44.1 years with a standard deviation of 14.9 years. Their age range was from 18 to 85 years. They did not differ in age from patients over 18 who were excluded (41.9 vs. 41.6). The sample was 41.7% male and 95% Caucasian. Based upon the MiniSCID, 82 patients were classified as clinically depressed, 75 patients were classified as not depressed, and data were missing on 18 patients.

CTT analysis

Tables 1–3 summarize the classical test theory analyses for the QIDS-SR₁₆, CDRS-SR₁₇, and the SCL-D₁₃, respectively. The SCL-D₁₃ was the most reliable (α = 0.90), followed by the CDRS-SR₁₇ (α = 0.84), and the QIDS-SR₁₆ (α = 0.81). The latter two differ minimally. The rank ordering of these values of α is consistent with the differences in scale standard deviation, as expected.

Table 1.

Domain means, item/total correlations, scale internal consistency (coefficient α), scale mean, and scale standard deviation for the QIDS-SR₁₆

Domain	Mean	rit
1. Sleep	2.19	.33
2. Sad Mood	1.42	.65
3. Appetite	1.37	.23
4. Concentration/Decision Making	1.14	.59
5. Self View	.89	.52
6. Thoughts of Death or Suicide	.42	.42
7. General Interest	1.07	.59
8. Energy Level	1.10	.63
9. Restlessness/Agitation	.97	.65
α	.81
Scale Mean	10.57
Scale SD	5.54

Table 3.

Domain means, item/total correlations, scale internal consistency (coefficient α), scale mean, and scale standard deviation for the SCL-D₁₃

Item	Mean	rit
1. Loss of sexual interest or pleasure	1.66	.37
2. Feeling low in energy or slowed down	2.22	.65
3. Thought of ending your life	.68	.58
4. Crying easily	1.46	.45
5. Feelings of being trapped or caught	.93	.52
6. Blaming yourself for things	1.85	.61
7. Feeling lonely	1.91	.69
8. Feeling blue	2.31	.79
9. Worrying too much about things	2.57	.60
10. Feeling no interest in things	1.83	.65
11. Feeling hopeless about the future	1.61	.69
12. Feeling everything is an effort	1.51	.68
13. Feelings of worthlessness	1.52	.73
α	.90
Scale Mean	22.05
Scale SD	12.08

Total scores on the QIDS-SR₁₆ correlated .86 with total scores on the CDRS-SR₁₇. This correlation is effectively 1.0 when corrected for unreliability (disattenuated).²² Similarly, scores on the QIDS-SR₁₆ correlated .68 with SCL-D₁₃ scores. Total scores on the CDRS-SR₁₇ correlated .63 with SCL-D₁₃ scores. These latter two correlations became .74 and .73 when disattenuated.

Scale Dimensionality

Figures 1a, 1b, and 1c respectively contain the scree plots for the QIDS-SR₁₆, CDRS-SR₁₇, and SCL-D₁₃. Using the previously cited parallel analysis criterion for imensionality, the QIDS-SR₁₆ and SCL-D₁₃ are unidimensional. However, the CDRS-SR₁₇ is not.

Figure 1.

Scree plots of the QIDS-SR₁₆ (top panel), the CDRS-SR₁₇ (middle panel), and the SCL-R₁₃ (bottom panel)

Test Information Functions

Figure 2 contains the test information functions for the three measures; these functions describe the ability of a scale to detect differences in the magnitude of the trait under study (depression in this case). The abscissa is in z-score units so 0 represents the sample mean, +1 is one standard deviation above this mean, etc. As can be seen, the SCL-D₁₃ is the most sensitive up to about +2 z-score units above the mean, which incorporates nearly all cases. This is consistent with the fact that its coefficient α was the largest of the three. The CDRS-SR₁₇ was the most sensitive to those most highly depressed. Note, however, that these are measures of reliability and not validity.

Figure 2.

Test information functions

Diagnostic Validity Based Upon Effect Sizes and ANOVA

The effect sizes were defined as the mean difference between the depressed and nondepressed groups divided by their pooled standard deviation. The respective values for the QIDS-SR₁₆, CDRS-SR₁₇, and SCL-D₁₃ were 1.37, 1.21, and .95 thus suggesting the QIDS-SR₁₆ is the most discriminating of depressed vs. not depressed groups. ANOVAs conducted in conjunction with these effect sizes indicated that all three differences were significantly greater than zero, F(1,166) = 73.69 for the QIDS-SR₁₆, 57.07 for the CDRS-SR₁₇, and 35.46 for the SCL-D₁₃, all ps < .0001.

Diagnostic Validity Based Upon Univariate Logistic Regression and ROC Analysis

A logistic regression with only the intercept entered provided a residual chi-square of 217.34 on 167 df. Individually, the QIDS-SR₁₆, CDRS-SR₁₇, and SCL-D₁₃ reduced this residual by 58.98, 48.17, and 31.90. These decreases are significant on 1 df, ps < .001. Thus, the QIDS-SR₁₆ again was the most discriminating of the three based on the MiniSCID diagnoses. The SCL-D₁₃ was the least discriminating.

Figure 3 contains the ROC curve derived from these three analyses. As can be seen, the QIDS-SR₁₆ is most sensitive from a false alarm rate of .0 (specificity of 1.0) up (down) to a false alarm rate of .2 (specificity of .8) past which point the three curves converge. Conversely, the SCL-D₁₃ is the least sensitive to this point.

Figure 3.

ROC curves

Table 4 contains thresholds, sensitivities, and specificities for selected points along each measure’s continua. These points are low (the lowest score for which the sensitivity was at least 30%), medium (the lowest score for which the sensitivity was at least 50%), high (the lowest score for which the sensitivity was at least 70%) and very high (the lowest score for which the sensitivity was at least 90%). The areas under these curves (c-statistic) were .814, .800, and .744 for the QIDS-SR₁₆, CDRS-SR₁₇, and SCL-D₁₃, respectively.

Table 4.

Threshold scores (Thresh.), Sensitivities (Sens.), and Specificities (Spec.) at Four Levels of Severity for the QIDS-SR₁₆, CDRS-SR₁₇, and SCL-D₁₃

	QIDS-SR16			CDRS-SR17			SCL-D13
Level	Thresh.	Sens.	Spec.	Thresh.	Sens.	Spec.	Thresh.	Sens.	Spec.
Low	11	30	13	15	32	21	20	30	34
Medium	14	52	3	20	52	8	27	50	11
High	16	73	1	24	74	3	34	73	6
Very High	19	90	1	31	92	1	40	90	4

Note: Low is the first level of each test for which the sensitivity was ≥30%; Medium is the first level of each test for which the sensitivity was ≥50%, High is the first level of each test for which the sensitivity was ≥70%, and Very High is the first level of each test for which the sensitivity was 90% or higher.

Diagnostic Validity Based Upon MANOVA

A MANOVA utilizing all three scales produce a Wilkes’ Λ of 25.31 on 3 and 153 df, p < .001. Of greater importance is the discriminant axis, whose weights were .013, .002, and .001 for the QIDS-SR₁₆, CDRS-SR₁₇, and SCL-D₁₃, respectively. This is also consistent with the fact that the QIDS-SR₁₆ increments prediction the most while controlling for the other two scales, whereas the SCL-D₁₃ increments the least.

Diagnostic Validity Based upon Multivariate Logistic Regression

When the three scales were all combined in a series of logistic regression equations, the QIDS-SR₁₆ significantly incremented the joint contribution of the CDRS-SR₁₇ and the SCL-D₁₃, χ²(1) = 8.41, p < .01. However, the CDRS-SR₁₇ did not increment the joint contribution of the QIDS-SR₁₆ and SCL-D₁₃, and the SCL-D₁₃ did not increment the joint contribution of the QIDS-SR₁₆ and CDRS-SR₁₇, χ²(1) = 1.03 and < 1, ns. The QIDS-SR₁₆ incremented the CDRS-SR₁₇ and SCL-D₁₃ scales individually, χ²(1) = 12.05 and 27.97, ps < .001, the CDRS-SR₁₇ incremented the SCL-D₁₃, χ²(1) = 20.59, p < .001, and the SCL-D₁₃ incremented the CDRS-SR₁₇. χ²(1) = 4.33, p < .05.

Equated Scale Scores

Table 5 shows how the scores for the three scales may be equated, using thresholds of non, mild, moderate, severe, and very severe defined using the QIDS-SR₁₆ total score (≤5, 6–10, 11–15, 16–20, 21+).

Table 5.

Equated Scale Scores on the QIDS-SR₁₆, CDRS-SR₁₇, and SCL-D₁₃

Θ	QIDS-SR16	CDRS-SR17	SCL-D13
−2.0	0		1
−1.3	3	3	6
−.9	5	6	10
.0	10	15	22
.4	13	19–20	28
.7	15	24	32–33
1.0	17	28	37
1.1	18	29–30	38
1.4	20	33–34	42
1.9	23	39	47
2.8	27	47	52

Discussion

The main finding is that although the SCL-D₁₃ was the most reliable in both the CTT and IRT senses, it was the least valid. Conversely, the QIDS-SR₁₆ was the most valid based on four different analyses (effect size/ANOVA, univariate logistic regression/ROC analysis, MANOVA, and multivariate logistic regression) - albeit only slightly more so. The multidimensionality of the CDRS-SR₁₇ might be an advantage as it is potentially sensitive to both the signs and symptoms of depression, thus affording it two opportunities to detect depression. In contrast, the unidimensionality of the QIDS-SR₁₆ suggests that it is limited to the symptoms of depression. It is not surprising that the CDRS-SR₁₇ was multidimensional, as we have previously observed that the Hamilton Rating Scale for Depression upon which it is based, is also multidimensional.^4,19 However, the two scales basically correlate perfectly within the limits imposed by their unreliability. In contrast, both are somewhat different from the SCL-D₁₃. As one would expect from the fact that there is substantial similarity among the three sets of items, these differences tend to be small.

The QIDS-SR₁₆ and CDRS-SR₁₇ item means are also fairly consistent with values we have reported for other samples.^4,19 The QIDS-SR₁₆ is a very user friendly tool for busy practitioners. It can be scored in less than a minute and can give a rapid view of the core symptom domains of depression. It has also clearly been shown to be unidimensional in various settings (e.g., Bernstein et al.,²³ Carmody et al..²⁰ Rush et al.⁴). By its very design, its items clearly reflect the core DSM criteria for depression. The CDRS-SR₁₇ is a self-rated version used in this practice and only takes about 2 minutes to score by an experienced assistant. However, It does not relate as clearly to these core symptom domains. The SCL-D₁₃ takes longer to complete for both the patient and the scorer. The scores can be plotted on a graph which can visually indicate the areas of elevation. It too does not relate clearly to the core symptom domains.

Limitations in this report are noteworthy. The diagnostic evaluation was not completely blind to the test results. The population is small and representative of a single practice. On the other hand, diagnoses were based on a structured interview, and the patients were typical of psychiatric practices. Finally the order of test administration was not randomized.

In summary, given the slightly greater validity of the QIDS-SR₁₆ compared to the CDRS-SR₁₇ and the SCL-D₁₃ by all criteria, it should become the tool of choice in private practice settings although all three measures are acceptable.

Table 2.

Domain means, item/total correlations, scale internal consistency (coefficient α), scale mean, and scale standard deviation for the CDRS-SR₁₇

Item	Mean	rit
1. Depressed mood	1.21	.69
2. Guilt Mood	1.11	.48
3. Suicidal feelings	.35	.41
4. Initial insomnia	.91	.37
5. Middle insomnia	1.12	.37
6. Delayed insomnia	.79	.24
7. Work and interests	1.50	.66
8. Retardation	1.42	.63
9. Agitation	1.18	.54
10. Psychic anxiety	1.41	.67
11. Somatic anxiety	.85	.40
12. GI symptoms	.42	.51
13. Somatic symptoms	1.18	.51
14. Libido	.67	.38
15. Hypochondriasis	.82	.41
16. Loss of insight	.54	.11
17. Loss of weight	.22	.00
α	.84
Scale Mean	15.70
Scale SD	8.69