Application of a stratum-specific likelihood ratio analysis in a screen for depression among a community-dwelling population in Japan

Abstract

Background

Efficient screening for depression is important in community mental health. In this study, we applied a stratum-specific likelihood ratio (SSLR) analysis, which is independent of the prevalence of the target disease, to screen for depression among community-dwelling individuals.

Method

The Center for Epidemiologic Studies Depression Scale (CES-D) and the Mini International Neuropsychiatric Interview (MINI) were administered to 789 individuals (19–87 years of age) who participated in the Iwaki Health Promotion Project 2011. Major depressive disorder (MDD) was assessed using the MINI.

Results

For MDD, the SSLRs were 0.13 (95% CI 0.04–0.40), 3.68 (95% CI 1.37–9.89), and 24.77 (95% CI 14.97–40.98) for CES–D scores of 0–16, 17–20, and above 21, respectively.

Conclusion

The validity of the CES-D is confirmed, and SSLR analysis is recommended for its practical value for the detection of individuals with the risk of MDD in the Japanese community.

Introduction

Major depression is a serious, recurrent mental disorder with high lifetime prevalence worldwide.¹ Major depressive disorder (MDD) constitutes a crucial public health burden. In Japan, MDD has a lifetime prevalence of 6.2% and a 12-month prevalence of 2.2%, is more prevalent among females than among males.² The World Health Organization has projected that MDD will become the number one leading cause of worldwide disability-adjusted life-years (DALYs) by 2020.³

More than 90% of suicide victims were retrospectively diagnosed with a psychiatric problem at their time of committing suicide, and approximately two-thirds of suicide victims were diagnosed with depression.^4,5 Prior studies have identified depression as the major risk factor for suicidal ideation, suicide attempts, and successful suicides.^6–8 Given the strong relationship between suicidal behavior and depression, screening for and treating depressive disorder have been proposed as one approach to prevent suicide.⁹

The Center for Epidemiologic Studies Depression Scale (CES-D) is a relatively simple and quick assessment that is inexpensive to administer and nonhazardous to patients; thus, this scale is acceptable for screening a large population.¹⁰ Shima et al¹¹ first reported the clinical validity of the CES-D Japanese translation and its standard cutoff point of 15/16. Subsequently, other studies from Japan have supported this finding and shown a good level of criterion-related validity and internal reliability in a sample of Japanese workers.^12,13 Although continuous CES-D scores are informative, interpreting screening results by utilizing score categories high predictive values with respect to the administration of efficient interventions is a convenient method.

Stratum-specific likelihood ratios (SSLRs) could be used to obtain such categories.¹⁴ The likelihood ratio (LR) provides a direct estimate of how much a test result will change the odds of having a disease and incorporates both the sensitivity and specificity of the test.¹⁵ SSLRs are calculated by dividing the continuous likelihood ratios into strata.¹⁶ However, reports regarding the use of SSLR analysis to screen for depressive disorders with the CES-D among community populations.^17,18

The objective of this study was to determine the CES-D score categories that have predictive clinical value for community-based screening for depression.

Method

Participants

The subjects were 789 volunteers (19–87 years of age, 289 males and 500 females) who participated in the Iwaki Health Promotion Project 2011.¹⁹ These individuals were residents of the Iwaki district, which is a rural area of the city of Hirosaki, in northern Japan. Iwaki is a stable community with a population of 11,863 individuals. The mean age of the participants was 57.8 years (with SD of 11.4 years and a range of 19–87 years). The data collection procedures for this study were approved by the ethics committee of the Hirosaki University School of Medicine, and all subjects provided written informed consent before participating in this project. Demographic data (age and gender) were obtained from self-reported questionnaires and interviews.

Procedure

The Japanese version of the CES-D was administered to all participants to measure depressive symptoms.^10,20 The CES-D is a 20-item, self-reported measure that focuses on depressive symptoms during the week prior to the administration of the questionnaire. The maximum score on this scale is 60, and a CES-D score of 16 or more is regarded as indicative of the presence of depression.

The Mini International Neuropsychiatric Interview (MINI) is a short, structured diagnostic interview for psychiatric disorders in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) and the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10).^21,22 In this study, we applied the portion of the MINI that is used to identify a major depressive episode. Among participants who responded “Yes” to A1 (depressed mood) and/or A2 (loss of interest), MDD was defined as a score of ≥5 on this portion of the major depressive episode section of the MINI.

Statistical analysis

The sensitivity, specificity, positive predictive value, and negative predictive value were calculated for several cutoff scores, as well as for the traditional cutoff score of 16 on the CES-D scale. Receiver operating characteristic (ROC) analysis was performed, with the true-positive rate (sensitivity) plotted on the vertical axis and the false-positive rate (1 – specificity) plotted on the horizontal axis; this approach allows display of all pairs of sensitivity and specificity values achievable as the threshold is changed from a low score to a high score.²³ The area under the curve (AUC) can be used as a quantitative indicator of the information content of a test. An AUC of 1.0 indicates perfect accuracy, whereas an AUC of 0.5 indicates a nondiscriminating test. The software used for the ROC analysis was Statistical Package for the Social Sciences (SPSS), version 24. The ROC curve is also used to determine the score that maximizes a screening test’s efficacy. The point on the ROC curve with the shortest distance to the intersection of the sensitivity and 1 – specificity values on the ROC graph is defined to be the optimal cutoff score.

We computed the SSLRs and 95% CIs using a program developed by Peirce and Cornell.²⁴ An SSLR indicates how much more likely or less likely a specific test result is for individuals with a disease than for individuals without this disease; thus, this ratio could reveal the efficiency of a screening test. LRs >10 and <0.1 indicate strong evidence for diagnosis and exclusion of diseases in clinical practice, respectively.¹⁴ To achieve the optimum number of strata, we followed the rules proposed by previous studies^24,25 as follows: 1) provide sufficient disordered and nondisordered subjects in each stratum to allow the SSLRs to be monotonically related; and 2) collapse strata where the SSLRs are close to one another and their 95% CIs easily overlap.

Logistic regression models were used to test whether the strata provided significantly more information than a single cutoff point.²⁶ First, a logistic regression model with a dichotomous predictor was fitted. Then, a second model with the same dichotomous predictor and the stratum as the categorical predictor was fitted. The difference between these two values was analyzed using the chi-square statistic under the null hypothesis that the strata predictor did not add more predictive ability than the single cutoff point.

Results

The overall scale was found to be reliable (alpha =0.77), and 12% (95/789) of the participants had a CES-D score of at least 16.

The sensitivities, specificities, positive predictive values, and negative predictive values are presented in Table 1. The ROC curve for the CES-D is depicted in Figure 1. The cutoff value, which was determined based on the shortest distance between any point on the ROC curve and the upper left intersection of the sensitivity and 1 – specificity values on the ROC graph, was 16. The AUC calculated using the ROC analysis was 0.98 (95% CI 0.96–1.00; p<0.001).

Table 1.

Sensitivity, specificity, and positive and negative predictive values of the CES-D

Cutoff point	Sensitivity	Specificity	Positive predictive value	Negative predictive value
13/14	1.000	0.813	0.106 (17/161)	1.000 (628/628)
14/15	0.941	0.863	0.131 (16/122)	0.999 (666/667)
15/16	0.941	0.898	0.168 (16/95)	0.999 (663/694)
16/17	0.882	0.924	0.203 (15/74)	0.997 (713/715)
17/18	0.882	0.946	0.263 (15/57)	0.997 (730/732)
18/19	0.824	0.953	0.280 (14/50)	0.996 (736/739)
19/20	0.824	0.968	0.359 (14/39)	0.996 (747/750)
20/21	0.706	0.972	0.353 (12/34)	0.993 (750/755)
21/22	0.706	0.978	0.414 (12/29)	0.993 (755/760)
22/23	0.647	0.982	0.440 (11/25)	0.992 (758/764)
23/24	0.647	0.987	0.524 (11/21)	0.992 (762/768)
24/25	0.588	0.990	0.556 (10/18)	0.991 (764/771)
25/26	0.529	0.991	0.563 (9/16)	0.990 (765/773)
26/27	0.471	0.994	0.615 (8/13)	0.988 (767/776)
27/28	0.412	0.995	0.636 (7/11)	0.987 (768/778)
28/29	0.412	0.995	0.636 (7/11)	0.987 (768/778)
29/30	0.412	0.997	0.700 (7/10)	0.987 (769/779)
30/31	0.412	0.997	0.778 (7/9)	0.987 (770/780)
31/32	0.412	0.999	0.875 (7/8)	0.987 (771/781)
32/33	0.353	0.999	0.857 (6/7)	0.986 (771/782)
33/34	0.176	1.000	1.000 (3/3)	0.982 (772/786)

Abbreviation: CES-D, Center for Epidemiologic Studies Depression Scale.

Figure 1.

ROC curve for depressive disorder determined using the MINI and the CES-D.

Abbreviations: CES-D, Center for Epidemiologic Studies Depression Scale; MINI, Mini International Neuropsychiatric Interview; ROC, receiver operating characteristic.

The SSLR analysis results are indicated in Table 2. The recommended SSLRs determined for MDD were 0.13 (95% CI 0.04–0.40), 3.68 (95% CI 1.37–9.89), and 24.77 (95% CI 14.97–40.98) for CES–D scores of 0–16, 17–20, and 21–60, respectively. Given the prior probability of the base rate of MDD used in the present study (2.1%), CES-D scores falling in the range of 0–16 had an SSLR <1.0 and thus shifted the posttest probability of having that disorder to a very low level (0.29%).

Table 2.

SSLRs obtained for MDD in this study

CES-D score	MDD, n	Non-MDD, n	SSLR	95% CI	Posttest probability, %
0–16	2	713	0.13	(0.04–0.40)	0.29
17–20	3	37	3.68	(1.37–9.89)	7.50
21–60	12	22	24.77	(14.97–40.98)	35.29

Abbreviations: CES-D, Center for Epidemiologic Studies Depression Scale; MDD, major depressive disorder; SSLR, stratum-specific likelihood ratio.

For MDD, the −2⋅log LR for the model with a single cutoff point (cutoff of 15/16) was 101.2. The −2⋅log LR for the model with the stratum as a categorical predictor was 93.0. The difference between these two values was 8.3. The corresponding p-value was small (p=0.004), indicating significant improvement when the stratum for MDD was used.

Discussion

Our results in which the CES-D had an AUC value of 0.98 demonstrated that this scale had high validity for the detection of MDD evaluated by the MINI in a community-dwelling population. Furthermore, a score of 16, which is traditionally used as the cutoff score, exhibited the highest total sensitivity and specificity (0.941 and 0.898, respectively). However, the positive predictive value (0.168) for this cutoff, which depends on the prevalence of the examined disease, indicates that many of the positive results obtained using this testing procedure are false positives. Given the prevalence of depression in prior community-based studies,^2,9 measures that are independent of the prevalence are needed for community-based screening.

In this study, we utilized the SSLR approach, which is independent of the target disease prevalence, and obtained three CES-D strata. By accounting for an individual’s prior probability, this approach could provide a basis to calculate the posttest probability in each specific stratum. Our results demonstrated that higher CES-D scores were associated with higher risks of MDD. Individuals with a CES-D score of 21 or greater had an SSLR that was significantly >10. Relative to the other categories, these score categories will provide primary care physicians and psychiatrists with greater opportunities to detect individuals with MDD via further investigation following screening using the CES-D.

A prior study conducted in first-visit psychiatric patients aged 36.9±16.0 (mean ± SD) years classified CES-D scores into three strata of <29, 30–49, and >50; the SSLRs for these three strata were 0.35 (95% CI 0.25–0.49), 2.3 (95% CI 1.8–3.1), and 11.7 (95% CI 3.1–44.0), respectively.²⁵ Another study conducted in Japan identified three strata <16, 17–19, and >20 for working individuals aged 42.0±11.4 (mean ± SD) years; the SSLRs for these three strata were 0.06 (95% CI 0.02–0.18), 1.9 (95% CI 1.78–4.62), and 12.4 (95% CI 10.2–15.1), respectively.¹⁷ However, an imbalanced gender ratio in that study (males: n=1,868 and females: n=351) may have influenced the results. Among Taiwanese adolescents aged 12–16 years, Yang et al¹⁸ showed three strata of <28, 29–48, and >49. The SSLRs for these three strata were 0.63 (95% CI 0.42–0.92), 4.0 (95% CI 2.4–6.8), and 11.8 (95% CI 3.4–50.9), respectively. These studies differed in the mean age, gender ratio, and setting (clinical or community), which might have affected the depressive symptomatology. Furthermore, differences concerning the assessment of depressive disorder (DSM-III-R or Kiddie Schedule for Affective Disorders and Schizophrenia [K-SADS]) could affect the CES-D score categories. Although differences among the score categories have been identified to date, understanding the existence of a dose-dependent relationship between the CES-D score and the risk of MDD. Compared with previous studies, our score categories of the CES-D might be useful for screening the older population in a community for depression.^17,18,25

We must consider the interpretation of the false-positive results obtained when screening using the CES-D. Prior studies have found poor positive predictive power associated with using the CES-D to screen for depression.^27,28 False positives might be induced by active medical conditions, alcohol-related disorders, and/or dysthymia.²⁹ Although individuals with false-positive results do not satisfy the criteria for MDD, they could benefit from follow-up with primary care physicians or psychiatrists. We should also consider false-negative cases. Individuals who were afraid of being diagnosed with MDD or who were unaware of their illness might not express their symptoms during the screening process.³⁰ An objective biomarker for diagnosing MDD is needed to identify these individuals.

Although screening for depression is important, it is also necessary to provide consistent treatment and follow-up. Merely identifying individuals with depression would not be beneficial. Furthermore, recent evidence has raised questions regarding the degree to which standard treatments for depression benefit patients who are identified via screening.³¹ A meta-analysis reported that the benefits of antidepressant medication compared with those of a placebo were minimal or nonexistent in patients with mild or moderate depressive symptoms.^32,33 Therefore, a combination of prompt detection and treatment appropriate for the symptom severity is needed for effective community-based screening for depression.

This study has certain limitations. First, all of the participants were volunteers who were interested in their health and may therefore have been healthier than the general population. Thus, the community members who were not included in the study may have experienced different depressive symptoms than the study participants. This type of “selection bias” must be considered in studies of community populations. Second, our participants may not be representative of all Japanese community populations because our study was conducted only in a rural district. Even if the prevalence of depression does not differ across communities, certain risk factors could differ among different populations.³⁴

Conclusion

The CES-D exhibited good validity, and a score of 16 is the optimal cutoff score for assessment of MDD in a community population. Moreover, the use of SSLRs could be a convenient and intuitive measure for understanding the results of community-based screening for depression. Programs combining sequential screening for depression and feedback with adequate support are recommended.