Assessing depression in a geriatric cancer population

Abstract

Objective

To examine the ability of three popular self-report measures of depression to assess depression in a geriatric cancer setting.

Method

Cancer patients 70 years or older and on active treatment completed the Geriatric Depression Scale-Short Form, the Hospital Anxiety and Depression Scale, and the Center for Epidemiological Studies Depression Scale—Revised, and were interviewed using the depression module of the Structured Clinical Interview for DSM disorders (SCID) as the ‘gold standard.’ Analyses included calculating internal consistency, ROC curves, and the sensitivity and specificity to detect major depression (MDD) or minor depression (i.e. subthreshold depression).

Results

In a sample of 201 cancer patients (85% White; 64% completed college degree or higher), all three of the self-report measures produced adequate internal consistency and predicted depression greater than chance. However, the published cutoff scores for detecting MDD produced inadequate sensitivity, suggesting these scores will miss as many as 33%–83% of geriatric cancer patients who are depressed. Revised cutoff scores were lower than published cutoff scores.

Conclusion

Although these measures produced good internal consistency and were better than chance at predicting depression in a geriatric cancer sample, the published cutoff scores for these measures did not perform well in predicting MDD nor minor depression. Of the three measures, the CES-D appeared to have the most utility. This data suggests that these popular screening measures may be inadequate for reliably identifying depression in a geriatric cancer population. Researchers and clinicians, therefore, should use caution when selecting depression measures for geriatric cancer patients and consider using the lower cut-off scores presented here.

Introduction

Depression is one of the most frequent causes of emotional distress in older adults [1,2]. Estimates of significant depressive symptoms in community samples of elderly adults range from 8% to 16% [3–5], and these rates jump to as high as 23% when estimating mild depressive symptoms [6,7]. Moreover, two out of every three older Americans have multiple chronic medical conditions [8], and the prevalence of cancer in older adults is expected to double in the next 20 years worldwide [9]. For those with cancer, the prevalence of depression currently ranges from 3 to 31% [10–12]. The wide prevalence range, however, reflects a diversity of measurement strategies, cancer types, ages, and treatment statuses, and therefore, a reliable point prevalence for depression in older cancer patients remains to be determined. Depressive symptoms, whether mild or severe, are associated with greater impairment in illness management and adjustment. Additionally, depressive symptoms are associated with increasing costs of care [13], as depressed individuals are 2 to 3 times more likely to access medical services than those who are not depressed [14–16]. These issues underscore the need for proper screening, assessment, and treatment of depressed older adults with cancer [2,13].

Unfortunately, depression in the elderly is often under-recognized and under-treated due in part to the tendency for older adults to report depressive symptoms differently than younger adults [2,17]. In cancer patients, diagnosing depression can also prove to be challenging [18]. The primary source of this difficulty lies in the overlap between the diagnostic criteria for depression, as detailed in the Diagnostic and Statistical Manual for Mental Disorders (DSM [19]), and the symptoms often attributable to cancer and/or the side effects of treatment [20–22]. The overlapping symptoms may include: sleep disturbances, decreased interest in sexual activity, and lack of energy [23–25]. These same symptoms may arise from depression, from the cancer itself, from treatment side effects, or from some combination of the three [26,27]. Thus, identifying depression in older patients with cancer combines both the complexity of diagnosing depression in cancer patients with the difficulty of identifying depression in older adults [18].

Given these intricacies, commonly used measures of depression may not accurately assess depressive symptoms in older cancer patients. For example, the Geriatric Depression Scale (GDS [23]), the most commonly used depression measure among older adults, has a number of questions that may be either inappropriate or inaccurate assessments of depression in older cancer patients. Older adults with cancer may struggle with how to interpret questions such as ‘Are you afraid that something bad is going to happen to you?’, ‘Do you feel your situation is hopeless?’, and ‘Do you feel full of energy?’ Because of their cancer, they may accurately think that ‘something bad’ will happen to them, they may struggle with how to interpret ‘hopeless’ in their specific context, and the symptoms of the disease or side effects of treatment may trigger a decrease in their energy level.

There is a paucity of literature that provides empirical evidence for the validity or appropriateness of the GDS and other common depression measures in a geriatric cancer setting. In one review, Nelson and colleagues [28] examined eight commonly used depression measures to assess their utility with older cancer patients. A symptom profile analysis was conducted to determine if the measures addressed symptoms specific to older adults with depression including general aches and pains, diffuse somatic complaints, late insomnia, general malaise, hopelessness, mood variation, and change in sexual interest. None of the eight measures were deemed to adequately address these constructs. Thus, the authors concluded that further validation and measure development with older cancer patient samples are needed. Additional reviews of depression measurement in both geriatrics and psycho-oncology have also concluded that more research is needed in order to specifically evaluate the psychometric properties of measures of depression in older cancer patients [12,29].

The goal of this current study is to test the sensitivity and specificity of three common depression measures to detect depression in older cancer patients. We also set out to develop cut-off scores for these measures that are specific to a geriatric cancer population. This study provides some of the first validation data of depression measures among older cancer patients.

Method

Participants

Study participants were recruited at Memorial Sloan Kettering Cancer Center (MSKCC) in New York City. To participate, patients needed to be age 70 and older, have a diagnosis of prostate or breast cancer, receiving treatment with chemotherapy, radiation or hormone therapy, able to provide informed consent, and English-speaking.¹ The study was approved by the MSKCC Institutional Review Board, and all participants provided informed consent before beginning the study.

Procedure

Research assessments took place in a single session in the outpatient departments at MSKCC. Research staff were trained research assistants who were blinded to participants' questionnaire responses before administering the SCID. Participants were first asked to complete a sociodemographic questionnaire and three widely used measures of depression: Geriatric Depression Scale Short Form (GDS-SF), Hospital Anxiety and Depression Scale (HADS), and Center for Epidemiologic Studies of Depression (CESD-R). Next, participants were interviewed by the same research staff using the Structured Clinical Interview (SCID) of the Diagnostic and Statistical Manual-IV (DSM-IV) to identify both major and minor depression (operationalized as the presence of at least one gateway symptom and two additional depressive symptoms). This was used as the ‘gold standard’ for depression and the criteria used to test the sensitivity and specificity of the self-report measures.

Measures

Sociodemographic questionnaire

Sociodemographic information was obtained for all patients regarding age, ethnicity, education, employment history, marital status, and household composition.

Depression measures

Geriatric Depression Scale-Short Form (GDS-SF)

This is the most commonly used measure to assess depression in a geriatric population [23]. It was specifically created to address the need to focus on the unique aspects of geriatric depression. The current study utilized the 15-item short version of the measure in which items are rated 0 (no; symptom absent) or 1 (yes; symptom present). The authors have also created a 15-item short version of the measure, which was used in the current study. The cutoff score that indicates the presence of depression is 5. This has demonstrated good internal consistency with a Cronbach's α = .94, and good concurrent validity with the (.84) Zung Self-Rating Depression Scale and (.83) the Hamilton Depression Rating scale [23].

Hospital Anxiety and Depression Scale (HADS)

This is a 14 item self-rated questionnaire, which has been well tested in cancer populations, with Depression and Anxiety subscales of seven items each (HADS-D and HADS-A, respectively) [30]. It was developed for use with patients with chronic disease because of the absence of somatic items that often confound the determination of psychiatric problems among the medically ill. For the current study, we evaluated the HADS-D subscale only, which is widely used for depression screening in oncology and other medical settings [31]. The recommended cut-off score for detecting depression is 8 [30].

Center for Epidemiologic Studies Depression Scale—Revised

The CESD-R assesses the frequency of depressive symptoms in the past week. The measure consists of 20 items that respondents rate on a 5-point scale, i.e. 1 ‘not at all, or less than 1 day last week,’ 2 ‘1 or 2 days last week,’ 3 ‘3 to 4 days last week,’ 4 ‘5 to 7 days last week,’ and 5 ‘Nearly every day for 2 weeks.’ Scores range from 0 to 80 (higher scores indicate greater depressive symptoms) [32]. The traditional cut-off point used to categorize respondents as depressed is a score of 16 or higher. The measure has demonstrated good internal consistency, with coefficient alpha of .92 or above, and has demonstrated high reliability and validity [32].

Structured Clinical Interview for DSM-IV-TR (SCID)

The SCID is a structured psychiatric interview administered by a trained clinical interviewer that yields diagnoses based on algorithms and criteria of the revised fourth edition of the DSM (DSM-IV-TR) [33]. The SCID Depression module was used to establish the presence or absence of major and minor depressive disorder. Participants are asked about the presence of depressive symptoms during the same two-week period over the past month. Major depression is defined as one or both of the core symptoms (depressed mood and anhedonia) and four or more of the other defined symptoms of depression (i.e. significant weight changes or changes in appetite, sleep disturbances, psychomotor agitation or retardation, fatigue or loss of energy, guilt or worthlessness, diminished concentration, and suicidal ideation). Minor depression is defined as the presence of one of the core symptoms of depression and the addition of two to three of the other symptoms of depression (i.e. DSM-IV-TR definition).

Statistical analysis

The goal of the analyses was to calculate the sensitivity and specificity for each measure, and to highlight appropriate cut-off scores for both MDD and minor depression (i.e. indicating subthreshold depression as distinct from MDD). For preliminary analyses, we calculated Cronbach's alpha coefficient as an indicator of internal consistency for each measure. Next, we calculated the predictive accuracy of the GDS-SF, HADS-D, and CESD-R using the SCID diagnosis of MDD or minor depression as the ‘gold standard’ assessment of depression. We generated Receiver Operating Characteristic (ROC) curves and examined Area Under the Curve (AUC) for each measure to determine if these measures predicted both MDD and minor depression greater than chance. We also used the ROC analyses to calculate sensitivity and specificity for the three questionnaires separately for both MDD and minor depression. The sensitivity and specificity provide a useful way to evaluate the performance of classification schemes that categorize cases into one of two groups (i.e. meet criteria vs. does not meet criteria). The sensitivity and specificity are reported for published cut-off scores for each measure: GDS-SF cutoff score of 5, HADS-D cut-off score of 8, and the CESD-R cut-off score of 16. These are cut-off scores cited in the literature for indicating the presence of depressive symptoms in the general geriatric population. We examined whether or not these cut-off scores are comparable to the optimal cut-off scores in the oncologic geriatric population. Appropriate cut-off scores were selected that produced the optimal combination of sensitivity and specificity. In the absence of clearly established empirical guidelines for determining optimal cut-off scores for screening measures in mental health, we decided on a two-step process a priori. In the first step, we outlined acceptable levels of sensitivity and specificity for the purpose of screening for depression in older cancer patients [34]. We set the acceptable level of sensitivity for a depression screening instrument for our sample at 80%. Ideally, specificity will also be 80% or higher but we were willing to accept some decrements in specificity in order to maintain higher sensitivity. Thus, we aimed to select cut-off scores that achieve at least 70% specificity. If these parameters were not met, we determined the ‘optimal’ cut-off score by calculating the Youden Index (J) (i.e. J = maximum [Sensitivity + Specificity – 1]), which is recognized as one of the more reliable ways of determining an optimal cut-off score compared to visual inspection of ROC curves [35,36]. We also decided to make conservative estimates and therefore did not include cut-off scores calculated as having 100% sensitivity or specificity. Confidence intervals (95%, two-tailed) of the sensitivity, specificity, and the AUC statistics were also generated to describe the uncertainty associated with these estimates. We also calculated the positive likelihood ratios (LR+) for our chosen cut-offs as an additional indicator of each measure's ability to accurately predict the presence of depression, with higher values suggesting better concurrent validity. If the optimal cut-off score was different than what has been cited in previous research, we recommended a cut-off score for each depression measure.

Results

Participant characteristics

Participants were recruited between December 2006 and October 2008. The sample included 102 women with breast cancer (70 early stage and 32 late stage) and 99 men with prostate cancer (24 early stage and 75 late stage) for a total sample of 201 participants. Early stage disease was defined as those patients without distant metastatic disease, and late stage was defined as those with distant metastatic disease. Mean age of the sample was 77 (SD = 4.84), ranging from 68 to 90 years old. The majority of the sample was White (85%), 13% identified as Black or African American; One participant identified as Hispanic. Roughly half of the sample was married or living with a partner (55%), while 27% were widowed, and the remainder were single or divorced (18%). The sample was highly educated with 64% having completed college or graduate degrees.

Depressive symptom endorsement

According to SCID, 7% of the sample met criteria for minor depression (n = 14) and 3% met criteria for MDD (n = 6). Of those who met criteria for MDD based on the SCID, all endorsed depressed mood and fatigue. For the remaining six items, endorsement frequencies varied. Half of the sample endorsed anhedonia, insomnia or hypersomnia, psycho-motor agitation or retardation, and worthlessness or guilt.

Cronbach's alpha and AUC

Overall, all three of the self-report measures had acceptable psychometric properties as suggested by indicators of reliability and validity. Each measure had adequate internal consistency as indicated by Cronbach's alpha ranging from 0.75 to 0.84, not improved by deletion of any single item (Table 1). Additionally, all scales predicted MDD and minor depression greater than chance.

Table 1. Scale properties and typical cut-offs for detecting MDD.

Measure	Cronbach's alpha coefficient	AUC (95% CI)	Typical cutoff	Sensitivity (95% CI)	Specificity (95% CI)	LR+	% of sample identified as depressed
CESD-R	.84	.87 (.74–1)	16	.67 (29.93–92.51)	.89 (85.05–93.54)	6.00	14.1 (n = 28)
HADS-D	.75	.88 (.81–.95)	8	.17 (.42–64.12)	.93 (87.47–95.26)	2.26	9 (n = 18)
GDS-SF	.75	.88 (.80–.95)	5	.67 (22.28–95.67)	.88 (84.13–92.85)	5.51	14.9 (n = 30)

AUC for GDS-SF, HADS-D, and CESD-R ranged from 0.87 to 0.88 for detecting MDD and from .67 to .78 for detecting minor depression (p < 0.002; see Tables 1 and 3). There were notable differences, however, between the measures in terms of optimizing sensitivity and specificity for detecting major and minor depression. Data for each measure are described separately below.

Table 3. Typical cut-offs: detecting minor depression.

Measure	AUC (95% CI)	Typical cutoff	Sensitivity (95% CI)	Specificity (95% CI)	LR+	% of sample identified as depressed
CESD-R	.78 (.65–.91)	16	.50 (23.04–76.96)	.90 (84.98–93.64)	5.0	14.1 (n = 28)
HADS-D	.76 (.63–.90)	8	.43 (17.66–71.14)	.95 (89.70–96.85)	8.6	9 (n = 18)
GDS-SF	.67 (.52–.83)	5	.36 (12.76–64.86)	.88 (83.09–92.22)	3	14.9 (n = 30)

Sensitivity and specificity

GDS-SF

MDD

The published cut-off score of 5 for detecting MDD produced a sensitivity of .67 and specificity of .88 for the GDS-SF (Table 1). ROC curve analysis and Youden's Index indicated that the best cut-off score for detecting MDD in this sample was 4, yielding higher sensitivity but lower specificity (.83 and .78, respectively; see Table 2). The LR+ was small to moderate for the probability of MDD.

Table 2. Suggested cut-offs for detecting MDD.

Measure	Selected cut-off	Sensitivity (95% CI)	Specificity (95% CI)	LR+	% of sample identified as depressed
CESD-R	15	.83 (42.13–99.64)	.89 (84.59–93.19)	7.50	14.6 (n = 29)
HADS-D	6	.67 (22.28–95.67)	.85 (80.69–90.15)	4.36	17.9 (n = 36)
GDS-SF	4	.83 (35.88–99.58)	.78 (75.37–85.67)	3.77	24.9 (n = 50)

Minor depression

For detecting minor depression, the published cut-off score produced a low sensitivity (.36), while maintaining high specificity (.88). Similarly, the LR+ value of 3 indicates that the GDS-SF has a small effect on increasing the probability of the presence of minor depression. Only a cut-off score of 1 reached our predetermined levels of acceptable sensitivity or specificity (i.e. .93 and .73, respectively).

HADS-D

MDD

For the HADS-D, sensitivity was .17, and specificity was .93 for detecting MDD based on the recommended cut score of 8 (Table 1). ROC analyses and Youden's Index, however, revealed that a better cut-off score for use with our sample would be 6, with increases in sensitivity and a small decrease in specificity (.67 and .85, respectively; Table 2). The LR+ value for the HADS-D was small regardless of which cut-off score was used.

Minor depression

Minor depression was also unreliably detected using the cut-off score of 8 (i.e. sensitivity = .43, specificity = .95), with slight improvement in sensitivity if lowered to 4 (i.e. sensitivity = .71, specificity = .68; Tables 3 and 4). The LR+ value for the HADS-D decreased from large to small when using the cut-off score of 4, which is not surprising given the high specificity associated with the original cut-off score of 8.

Table 4. Suggested cut-offs for detecting minor depression.

Measure	Selected cut-off	Sensitivity (CI)	Specificity (CI)	LR+	% of sample identified as depressed
CESD-R	13	.57 (28.86–82.34)	.89 (84.50–93.28)	5.18	15.1 (n = 20)
HADS-D	4	.71 (41.90–91.61)	.68 (69.56–80.63)	2.23	27.9 (n = 71)
GDS-SF	N/A	N/A	N/A	N/A	N/A

CESD-R

MDD

Using the published cut-off score of 16 for detecting MDD, the CESD-R produced inadequate sensitivity but acceptable specificity (.67 and .89, respectively, Table 1). ROC curve analysis and calculation of Youden's Index suggested that the optimal cut-off score for detecting MDD was 15, producing an improved sensitivity of .83 and maintaining specificity of .89. For detecting minor depression, the published cut score of 16 again had inadequate sensitivity (.50) and high specificity (.90). The LR+ values suggested that the CES-D has a large effect on increasing the probability of MDD presence.

Minor depression

As with the GDS and HADS-D, there was no obvious optimal cut-off score based on the ROC curve analyses for detecting minor depression, as all options with acceptable levels of sensitivity had significant decrements in specificity and vice versa. We determined that a cut-off score of 13 produced maximum sensitivity and specificity according to Youden's Index (.57 and .89, respectively) for detecting minor depression with the CESD-R (Table 4). As with the LR+ values for MDD, the CES-D also produced large LR+ values for predicting the presence of minor depression.

Discussion

The three self-report measures examined in the current study produced adequate internal consistency and were all able to significantly predict depression in this sample of geriatric cancer patients using the SCID as the criterion measure. The published cut-off score for each of the measures produced strong specificity (Tables 1 and 3). However, these scores were associated with low levels of sensitivity. In fact, the published cut-off scores for these measures potentially missed anywhere from 33% to 83% of those patients reporting major or minor depression. These inadequacies are of significant concern, as undetected depression, both minor and major, are associated with significant psychological distress, functional impairment, and greater health care costs.

Given the poor performance of the existing cut-off scores, we examined each potential cut-off point for the three measures in order to optimize sensitivity and specificity. While there is no single ‘correct’ way for selecting the optimal cut-off score, we first applied a priori criteria based on clinical utility—we would accept 80% sensitivity and 70% specificity. If these conditions were not met, we calculated Youden's Index to approximate the optimal cut-off scores. Based on these calculations, we have suggested new cut-off scores with higher specificity and sensitivity which can better help detect MDD and minor depression.

MDD

For MDD, the optimal cut-off scores for each measure were 15 on the CESD-R, 6 on the HADS-D, and 4 on the GDS-SF (Table 2). Only the CESD-R met our strict criteria and achieved over 80% sensitivity and specificity for detecting MDD. The inadequate performance of these measures reflects further our contention that existing scales do not accurately capture the phenomenology of depression in older cancer patients.

Minor depression

Although we were generally able to identify better cut-off scores for detecting MDD, there still did not appear to be any potential cut-off scores on any of the three measures that produced adequate sensitivity and specificity for detecting minor depression. Across the board, our recommended cut-off scores are lower than the published cut-off scores, which reiterate that depression runs the risk of being undetected in routine screening. Our analyses suggest that for detecting minor depression, the optimal cut-off score is 13 for the CESD-R and 4 on the HADS-D (Table 4). For the GDS-SF, however, there was no score that we could reliably propose for detecting minor depression, as a score of 1 as suggested by our analyses is unacceptably low.

We also note that when making decisions about tradeoffs in sensitivity and specificity, one must consider the purpose of the depression assessment and the setting in which it will be implemented. For example, for routine screening in an oncology clinic, one might prefer higher sensitivity in order to flag an at risk individual for further evaluation, while a research team may prefer to achieve higher specificity in order to maximize diagnostic reliability. Both options have relative risks and benefits. For the purposes of the current study, we determined that the potential over-identification of patients as depressed by screening would be preferable the risk of under-detection. For example, the risk of suicide or the failure to maintain optimal physical health is a potential consequence of undetected depression in elderly cancer patients. However, we also recognize that the over-diagnosis of depression by being overly inclusive also runs the risk of placing excessive demands on limited mental health resources. Likewise, there are clear risks to prescribing antidepressants to non-depressed patients, and patients may react negatively to the suggestion that they are ‘depressed’ when indeed they are not. By improving the accuracy of screening measures such as the three explored in the current study, clinicians may feel more confident in differentiating true from false positives for depression in the context of a life-threatening illness such as cancer.

We were struck by the lack of guidance in the literature for determining optimal cut-off scores for continuous variables. Vickers and Sjoberg [34] noted that simple visual inspection of ROC curves is not sufficient for determining optimal cut-off scores, and that reporting sensitivity and specificity can even be too abstract at times. We applied a set of a priori rules, and highlight this gap in the measure development literature as a whole to provide sound guidance in selecting cut-off scores for psychosocial measures.

The present sample endorsed very few depressive symptoms. When they did endorse these symptoms, they tended to be mild. Specifically, 7% of our sample endorsed minor depression and only 3% endorsed major according to the SCID. We believe that this low endorsement underscores the shortcomings of existing measurement, specifically the ability of the SCID diagnostic criteria to accurately identify depression in older cancer patients. The estimated population prevalence of minor depression in older adults is roughly 2 to 5%, and MDD tends to be around 7% [37,38]. The fact that our sample of cancer patients reported lower prevalence of MDD than the general population seems to suggest that the SCID diagnostic criteria do not adequately capture the experiences of older adults with cancer.

A potential limitation of the study method is that SCID was completed after participants had filled out the three study questionnaires; thus, they may have been fatigued and not endorsed as many symptoms as they might have if the SCID was conducted first. Our sample was also relatively homogenous, characterized as highly educated and predominately White. Such a lack of diversity may have contributed to the low frequency of depressive symptoms endorsed in our sample [39]. Similarly, given the small number of MDD cases, we were not able to explore the potential role of gender in depressive symptom endorsement. Additionally, as noted, older adults tend to report fewer classic mood symptoms than younger, and may even have a more somatic depression presentation [17]. Therefore, use of the SCID as our criterion measure may have limited our ability to capture all participants who were experiencing clinically significant distress. Similarly, the CESD-R, which performed the ‘best,’ is the only one of the three measures that, like the SCID, include the somatic symptoms of depression. Therefore, the better performance of the CESD-R may reflect the concordance of its items with SCID items.

Taken together, our findings suggest that several popular screening measures are inadequate for identifying depression in a geriatric cancer population. Of the three measures examined, the CESD-R appeared to demonstrate best specificity and sensitivity for detecting major depression, but as discussed, this may reflect the circularity inherent in using the SCID as our criterion measure. None of these measures appeared satisfactory for detecting minor depression. Thus, no single measure stood out as ‘optimal.’ Researchers and clinicians, therefore, should use caution when selecting depression measures for geriatric cancer patients, as the risk of overlooking subthreshold depression is substantial. Future research should continue to develop alternate ways of assessing depression in a geriatric cancer population. This effort will likely require a multi-modal approach, such as including both qualitative and quantitative analyses to better understand the phenomenology of depression in older cancer patients (e.g. [40]).