A systematic review of depression and anxiety measures with individuals with spinal cord injury

Abstract

Study design

A systematic review.

Objectives

To review and assess the psychometric properties of depression and anxiety instruments used with spinal cord injury (SCI) populations.

Setting

Vancouver, Canada

Methods

Electronic databases were searched for papers reporting psychometric properties of depression and anxiety instruments. Pre-established criteria were used to assess the psychometric properties.

Results

Thirteen papers reporting on the psychometric properties of 13 depression and anxiety instruments are in this review, and include: BDI, BSI, CESD-20, CESD-10, DASS-21, GHQ-28, HADS, Ilfeld-PSI, MEDS, PHQ-9, PHQ-9-Short, SCL-90-R, and the Zung SRS. Reliability data is available for 10 instruments, and validity results are available for 12. Evidence spanned the spectrum of evaluation criteria varying from poor to excellent. Responsiveness data is generally lacking.

Conclusion

Given that the reliability and validity findings range for the most part from adequate to excellent, and the large amount of work to develop cutoff scores specific for SCI populations, there is at present no need to develop SCI specific instruments. Because one measure’s psychometric properties do not clearly stand out, it is difficult to recommend the use of one over another. Overall, more psychometric data is needed, and if the instruments are to be used to evaluate treatment outcomes or change over time, responsiveness data is also required. Administering the instruments in tandem with each other and with clinical diagnostic interviews would provide valuable information, as would comparison of results to normative data specific to persons with SCI.

Introduction

Depression and anxiety disorders and/or symptoms are commonly reported after SCI. Despite a conceptual distinction between depression and anxiety, clinically differentiating the two constructs has proven difficult, as people who experience anxiety are often depressed as well.^1,2 In a sample of 394 primary care patients, Mergl et al (2007) found that depression without comorbidity occurred significantly less than expected by chance. ² Further, a high comorbidity odds ratio (6.25) between depressive and anxiety disorders was found, leading to the conclusion that depression and anxiety comorbidity occurs more often than expected. ² For this reason, it is important to assess both depression and anxiety, disorders or symptoms, in tandem.

Diagnosis of depression and anxiety disorders are typically conducted via structured interviews based on the Diagnostic and Statistical Manual of Mental Disorders, 4^th Edition (DSM-IV).³ A diagnosis of a depressive disorder as per the DSM-IV requires a minimum number of symptoms presenting for at least two weeks. Major Depressive Disorder (MDD), for example, is diagnosed based on the presence of a depressed mood and/or a loss of interest, in addition to three or four of: significant weight loss, insomnia or hypersomnia, psychomotor agitation or retardation, fatigue, feelings of worthlessness, lack of ability to think or concentrate, and recurrent thoughts of death or suicide.³ Similarly, anxiety disorders are diagnosed based on a constellation of symptoms, with a common feature of inappropriate anxiety. Anxiety disorders may include symptoms such as: increased heart rate, tensed muscles, fear of dying, inability to relax, irritability, and trouble concentrating. To be diagnosed with either a depression or anxiety disorder, the symptoms must be independent of other medical conditions.³

Because the DSM-IV diagnostic process is time consuming for both clinicians and patients, self-report instruments are frequently utilized as devices to identify the possible presence of a depression or anxiety disorder or to assess the severity of symptoms. Importantly, such instruments serve to alert the professional to the need for further clinical evaluation or treatment. Accordingly, both clinicians and researchers are advantaged by instruments that can offer an efficient and effective means to screen for depression and anxiety disorders or determine levels of symptom severity.

Depression and Anxiety Measurement Issues in SCI

While many self-reporting instruments are available, the application of generic instruments with SCI populations is not without concern. Use of generic instruments that are neither reliable nor valid among SCI populations are a likely source of bias. As an example, in a review of 300 randomized controlled trials, Marshall et al. (2000) reported that treatment for schizophrenia was 36% more likely to be found effective if an unpublished scale was used.⁴ The authors also determined that one-third of the claims of non-drug treatment being preferred over the control would not have been made had published, reliable, and valid scales been used. Despite the findings being specific to schizophreina scales, it exemplifies the need for reliable and valid scales specific to certain populations, such as SCI.

Another issue with the use of generic instruments is that people with disabilities who are undergoing rehabilitation have different needs and problems from those of the general population. Without accounting for such differences, measurement problems will persist. For example, many measures include somatic symptoms of depression and anxiety disorders, however, symptoms of weight and energy loss, loss of appetite, and disruptions in sleep cycles are also commonly reported following SCI. Anxiety symptoms such as increased blood pressure, sweating and rapid heart rate are also present during episodes of autonomic dysreflexia among persons with SCI. Simply excluding somatic questions from generic instruments is problematic, potentially altering the properties of the questionnaire, and possibly removing true indicators of depression or anxiety disorders. Alternatively, if somatic criteria are maintained in the scales, there is the potential for these measures to overestimate the prevalence of depression and anxiety.

Finally, another concern with the current instruments in use revolves around the differing screening criteria and timing of assessments. Such differences have raised questions as to the definitions used to assess depression and anxiety, and the concern that not all instruments may be screening or measuring the same construct. Such concerns limit the comparison of results, and lead to variability in the estimates of depression and anxiety. Nonetheless, a recent review of psychological morbidity and SCI⁵ found that 20%⁶ to 43%⁷ of people with SCI are at risk of having a depressive disorder during rehabilitation, and 11%⁸ to 60%⁹ are at risk of having raised depressive symptoms when living in the community. The same review also established that anxiety disorders are more prevalent in SCI populations, with estimates ranging from 13%¹⁰ to 44%¹¹ as are higher levels of anxiety symptoms. Although the studies in this review commonly reported higher rates of depression and anxiety among SCI populations than that in the general populations, the comparison of findings are limited due to the use of different instruments.

Establishing Accurate Measurement of Depression and Anxiety with SCI

Establishing psychometric properties of validity and reliability of generic depression and anxiety instruments within SCI populations prior to their use is critically important. Such evidence will determine utility and help to avoid possible sources of bias. Systematic measurement also provides important information that enables clinicians to identify individuals who may require further evaluation, benefit from certain therapies, evaluate whether treatments are effective, and monitor progress. Such analyses of the various instruments in use among SCI populations will either justify the need for the development of a new instrument, should existing ones have poor reliability and validity, or identify instruments that are working properly and as intended. Agreement on the use of common instruments are beneficial as agreement will increase generalizability of findings and allow for the comparison of outcomes, which has both clinical and research implications.

In a recent review of depression instruments by Kalpakjian et al. (2009),¹² 24 studies were found that reported psychometric data for seven screening and/or symptom severity measures. A range of studies that provided psychometric data were included and classified into five levels. The levels ranged from level one studies being the most important with primary purposes of evaluating the psychometric properties of depression measures, to level three studies with depression as a secondary outcome and not the primary focus, to level five studies estimating the prevalence in an SCI sample. ¹² Findings were that reliability was good to excellent, validity was limited to concurrent, construct and clinical utility, and that the instruments were comparable in terms of internal consistency, factor structure, and clinical utility.¹² It was concluded that there is insufficient evidence to recommend the use of one instrument over another. Despite being a comprehensive study, the criteria used to evaluate and compare the psychometric properties of reliability and validity is unclear. It is therefore difficult to identify and differentiate each instrument’s strength of psychometric evidence.

Because systematic reviews conducted independently of each other on a similar topic often have methodological differences, inconsistent findings may reveal areas where further work is needed to resolve differences, whereas similar findings will provide valuable information to both clinicians and researchers.¹³ Therefore, the purpose of this independent review is to identify the depression and anxiety screening devices and symptom severity scales that have had their psychometric properties assessed among SCI populations, and to systematically evaluate the properties according to pre-established evaluation criteria.

Methods

Search Strategy

The Pubmed, CINAHL, Embase, Medline, HaPI, Psycinfo, and Sportdiscus e-databases were searched for papers, published between the dates of 1949 to July 2008, reporting on depression and anxiety instruments specific to SCI populations. Additional searching was conducted by reviewing the references of papers obtained from the electronic search. The keyword spinal cord injury and its related terms, paraplegia, quadriplegia, or tetraplegia, were used in conjunction with the psychometric terms, validity, reliability, responsiveness, reproducibility of results, and data collection. The search was completed by combining these terms with the names and abbreviations of familiar instruments used to screen for depression and anxiety along with the key words: depression, anxiety, depression measures, and anxiety measures.

Inclusion Criteria

To be included in this review, the instruments had to satisfy several requirements: 1) a depression and/or anxiety paper in which evaluation of the psychometric properties was the primary purpose (i.e., Level 1 papers based on the classification of Kalpakjian et al. (2009)); 2) a SCI population, 18 years of age and older; 3) SCI specific data; 4) to have been published in a peer-reviewed journal; and 5) to have been written in English.

Selection Process

The selection of articles used a multi-step process to ensure the inclusion of all relevant articles. First, the titles and abstracts of articles found through the electronic search were reviewed. Any study that referred to spinal cord injury, depression and/or anxiety in the title or abstract was imported to the online reference database manager, RefWorks.¹⁴ Second, after deleting non-relevant and duplicate papers, a research assistant and the primary author reviewed the titles and abstracts of all articles. Third, resulting articles were printed and re-reviewed to ensure that the paper was a psychometric paper on depression and/or anxiety instruments, and evaluated among persons with SCI. Finally, discrepancies in the retrieved studies were resolved through discussion with another author.

Data Extraction and Analysis

Consistent with the Spinal Cord Injury Rehabilitation Evidence (SCIRE)¹⁵ process, data extraction methods and standards for this review were based on the work by Fitzpatrick et al. (1998)¹⁶ and Andresen (2000).¹⁷ Fitzpatrick et al. (1998), provided the methods and standards for the data extraction and extraction form. Specifically, extracted data included reliability, validity, responsiveness, advantages and limitations of the instrument, interpretability of the scores, acceptability in terms of respondent burden, and feasibility in terms of administrative burden. The standards for summarizing the quality of the instruments were adapted from Andresen’s (2000) overview of criteria for assessing instruments, and can be found in table 1 along with the criteria used to assess rigor. In using these criteria, the psychometric properties were assigned a strength of evidence of either ‘excellent’, ‘adequate’, ‘poor’, or ‘n/a’ if there was insufficient information. For instruments with more than one report for a specific property, a range of evidence is given. Finally, rigor, or the thoroughness in the evaluation of the psychometric properties, was rated. If at least two studies corroborate each other’s findings, rigor is ‘excellent’, regardless of the strength of evidence. A rating of ‘adequate’ is given if a single study has ‘adequate’ to ‘excellent’ strength’s of evidence, while rigor is considered ‘poor’ if only a single study with a ‘poor’ strength of evidence is available.

Table 1.

Criteria for rating psychometric properties and clinical utility¹⁷

Criterion	Definition	Standard
Reliability	Degree to which the score is free from error.	Alpha for internal consistency ratings are excellent (≥0.80), adequate (0.70 – 0.79) or poor (≤0.69) ICC and Kappa for inter/intra and test-retest ratings are: excellent (≥0.75), adequate (0.40 – 0.74) or poor (≤0.39)
Validity	Degree to which an instrument measures what it purports to measure based on comparison to a gold standard or to another measure in which the relationship is hypothesized.	Correlations are: excellent (≥0.60), adequate (0.30 – 0.59) or poor (≤0.29) ROC analysis – AUC are: excellent (≥0.90), adequate (0.70 – 0.89) or poor (<0.70)
Respondent burden	Ease of completing the instrument.	Excellent: completion time of ≤ 5 minutes Adequate: completion time between 6 to 15 minutes Poor: completion time of 16 minutes and longer
Administrative burden	Ease of administering, scoring, and interpreting the instrument.	Excellent: self-report, scoring by hand and the resulting metric relevant and interpretable for researchers and clinicians Adequate: interview, computer scoring and more obscure interpretation Poor: costly and/or complex scoring and/or interpretation
Rigor	Thoroughness of the evaluation of the tools’ psychometric properties.	Excellent: if at least two studies have corroborative findings Adequate: if a single study has adequate or excellent measurement property findings, or if two studies have different findings Poor: if a single study has poor measurement property findings

For purposes of this review, we include instruments that screen for disorders and/or assess symptom severity. Instruments are defined as symptom severity scales if the response scale asks ‘how much’ or the frequency of symptoms, and screening instruments are defined as those that screen for a specific disorder and/or if cutoff scores are provided to indicate the need for further evaluation.

Results

In our literature search of seven electronic databases, 577 articles met the search criteria, and 13 papers reporting on 13 instruments were found that met the inclusion criteria. The instruments included in this paper are the Beck Depression Inventory (BDI),¹⁸ the Brief Symptom Inventory (BSI),¹⁹ the Center for Epidemiological Studies Depression Scale (CESD-20 and CESD-10),²⁰ the Depression, Anxiety, Stress Scales -21 (DASS-21),²¹ the General Health Questionnaire,²² the Hospital Anxiety and Depression Scale (HADS),²³ the Ilfeld Psychiatric Symptom Inventory (Ilfeld-PSI),²⁴ the Medical Emotional Distress Scale (MEDS),²⁵ the Patient Health Questionnaire (PHQ-9 and PHQ-9-Short),^26,27 the Symptoms Checklist-90-Revised (SCL-90-R) Research Subscales,^28,29 and the Zung Self-Rating Depression Scale (SRS).³⁰Table two provides a brief description of the instruments and table three describes the studies.

Table 2.

Description of instruments

Measure	Purpose	Construct	Subscales/Items	Duration (min)	Procedure	Scoring	Interpretation/Cutoff
BDI18	Screening and symptom severity	Depressive behaviour	21-items	5–10	4-point scale. Higher scores indicate more severe symptoms over the past week.	Summed scores range from 0 to 63.	18 or 27 has been recommended for SCI.32
CESD-2020,45	Screening and symptom severity	Depressive behaviour	20-items	≤5	4-point scale. 0 = rarely or none of the time; 3 = most or all of the time over the past week.	Summed scores range from 0 to 60.	16 -or- 19 has been recommended for SCI.36
CESD-1020	Screening and symptom severity	Depressive behaviour	10-items	≤5	4-point scale. 0 = rarely or none of the time; 3 = most or all of the time over the past week.	Summed scores range from 0 to 30.	11
PHQ-926,46	Screening and symptom severity	MDD	9-items	≤5	4-point scale. 0 = not at all; 3 = nearly every day over the past two weeks.	Summed scores range 0 to 27.	10
PHQ-9-Short27	Screening and symptom severity	MDD	3-items	≤5	4-point scale. 0 = not at all; 3 = nearly every day over the past two weeks.	Summed scores range from 0 to 9.	3 or 4
SRS30	Screening and symptom severity	Depressive behaviour	20-items	≤10	4-point scale. 1 = little of the time; 4 = most of the time over the past week.	Summed scores range from 20 to 80.	50
GHQ-2822	Screening and symptom severity	Non-psychiatric, psychiatric disorders	28 items comprising 4 subscales including 7-item depression and anxiety subscales	≤15	0-0-1-1 scoring representing frequency of experiences over the past few weeks	Responses are given a score of 0 for the lesser two grades of severity and 1 for the higher grades of severity. Scores range from 0 to 28.	General psych. disorder: 4
BSI19	Screening and symptom severity	General psychological symptomatology	53 items comprising 9 subscales including 6-item depression and anxiety subscales.	≤10	5-point scale. 0 = not at all; 4 = extremely over the past week	Convert raw score to a t-scores and compare to normative data. -or- Divide subscale scores by the number of items. Total subscale scores range from 0 to 4.	t-score: > 63 SCI Raw score: Depression: 1.56, 2.42, 2.14* Anxiety: 1.59, 1.82,1.7142
DASS-2121	Screening and symptom severity	Depression, Anxiety, Stress	21 items comprising 3 subscales including 7-item depression and anxiety subscales.	≤10	4-point scale. 0 = did not apply to me; 3 = applied to me very much over the past week	Responses are summed and multiplied by two giving a range of 0 to 42 for each subscale.	Depression: 6, 11,15,18 Anxiety: 4, 8, 12, 17**21,47
HADS23	Screening and symptom severity	Depression and Anxiety	14 items comprising 7-item depression and anxiety subscales.	≤5	4-point scale. 0 = not al all; 3 = very often indeed over the past week	Summed scores for each subscale range from 0 to 21.	Depression: 8 Anxiety: 8
Ilfeld-PSI24	Screening and symptom severity	Depression, Anxiety, Anger, Cognitive disturbance	29 items comprising 4 subscales including a 10-item depression and an 11-item anxiety subscales.	n/a	4-point scale. 0 = never; 3 = very often over the past week.	Total scores are calculated using: (A1 + A2 … An) /(3n × 100 ) where: A=question scores n=# of questions. Scores range from 0 to 100.	Depression: 10 Anxiety: 10
MEDS25	Symptom severity	Emotional distress	60 items comprising 7 subscales.	≤45	5-point scale. 0 = never or not at all; 4 = always or very much over the past week.	n/a	n/a
SCL-90-R Research Subscales28,29	Symptom severity	Depression and Anxiety	30 items comprising a 20-item depression and a 10-item anxiety subscales.	n/a	5-point scale. 0 = not at all; 4 = extremely over the past week	Summed subscale scores are divided by the number of items in the subscale. Total subscale scores range from 0 to 4.	n/a

n/a = information not available

^*Cutoff scores for SCI at discharge from hospital, from 0 – 24 months post discharge, and >24 months post discharge respectively.

^**Mild, moderate, severe, extremely respectively.

Table 3.

Included studies

Author	Measure	n	Objective	Procedure
Radnitz et al. (1997)32	BDI	124	Investigate the discriminative ability of seven items on the BDI.	Discriminant function analysis to identify poor discriminators.
Kuptniratsaikul et al. (2002)36	CESD-20 (Thai Version)	83	Investigate the CESD as an instrument to assess depression.	Comparison of the CESD (Thai Version) to the Thai depression inventory (TDI) and DSM-IV.
Miller et al. (2008)33	CESD-20 CESD-10	47	Investigate the psychometric properties of the CESD-10 and CESD-20.	Two week retest study among SCI individuals.
Bombardier et al. (2004)31	PHQ-9	849	Investigate the psychometric properties of the PHQ-9, and the role of somatic symptoms of depression in diagnosing depression.	Satistics were computed on rates of depressive symptoms and probable MDD.
Graves et al. (2008)27	PHQ-9-Short	3652	Investigate the psychometric properties of a short version of the PHQ-9.	PHQ-9 data were analyzed using Confirmatory Factor Analysis, Item Response Theory, Graded Response Model analysis and sensitivity and specificity analysis.
Tate et al. (1993)34	BSI SRS	162	Investigate the psychometric properties of the BSI and Zung.	Comparison of responses were made between the two scales and with clinical diagnoses.
Griffiths et al. (1993)39	GHQ-28	60	Validation study of the GHQ-28.	Clinical Interview Schedules (CIS) interviews were video-taped and scored. CIS results were compared to the results of the GHQ-28.
Heinrich et al. (1996)38	BSI	215	Investigate underlying factors of the BSI.	Item responses were analyzed with principal components and maximum likelihood factor estimation.
Mitchell et al. (2008)40	DASS-21	40	Investigate the psychometric properties of the DASS-21.	Comparison of the DASS-21 to the BSI, and the Mini International Neuropsychiatric Interview.
Woolrich et al. (2006)41	HADS	963	Investigate the psychometric properties of the HADS.	Comparison of the HADS to the Life Satisfaction Questionnaire; internal consistency and factor analysis were also conducted.
Campagnolo et al. (2002)35	Ilfeld-PSI	59	Assess the usefulness of the Ilfeld-PSI.	Comparison of the Ilfeld-PSI to the SRS.
Overholser et al. (1993)25	MEDS	81	Investigate the psychometric properties of the MEDS.	Comparison of the MEDS to the SCL-90-R, SRS, the Hopelessness Scale, and the Rosenberg Self-Esteem Scale.
Buckelew et al. (1988)37	SCL-90-R Research Subscales	275 (52 SCI subjects)	Investigate the psychometri properties of the SCL-90-R depression research subscale.	Internal consistency was assessed, as was the validity by using standardized clinical scales measuring similar constructs. Item response patterning with pain subjects and SCI subjects was also conducted.

Instruments Specific to Depression

The PHQ-9 and the PHQ-9-Short are two instruments that screen and assess symptom severity for MDD. The PHQ-9 is the only instrument that parallels the DSM-IV criteria in which clinical diagnoses are based; responses to the questions represent frequency of experiences over the past two weeks, all other instruments only refer to the past week. Reliability of the PHQ-9 is excellent (alpha = 0.87).³¹ Table four summarizes the reliability data for the PHQ-9 in addition to other instruments.

Table 4.

Depression and anxiety instruments – reliability in SCI

		Internal Consistency	Test-retest
Author	Measure
		Cronbach’s α	Retest	Coefficient
Radnitz et al. (1997)32	BDI	0.89	-	-
Miller et al. (2008)33	CESD 20	0.91	2 weeks	ICC = 0.87
	CESD 10	0.86		ICC = 0.85
Bombardier et al. (2004)31	PHQ-9	0.87	-	-
Overholser et al. (1993)25	MEDS	0.92
Tate et al. (1993)34	SRS	0.81	-	-
Griffiths et al. (1993)39	GHQ-28	0.82
Tate et al. (1993)34	BSI	0.87 (D)	-	-
Heinrich et al. (1996)38		0.85 (A) and 0.88 (D)*	-	-
Woolrich et al. (2006)41	HADS	0.85 (A) and 0.79 (D)	-	-
Buckelew et al. (1988)37	SCL-90-R Research Subscales	0.62 (SD) and 0.89 (CD)	-	-

A = anxiety, D = depression, SD = somatic depression symptoms, CD = cognitive depression symptoms

^*this study assessed 6 new subscales

Evidence in support of the PHQ-9’s validity range from adequate (r = −0.50 and −0.51) to excellent (r = 0.62), when compared to the the SF-36 subjective health question, the Satisfaction With Life Scale (SWLS), and the greater difficulty with daily role functioning component of the DSM-IV.³¹ Sensitivity and specificity data for the PHQ-9 are for individual items on the tool. Depressed mood, anhedonia, and feelings of failure in addition to two somatic symptoms of disturbed sleep, and decreased energy, are highly sensitive indicators of MDD.³¹ These findings suggest that both psychologic and somatic symptoms are indicative of major depressive disorder (MDD) among individuals with SCI.

The PHQ-9-Short is comprised of three items from the PHQ-9, including the items referring to little interest or pleasure in doing things, feeling down, depressed, or hopeless, and feeling bad about yourself – or that you are a failure of have let yourself or your family down. These three items have a relative efficiency of 0.66 compared to the PHQ-9.²⁷ When using a cutoff score of three, specificity was 93% and sensitivity was 87%.²⁷ A cutoff score of four yielded a specificity of 95%, and sensitivity of 82%.²⁷ The PHQ-9-Short excludes questions pertaining to somatic symptoms of depression. Table five presents the validity data for all instruments.

Table 5.

Depression and anxiety instruments – validity in SCI

Author	Measure	Concurrent Validity	Convergent Validity	Discriminant Validity	Specificity/Sensitivity	ROC analysis – AUC
Radnitz et al. (1997)32	BDI				Sensitivity = 50.0%; Specificity = 100.0% Cutoff = 27 Sensitivity = 83.3%; Specificity = 90.8% Cutoff = 18
Kuptniratsaikul et al. (2002)36	CESD 20				Sensitivity = 80.0%; Specificity = 69.8% Cutoff = 19	82.6%
Miller et al. (2008)33			VAS-F: r = 0.52	SF-36: r = −0.27 – −0.75*
	CESD 10		VAS-F: r = 0.57	SF-36: r = −0.37 – −0.71*
Bombardier et al. (2004)31	PHQ-9		DSM-IV: r = 0.62**	SWLS: r = −0.51 SF-36: r = −0.50***	Sensitivity = 37.9 – 93.8%; Specificity = 90.9 –97.7% Endorsement of 5/9 symptoms including one of anhedonia or depressed mood.
Graves et al. (2008)27	PHQ-9-Short				Sensitivity = 0.82; Specificity = 0.95 Cutoff= 4 Sensitivity = 0.87; Specificity = 0.93 Cutoff = 3
Griffiths et al. (1993)39	GHQ-28		CIS: r = 0.83		Sensitivity = 81.0%; Specificity = 82.0% Cutoff = 4	91%
Tate et al. (1993)34	SRS	BSI (D): r = 0.52			Depression Sensitivity = 86.0%; Specificity = 61.0% Cutoff = 55
	BSI	SRS: r = 0.52			Depression Sensitivity = 57.0%; Specificity = 87.0% Cutoff = 65
Mitchell et al. (2008)40	BSI	DASS-21 (D): r = 0.70 DASS-21 (A): r = 0.61			Depression Sensitivity = 57.0%; Specificity = 82.0% Cutoff = 63 Anxiety Sensitivity = 86.0%; Specificity = 88.0% Cutoff = 63
	DASS-21	BSI (D): r = 0.70 BSI (A): r = 0.61			Depression Sensitivity = 57.0%; Specificity = 76.0% Anxiety Sensitivity = 86.0%; Specificity = 64.0%
Woolrich et al. (2006)41	HADS			HADS (A) w/LSQ: r = −0.42 HADS (D) w/LSQ: r = −0.66
Campagnolo et al. (2002)35	Ilfeld-PSI (D)	SRS: r = 0.72				55%
	SRS	Ilfeld-PSI (D): r = 0.72				76%
Overholser et al. (1993)25	MEDS	SCL-90-R (D): r = 0.77 SRS: r = 0.71	Hopelessness Scale: r = 0.65	Rosenberg SE Scale: r = −0.75

A = anxiety, D = depression, SWLS = Satisfaction with Life Scale, SF-36 = Short Form Health Survey-36, CIS = Clinical Interview Schedule, LSQ = Life Satisfaction Questionnaire, VAS-F = Visual Analog Scale-Fatigue.

^*range for each of the 8 SF-36 subscales

^**daily role functioning; DSM IV criterion C

^***SF-36 subjective health question

The BDI, CESD-20, CESD-10, and the SRS are instruments that assess the severity of depressive symptoms and have cutoff scores that may be used to screen for depressed moods. Four of the 20 questions on the CESD-20 are reverse scored and 10 of the 20 questions on the SRS are worded negatively. Somatic symptoms of depression are included in these instruments. The reliability data for these instruments are excellent (alpha = 0.89, 0.91, 0.86, and 0.81 respectively).^{32, 33, 34} Only the CESD-20 and CESD-10 have two-week test-retest reliability data (ICC = 0.87 and 0.85 respectively).³³

The validity of the CESD-20 and CESD-10 was assessed against eight scales on the SF-36, and the Visual Analog Scale-Fatigue (VAS-F). Results range from poor (r = −0.27) to excellent (r = −0.75) for the CESD-20, and from adequate (r = −0.38) to excellent (r = −0.71) for the CESD-10.³³ Validity in support of the SRS range from adequate to excellent (r = 0.52, 0.71, and 0.78) as per the results when compared to the BSI, MEDS, and Ilfeld-PSI depression subscale.^34,25,35

Elevated cutoff scores have been identified for the BDI and CESD-20 to account for the inclusion of somatic symptoms and the possibility of overestimating depressed moods among SCI populations. When using a cut score of 27, the BDI has a sensitivity of 50% and specificity of 100%, and a sensitivity of 83.3 and specificity of 90.8 when a using a cutoff score of 18.³² A CESD-20 (Thai version) cutoff score of 19, when used with a Thai SCI population, was found to have sensitivity of 80% and specificity of 69.8%.³⁶ The SRS has a sensitivity of 86% and specificity of 61% when using a cut score of 55.³⁴

The MEDS is a measure that is used to assess the severity of depressive symptoms and has excellent internal consistency (alpha = 0.92).²⁵ The validity evidence in support of the MEDS are excellent (r = 0.65, 0.71, −0.75, and 0.77) when correlated with the Hopelessness Scale, SRS, Rosenberg Self Esteem Scale, and SCL-90-R depression subscale.²⁵

Instruments with Depression and Anxiety Subscales

The BSI, DASS-21, HADS, Ilfeld-PSI are instruments that have subscales to screen for both depression and anxiety symptoms and to assess symptom severity. Although the GHQ-28 has depression and anxiety subscales to assess symptom severity, the total score is used to screen for psychiatric disorders. The SCL-90-R Research Subscales has subscales to assess symptoms for both depression and anxiety, but psychometric data is limited to the depression subscales with poor reliability (alpha = 0.62) for the somatic symptoms subscale, and excellent reliability for the cognitive symptom subscale (alpha = 0.89).³⁷ The DASS-21 is the only instrument with subscales excluding reference to somatic symptoms.

The internal consistency evidence of both the BSI depression and anxiety subscales are excellent (alpha = 0.87 and 0.88 for the depression subscale and 0.85 for the anxiety subscale). ^38,34 The anxiety subscale on the HADS also has excellent internal consistency (alpha = 0.85), whereas evidence for the depression subscale is adequate (alpha = 0.79).³⁸ The GHQ-28 has excellent internal consistency (alpha = 0.82).³⁹

The BSI depression subscale’s validity ranges from adequate to excellent, when compared with the SRS (r = 0.52)³⁴ and DASS-21 (0.70).⁴⁰ Similarly, its anxiety subscale has excellent results when assessed against the DASS-21’s anxiety subscale (r = 0.61), and adequate results when compared to the SRS (r = 0.38).^40,34 The HADS anxiety subscale has adequate validity whereas the depression subscale’s is excellent when correlated with the Life Satisfaction Questionnaire (r= −0.42 (A) and −0.66 (D)).⁴¹ The Ilfeld-PSI depression subscale has excellent validity evidence when compared to the SRS (r = 0.78), and adequate evidence for the anxiety subscale (r = 0.59).³⁵ When compared to the Clinical Interview Schedule, the GHQ-28 has excellent validity (r = 0.83).³⁹

Two studies report sensitivity and specificity data for the BSI depression subscale and one for the anxiety subscale. When using an elevated t-score cutoff of 65, Tate et al. (1993) found the depression subscale to have a sensitivity of 57% and specificity of 87%.³⁴ When using the cutoff scores recommended for SCI,⁴² Mitchell et al. found sensitivity of 14% and specificity of 97%, and sensitivity of 57% and specificity of 82% when using the traditional cutoff scores.⁴⁰ The BSI anxiety subscale has sensitivity and specificity of 86% and 88% when using the traditional cutoff scores, and 43% and 100% when using the elevated cut scores.⁴⁰ The DASS-21 depression subscale has a sensitivity of 57% and specificity of 76%, and the anxiety subscale’s is 86% and 64%.⁴⁰ The GHQ-28 has a sensitivity and specificity of 81% and 82% when using a cutoff score of four.³⁹

In terms of administrative and respondent burden, the number of items on each instrument ranges from three (PHQ-9-Short) to 60 (MEDS), and most have reported completion times of 10 minutes or less. The MEDS may take up to 45 minutes depending on the individual’s level of distress. The Ilfeld-PSI has a more complex scoring system, and the interpretation of scores on several instruments vary from comparing raw scores to cutoff scores to converting raw scores to t-scores (BSI) or percentile scores (DASS-21). Table three provides an overview of the instruments along with the scoring system, and table six provides a summary of all ratings, including rigor.

Table 6.

Summary of ratings^*

Measure Reliability	BDI	CESD-20	CESD-10	PHQ-9	PHQ-9-Short	SRS	GHQ-28	BSI	DASS-21	HADS	Ilfeld-PSI	MEDS	SCL-90-R
Internal Cons.	+++	+++	+++	+++	−	+++	+++	+++ (D, A)	−	+++ (D), ++ (A)	−	+++	+ (SD), +++ (CD)
Test-retest	−	+++	+++	−	−	−	−	−	−	−	−	−	−
Rigor	++	++	++	++	−	++	−	+++	−	++	−	++	++
Measure	BDI	CESD-20	CESD-10	PHQ-9	PHQ-9-Short	SRS	GHQ-28	BSI	DASS-21	HADS	Ilfeld-PSI	MEDS	SCL-90-R
Validity	−	+ to +++	++ to +++	++ to +++	−	++ to +++	+++	++ (D), +++ (A)	+++	+++ (D), ++ (A)	+++	+++	−
Sensitivity/ Specificity	++	++	−	−	+++	++	+++	++ (D), +++ (A)	++	−	−	−	−
ROC	−	++	−	−	−	++	+++	−	−	−	+	−	−
Rigor	++	++	++	++	++	+++	++	+++ (D), ++ (A)	++	++	++	++	−
Measure	BDI	CESD-20	CESD-10	PHQ-9	PHQ-9-Short	SRS	GHQ-28	BSI	DASS-21	HADS	Ilfeld-PSI	MEDS	SCL-90-R
Respondent burden	++	+++	+++	+++	+++	++	++	++	++	+++	−	+	−
Admin- istrative burden	+++	+++	+++	+++	+++	+++	+++	++	++	+++	++	++	++

A = anxiety, D = depression, SD = somatic depression symptoms, CD = cognitive depression symptoms

^*+++ = excellent; ++ = adequate; + = poor; − = not available

Discussion

Depression and anxiety disorders and the effects of severe symptoms can be debilitating conditions for individuals with recent or longstanding spinal cord injuries. As diagnoses of depression and anxiety disorders are time consuming and costly, access to quick and inexpensive instruments to screen for disorders or assess the severity of symptoms to determine the need of additional evaluation is invaluable. The use of such instruments however is predicated upon an establishment of an instrument’s psychometric properties within specific populations. The purpose of this review was therefore to identify depression and anxiety screening and symptom severity measures in current use among SCI populations, according to our inclusion criteria, and consider their psychometric properties according to pre-established criteria.

This review identified 13 papers with a specific focus on assessing the psychometric properties of 13 depression and anxiety instruments that have been used with SCI populations. Reliability data was available for 10 instruments, and validity results were available for 12. Depending on the instrument, evidence spanned the spectrum of evaluation criteria varying from poor to excellent. Responsiveness data was not reported in any of the studies.

The variability of diagnostic criteria or symptoms and time periods utilized in the instruments is an issue. Such variability result in difficulties in applying the findings to established classification systems of mood or anxiety disorders (e.g., DSM-IV or International Classification of Diseases; ICD).^43,31 The PHQ-9 is the only instrument designed to parallel the DSM-IV symptom and duration criteria for diagnosing MDD. All other instruments assess symptom severity over the past week and use cutoff scores to determine the need for further evaluation.

Both the PHQ-9 and PHQ-9-Short screen for MDD and assess symptom severity. In the study by Bombardier et al. (2004), the sensitivity and specificity results for individual items predicting MDD indicate that somatic symptoms of depression such as appetite change, sleep disturbance, and poor energy, are predictive of MDD and should be included in depression scales.³¹ In a subsequent study, Krause et al. (2008)found evidence in support of an underlying somatic factor, in addition to a general factor, with the PHQ-9.⁴⁴ The somatic factor comprised three items including, appetite change, sleep disturbance, and poor energy. These results suggest that symptoms of depression may be no different in the SCI population than in the general population, and that issues related to overestimating the prevalence of depression are due to SCI sequlae.

Overestimating the prevalence of symptoms is an issue that has affected the results of several instruments included in this review. Radnitz et al. (1997), found three items on the BDI to be poor discriminators of depression in a SCI sample, resulting in artificially inflated scores.³² They therefore recommend higher cutoff scores to correct for the similarities between somatic symptoms and SCI sequelae. Similarly, Kuptniratsaikul et al. (2002) has recommended a cutoff score of 19, versus the traditional cutoff score of 16, when the CESD-20 is used with SCI populations.³⁶ However, because the Thai version of the CESD-20 was used in this study is it unclear if the results are applicable to the original CESD-20. The BSI depression and anxiety subscales have also been found to overestimate the prevalence of depression and anxiety symptoms.⁴² Heinrich et al. (1994), developed cutoff scores specific for use with SCI populations. ⁴² In a subsequent study, Heinrich and Tate (1996) developed depression and anxiety subscales specific for use with SCI individuals.³⁸ They recommend the SCI specific subscales for use in clinical settings, or using the SCI specific cutoff scores when administering the traditional subscales.

In considering direct comparisons between instruments, Mitchell et al. (2008) used both the BSI traditional and SCI specific cutoff scores, and compared them to the DASS-21 depression and anxiety subscales. Prevalence of depression and anxiety were higher on the DASS-21, despite the DASS-21 excluding many somatic symptoms not relevant to SCI populations.⁴⁰ The DASS-21 was as sensitive as the BSI but had lower specificity to identify either depression or anxiety.⁴⁰ In another direct comparison between the BSI depression scale and SRS, the SRS was found to be superior, due to a higher level of sensitivity, in identifying people with SCI who were at risk of being depressed.³⁴ Campagnolo et al. (2002) compared the efficacy of the Ilfeld-PSI to the SRS in persons with SCI. They found that the Ilfeld-PSI poorly discriminates somatic symptoms unrelated to depression, and therefore is not an adequate instrument for use in SCI populations.³⁵ The SRS, however, showed acceptable discriminative ability (AUC = 0.76), warranting its use among SCI populations.³⁵

The instrument that is closest to being specific for SCI is the MEDS. It is designed to assess the severity of depressive symptoms for use in populations with physical disability or illness. It excludes somatic symptoms that could be the result of the condition, such as SCI. Although the MEDS has excellent reliability and validity, as per our criteria, it possesses the greatest administrative burden because it has the most items and can take as long as 45 minutes to complete depending on the amount of distress experienced by the patient. At present, there is only one psychometric specific study that has assessed its properties among SCI populations, and its use among SCI populations is very limited.

Limitations

Because studies were excluded if they were not published in English or did not have a specific focus to assess the psychometric properties of depression and anxiety instruments used with SCI populations, evidence is potentially missing that could influence both the psychometric ratings of reliability and validity, and rigor results.

Conclusion

Given that the current reliability and validity findings of the depression and anxiety instruments range for the most part from adequate to excellent, and the amount of work to develop cutoff scores specific for SCI populations, there is at present no need to develop SCI specific instruments. At the same time, however, because one instruments’ psychometric properties do not clearly stand above the others, it is difficult to recommend the use of one over another. We can, however, mention that because the Ilfeld-PSI has poor discriminative ability it should not be used with SCI populations until further evaluation proves it useful. If we consider the few direct comparisons between some of the measures, the SRS was found to be superior over both the Ilfeld-PSI and the BSI, and the BSI was found to be as sensitive as the DASS-21 but with higher specificity to detect both depression and anxiety. The administrative and respondent burden are similar for most instruments except for the MEDS which is comprised of the most number of items and has the potential to take 45 minutes to complete. As a result, when selecting an instrument, the clinician must consider the specific purpose, in addition to other clinical considerations within the context of their practice and intended use (e.g., hospital, outpatient clinic, drug/alcohol treatment programs).

In this review, the SRS and BSI were the only instruments with more than one paper reporting psychometric evidence. Therefore, all instruments require additional investigation in SCI samples for reliability and validity purposes, and if they are to be used to evaluate outcomes related to treatment or change over time, responsiveness data should be investigated as well. Longitudinal studies and/or appropriately configured clinical trials are required. Administering the instruments in tandem with each other and with clinical interviews for diagnostic purposes would provide valuable information, as would comparison of results to normative data specific to persons with SCI.

More data on the psychometric properties of the depression and anxiety instruments used with SCI populations is key. Further evidence will lead to either determining the need for the development of new instruments specific for SCI populations, or identifying and resolving the problems in the instruments currently used, potentially leading to agreement on the use of common instruments with SCI populations.