Performance of Screening Tools in Detecting Major Depressive Disorder among Patients with Coronary Heart Disease: A Systematic Review

Abstract

Background

Major depressive disorder (MDD) is common in patients with coronary heart disease (CHD) and there is no consensus on the optimal screening tool for use in identifying MDD. This study aimed to systematically review the performance of various screening tools in the identification of MDD.

Material/Methods

Eligible studies published before 31 Dec 2013 were identified from the following databases: Ovid Medline, EMBASE, PsycINFO, Scopus, Cochrane Library, CINAHL Plus, and Web of Science.

Results

Eight studies aiming to identify MDD in CHD patients were included, and there were 10 self-reporting questionnaires (such as PHQ-2, PHQ-9, PHQ categorical algorithm, HADS-D, BDI, BDI-II, BDI-II-cog, CES-D, SCL-90, 2 simple yes/no items) and 1 observer rating scale (Ham-D). For MDD alone, the sensitivity and specificity of various screening tools at the validity and optimal cut-off point varied from 0.34 [0.19, 0.52] to 0.96 [0.78, 1.00] and 0.69 [0.65, 0.73] to 0.97 [0.93, 0.99]. Results showed PHQ-9 (≥10), BDI-II (≥14 or ≥16), and HADS-D (≥5 or ≥4) were widely used for screening MDD in CHD patients.

Conclusions

There is no consensus on the optimal screening tool for MDD in CHD patients. When evaluating the performance of a screening tool, balancing the high sensitivity and negative predictive value (NPV) between specificity and positive predictive value (PPV) for screening or diagnostic purpose should be considered. After screening, further diagnosis, appropriate management, and necessary referral may also improve cardiovascular outcomes.

Background

Major depressive disorder (MDD) is a common disease in patients with coronary heart disease (CHD), with a prevalence of 20% to 30% [1–4] and is associated with worse cardiac prognosis [5].

The American Heart Association (AHA) and American Academy of Family Physicians (AAFP) have recommended routine depression screening for cardiac patients, including those suffering an myocardial infarction (MI) [6,7]. However, the UK National Institute for Health Care Effectiveness does not recommend the routine depression screening in primary care because of limited/conflicting findings on the optimal screening tools [8]. After screening, further psychosocial assessment of patients who respond positively to the screening tools is considered to effect treatment evaluation [9].

Depression has been categorized by 2 commonly used psychiatric classifications: the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) [10] and the International Classification of Diseases (ICD-10) [11]. The criteria for the categorization are based on the severity, sub-types of specific symptoms, duration, course, and whether it is secondary to a physical or other psychiatric condition. MDD is diagnosed if participants meet at least 1 core criterion (depressed mood or anhedonia) and at least 4 additional criteria within the previous 2 weeks.

In addition to these 2 diagnostic criteria, several well-established screening tools have been developed and include self-report questionnaires and observer rating scales for depression.

PHQ-9 is a self-rating instrument for depression developed in the late 1990s for the Primary Care Evaluation of Mental Disorders (PRIME-MD) [12]. It consists of 9 items designed to correspond to the DSM-IV for MDD [13]. The Hospital Anxiety and Depression Scale (HADS) rating scale has 14 items, 7 of which are designed to measure anxiety (HADS-A), and 7 for depression (HADS-D) [14]. HADS-D was developed to assess depression in medically ill patients and its items focus on the loss of interest and pleasure with somatic features excluded from measurement. PHQ-9 is mainly used in North American while HADS is more frequently used in Europe. Furthermore, PHQ-9 and HADS differ in important ways, such as the exclusion of somatic symptoms in the latter [15].

The Beck Depression Inventory (BDI) is 1 of the most commonly used self-rating scales in the evaluation of depression. Since the development of this tool in 1961, it has been employed in numerous empirical studies [16]. BDI has recently been updated (the Beck Depression Intervention-II [BDI-II]) to better match the current definition of MDD, as it measures symptoms in the preceding 2 weeks, as compared to 1 week in the BDI. BDI-II also has fewer items used to assess the somatic symptoms of depression because the somatic symptoms may confound the diagnosis of depression in post-MI patients. BDI-II-cog is an 8-item cognitive subscale of BDI-II. It is shorter and has no somatic items [17]. The 90-item Symptom Check List (SCL-90) has been proven to be a useful tool for identifying psychiatric symptoms in primary care and research. SCL-90 is frequently used for case identification [18]. CES-D, a 20-item questionnaire with the total score ranging from 0 to 60, has been used extensively in CHD patients, demonstrating utility [19], and a cut-off value of 16 has been found to be adequately sensitive and specific in the identification of CHD [20]. The 17-item Hamilton Depression Rating Scale (Ham-D-17) is an observer rating scale [21] used to evaluate the severity of depression.

There is currently no consensus on which commonly used screening tool is best for use in identifying MDD in CHD. Furthermore, MDD is under-diagnosed in CHD patients because healthcare providers rarely use a standardized screening instrument [22].

To the best of our knowledge, no systematic review has been published to comparatively evaluate the screening tools against the diagnostic criteria from depression in DSM-IV and ICD-10 in CHD patients. To investigate the performance of screening tools in identifying MDD in CHD patients, this systematic review was performed to assess the diagnostic accuracy (focusing on sensitivity and specificity) of various screening tools as compared to the diagnostic criteria.

Material and Methods

Search strategy and data collection

Two reviewers independently searched the following electronic databases in English: Ovid Medline, EMBASE, PsycINFO, Scopus, Cochrane Library, CINAHL Plus, and Web of Science before 31 Dec 2013. The researchers developed 2 comprehensive search themes. To identify studies related to CHD, following terms were used for searching: “coronary heart disease”, “ischemic heart disease” or “heart disease”; to identify relevant screening tools, a second search was performed using the following terms: “tool”, “measurement”, or “assessment”; and to identify studies related to depression, “depress*” was used as a term for searching. Then, the resultant literatures were merged.

Study selection

Resultant articles were independently evaluated by 2 authors. First, the authors screened the titles and abstracts for eligibility. Then, they performed full-text reviewing on each article meeting the inclusion criteria or having some uncertainties for eligibility. Any disagreements in data extraction and/or specific study inclusion were resolved through consensus by discussion.

Inclusion criteria were:

1. Participants were diagnosed as having CHD and randomly recruited from different healthcare settings, including hospitals and communities.

2. The diagnostic criteria for depression were from DSM-IV of the American Psychiatric Association [23] or ICD-10 of the World Health Organization [11].

3. The studies included MDD or severe depressive episode (MDE) diagnosed according to corresponding diagnostic criteria, other than minor depression or depressive syndrome.

Exclusion criteria were:

1. Participants were diagnosed with other cardiovascular diseases, including cardiomyopathy and heart failure other than CHD.

2. Studies did not clearly compare the screening tools with diagnostic criteria in DSM or ICD.

Data extraction and quality assessment

Studies meeting the inclusion criteria were assessed for methodological quality using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS), which is a tool used for quality assessment included in systematic reviews [24]. Each study was independently assessed by 2 authors, and data were extracted from articles according to the collecting data items of QUADAS [24] and Cochrane-handbook [25]. The following data were extracted: settings, gender, mean age, stage of depression, screening tools, diagnostic criteria, sensitivity and specificity of screening tools, and numbers of patients with MDD diagnosed by diagnostic criteria.

Statistical analysis

The sensitivity, specificity, likelihood ratios, and predictive values were determined and the binomial 95% confidence intervals were calculated for the sensitivity and specificity with Reference Manager (RevMan) Version 5.1 from the Cochrane Collaboration [25].

Results

Literature search

After searching Ovid Medline, EMBASE, PsycINFO, Scopus, Cochrane Library, CINAHL Plus, and Web of Science with corresponding terms, a total of 3878 articles were identified by the initial screening and included 769 from Ovid Medline, 411 from EMBASE, 562 from PsycINFO, 887 from Scopus, 288 from Cochrane Library, 229 from CINAHL Plus, and 662 from Web of Science. Two studies were searched from the references of included studies. We removed 1345 because of duplication. In the remaining 2535 studies, 2356 were excluded after screening their abstracts. Then, the full texts of 179 articles were assessed. In 167 studies, there were no diagnostic criteria for depression, 3 studies were conducted from 3 different viewpoints in the same population, and randomization was not performed in 1 study. Finally, 8 studies were included for further meta-analysis in accordance with the PRISMA Flow Chart [26]. The flow chart is shown in Figure 1.

Figure 1.

PRISMA flow chart.

Included studies

The characteristics of included studies are summarized in Table 1.

Table 1.

Summary characteristics of included studies.

Study	Country	Setting	Patients	Female	Mean age, year	Stage of depression	Heart disease	Instrument	Reference standard (diagnostic criteria)
Haddad et al., 2013 [29]	UK London	Primary care	730	214	71.44 (no depressive disorder)/65.28 (depressive disorder)	Depressive episode; MDD	Coronary heart disease	PHQ-9 HADS-D	CIS-R (ICD-10)
Bunevicius et al., 2012 [35]	Lithuania Palanga	Cardiovascular rehabilitation clinic	522	28	58±9	MDE	CAD undergoing rehabilitation	HADS-D BDI-II	MINI (DSM-IV-TR)
Swardfager et al., 2011 [30]	Canada Toronto	Rehabilitation Institute	195	39	63.4±11.5	MDD	CAD (post-CABG, post-MI, post-PCI)	CES-D	SCID (DSM-IV)
Huffman et al., 2010 [37]	US Boston	Coronary care unit or cardiac step-down unit of hospital	131	6	62.3±12.5	MDD	Post-MI	BDI-II BDI-II-cog	SCID (DSM-IV)
Frasure-Smith et al., 2008 [36]	Canada Montreal	Heart institute and hospital	804	155	62.0±10.6	MDD	2 month after hospital discharge for ACS	BDI-II	SCI (DSM-IV)
Stafford et al., 2007 [15]	Australia Geelong	Hospital	193	37	64.14±10.37	Major depression; minor depression; dysthymia	3 months post-discharge for patients of PTCA, AMI or CABG	PHQ-9 HADS	MINI (DSM-IV)
McManus et al., 2005 [33]	US California	Medical center	1024	185	67±11	MDD	Stable CHD	CES-D PHQ-9 PHQ-2 2 simple yes/no items	DIS (DSM)
Strik et al., 2001 [28]	Netherlands Maastricht	Hospital	206	50	59±10.6 (male) 62.9±10.7 (female)	Major depression; minor depression	One month post-MI	SCL-90 BDI HADS Ham-D	SCI (DSM-IV)

Accuracy of various screening tools in identifying depression in CHD patients

We re-analyzed the data of included studies with the methodology, the summary Receiver Operating Characteristic (sROC), and the bivariate approach [27]. In 8 studies included, there were 10 self-reporting questionnaires (such as PHQ-2, PHQ-9, PHQ categorical algorithm, HADS-D, BDI, BDI-II, BDI-II-cog, CES-D, SCL-90, 2 simple yes/no items) and 1 observer rating scale (Ham-D).

In some self-reporting questionnaires, different cut-off values were used to screen for MDD. The data were insufficient to calculate the sROC and the pooled statistic sensitivity and specificity of each screening tool and different cut-off values. Thus, the positive LR, negative LR, positive predictive value (PPV), negative predictive value (NPV), and prevalence were calculated using RevMan 5.1.

As shown in Table 2, for MDD alone, the sensitivity and specificity of different screening tools at the validity and optimal cut-off point varied from 0.34 [0.19, 0.52] to 0.96 [0.78, 1.00] and from 0.69 [0.65, 0.73] to 0.97 [0.93, 0.99], respectively.

Table 2.

Accuracy of different screening tools for identification for depression in patients with CHD.

Study	Instruments (Recommended cut-off point)	Total sample size	Sensitivity [95% CI]	Specificity [95% CI]	Positive LR	Negative LR	PPV	NPV	Prevalence
Haddad et al., 2013 [29]	PHQ categorical algorithm	730	0.59 [0.41, 0.76]	0.95 [0.93, 0.97]	12.5587	0.4264	0.3654	0.9808	0.0438
Bunevicius et al., 2012 [35]	HADS-D (≥5)	632	0.77 [0.64, 0.87]	0.69 [0.65, 0.73]	2.4849	0.3360	0.2299	0.9612	0.1073
	BDI-II (≥14)	632	0.89 [0.78, 0.96]	0.74 [0.70, 0.78]	3.4386	0.1447	0.2924	0.9829	0.1073
Swardfager et al., 2011 [30]	CES-D (≥16)	195	0.93 [0.81, 0.99]	0.78 [0.71, 0.85]	4.2847	0.0891	0.5479	0.9754	0.2205
Huffman et al., 2010 [37]	BDI-II (≥16)	131	0.88 [0.64, 0.99]	0.92 [0.8, 0.96]	11.1765	0.1277	0.6250	0.9813	0.1298
	BDI-II (≥14)	131	0.88 [0.64, 0.99]	0.84 [0.76, 0.90]	5.5882	0.1397	0.4545	0.9796	0.1298
	BDI-II-cog (≥3)	131	0.88 [0.64, 0.99]	0.82 [0.73, 0.88]	4.7899	0.1442	0.4167	0.9789	0.1298
Frasure-Smith et al., 2008 [36]	BDI-II (≥14)	804	0.91 [0.81, 0.97]	0.78 [0.74, 0.80]	4.0564	0.1132	0.2364	0.9914	0.0709
Stafford et al., 2007 [15]	PHQ-9 (≥10)	193	0.54 [0.37, 0.71]	0.91 [0.86, 0.95]	6.1265	0.5016	0.5758	0.9000	0.1813
	PHQ categorical algorithm	193	0.34 [0.19, 0.52]	0.97 [0.93, 0.99]	10.8343	0.6786	0.7059	0.8693	0.1813
	HADS-D (≥5)	193	0.86 [0.70, 0.95]	0.75 [0.68, 0.82]	3.4725	0.1897	0.4348	0.9597	0.1813
McManus et al., 2005 [33]	CES-D (≥10)	1024	0.76 [0.70, 0.81]	0.79 [0.76, 0.82]	3.6139	0.3052	0.5030	0.9213	0.2188
	PHQ-9 (≥10)	1024	0.54 [0.47, 0.61]	0.90 [0.88, 0.92]	5.4018	0.5109	0.6020	0.8748	0.2188
	PHQ-2 (≥3)	1024	0.39 [0.32, 0.46]	0.91 [0.89, 0.93]	4.2513	0.6731	0.5404	0.8431	0.2166
	2simple (≥1)	1024	0.90 [0.85, 0.93]	0.69 [0.66, 0.72]	2.8946	0.1488	0.4477	0.9600	0.2188
Strik et al., 2001 [28]	SCL-90 (≥25)	199	0.96 [0.78, 1.00]	0.74 [0.67, 0.80]	3.6597	0.0589	0.3235	0.9924	0.1156
	BDI (≥10)	199	0.83 [0.61, 0.95]	0.79 [0.72, 0.85]	3.9295	0.2202	0.3393	0.9720	0.1156
	HADS-D (≥4)	179	0.87 [0.66, 0.97]	0.75 [0.67, 0.82]	3.4783	0.1739	0.3390	0.9750	0.1285
	Ham-D (≥15)	206	0.87 [0.66, 0.97]	0.92 [0.87, 0.96]	11.3665	0.1412	0.5882	0.9826	0.1117

Quality assessment

The QUADAS was used to assess the quality of included studies, in which the patient spectrum, diagnostic criteria, disease progression bias, verification bias, clinical review bias, incorporation bias, test execution, study withdrawals, and indeterminate results were evaluated [24]. Assessment of the spectrum bias showed that participants in 8 studies were representative of CHD patients. As shown in Table 3, all the studies provided clear inclusion criteria. Although 5 different diagnostic criteria, such as the revised Clinical Interview Schedule (CIS-R), the Mini International Neuropsychiatric Interview (MINI), the Structure Clinical Interview for DSM-IV Axis Disorders (SCID), the Diagnostic Interview Schedule (DIS) and the Structure Clinical Interview (SCI), were used in the 8 studies, patients met the criterion standard for MDD according to DSM-IV or ICD-10. Ideally, the results of the index test and the reference standard should be collected from the same patients at the same time. If this is impossible and a delay occurs, misclassification may be present due to spontaneous recovery or disease progression. Assessment of disease progression bias showed 6/8 studies had high quality. In addition, there was no partial verification bias, differential verification bias, or incorporation bias. Sufficient description of index test and the reference standard were present in all the studies. Assessment of review bias showed 4/8 studies were blinded between index test and reference standard, which can be used in practice when test results are interpreted. There was explanation of withdrawals and no report of test results that were intermediate or could not be interpreted in any of the studies.

Table 3.

Assessment of included studies quality with QUADAS tool.

Item	Yes	No	Unclear
1. Was the spectrum of patients representative of the patients who will receive the test in practice?	8	0	0
2. Were selection criteria clearly described?	8	0	0
3. Is the reference standard likely to correctly classify the target condition?	8	0	0
4. Is the time period between reference standard and index test short enough to be reasonably sure that the target condition did not change between the two tests?	6	0	2
5. Did the whole sample or a random selection of the sample receive verification using a reference standard of diagnosis?	8	0	0
6. Did patients receive the same reference standard regardless of the index test result?	8	0	0
7. Was the reference standard independent of the index test (ie, the index test did not form part of the reference standard)?	8	0	0
8. Was the execution of the index test described in sufficient detail to permit replication of the test?	8	0	0
9. Was the execution of the reference standard described in sufficient detail to permit its replication?	8	0	0
10. Were the index test results interpreted without knowledge of the results of the reference standard?	4	0	4
11. Were the reference standard results interpreted without knowledge of the results of the index test?	4	0	4
12. Were the same clinical data available when test results were interpreted as would be available when the test is used in practice?	8	0	0
13. Were non-interpretable/intermediate test results reported?	0	8	0
14. Were withdrawals from the study explained?	8	0	0

Discussion

In this study, 8 studies conducted in CHD patients from primary care settings and hospitals were systemically reviewed. The sensitivity and specificity of screening tools were different from the diagnostic criteria in identifying MDD. The screening tools had the highest sensitivity without affecting the specificity in identifying the greatest number of true-positives.

For screening, high sensitivity and NPV are more important than high specificity and PPV [28]. Sensitivity should be maximized when choosing a screening tool for depression so that cases are not missed. In our study, CES-D (≥16) and SCL-90 (≥25) had high sensitivity and NPV as compared to other screening tools, with the sensitivity of 93% and 96%, respectively, and NPV of 97.54% and 99.24%, respectively. It seems that both tools are effective in screening MDD. However, the low PPV of these 2 tools (54.79% and 32.35%, respectively) meant that only about half of patients who have a positive result on the screening meet the diagnostic criteria for major depression. Thus, any patient who has positive results on depression screening should be followed up to confirm the diagnosis of depression.

For diagnosis, high specificity and PPV are more important [28]. The PPV depends, in part, on the prevalence of the disorder in the population. Due to the relatively low number of depressed patients as compared to non-depressed patients in all the studies, the PPVs (22.99% to 70.59%) were much lower than NPVs (84.31% to 79.24%). A cut-off value of ≥10 on PHQ-9 had the sensitivity of only 54% in 2 studies, but its high specificity (90% and 91%) and high PPV (57.58% and 60.20%) mean that patients who screen positive do not need a follow-up for confirming the diagnosis of depression.

Low PPV is caused by low prevalence. It has been found that up to 30% of patients having stable heart disease also develop depression [1–4]. The prevalence of MDD in CHD patients in the included studies varied from 4.38% [29] to 22.05% [30]. The study with the lowest prevalence (4.38%) was performed in CHD patients from primary care settings [29] and is similar to the 12-month prevalence of 4% to 7% in the communities, as previously reported [31,32]. Patients in the remaining 7 studies were from hospitals or clinics for cardiovascular rehabilitation.

The optimal cut-off value is another important factor in the comparisons of the accuracy among various measures. For PHQ-9, the optimal cut-off value of 10 for the identification of MDD was consistent in 2 included studies [15,33]. A study [29] on the PHQ-9 indicated that a lower cut-off value (≥8) resulted in an increased sensitivity with only modest reduction in the specificity when compared with the recommended cut-off value (≥10) [9]. However, it assessed the depressive disorder, but not MDD.

Some optimal cut-off values were lower than generally recommended, particularly in the screening for MDD. Lowering the cut-off value substantially improves the sensitivity of these tools while retaining the specificity, thereby improving their usefulness in screening for depression in CHD patients. In the study of Haddad et al., results showed that the performance of HADS-D at a standard cut-off value (≥8) [34] was weaker, with a sensitivity of 53% and specificity of 91%, a large proportion of depression patients were not diagnosed under this condition, and the satisfactory performance was found when the cut-off value was 5 or above (≥8) with the sensitivity of 81.3% and specificity of 76.7% [29]. However, there were no subgroups in the assessment of MDD and the accuracy of HADS-D was not evaluated. The other 3 studies on HADS-D showed that the recommended cut-off value was 5 in 2 studies [15,35] and 4 in 1 study [28]. The sensitivity of 77%, 86%, and 87%, respectively, and the specificity of 69%, 75%, and 75%, respectively, were comparable among 3 studies.

For BDI-II, cut-off values were different in 3 included studies. The recommended cut-off value for BDI-II (≥14) was used in the studies of Bunevicius et al. [35] and Frasure-Smith et al. [36], and their results showed it was effective to screen MDD with good sensitivity (89% and 74%, respectively) and specificity (91% and 78%, respectively). In the study of Huffman et al. [37], the cut-off value of 14 resulted in sensitivity of 88.2% and specificity of 84.2%. However, a BDI-II score of ≥16, which had equivalent sensitivity (88.2%) and better specificity (92.1%), was recommended for MDD. Both cut-off values resulted in very few cases with a false-negative (NPC=98.13% for 16 and NPV=97.96 for 14), which is important for a good screening tool. Using a cut-off of 16, 62.5% of patients with a positive result in screening were found to have MDD, and the cut-off value of 14 had a lower PPV (45.45%).

Depression is under-recognized in CHD patients because healthcare providers rarely use standardized screening tools [22] and there is no consensuses which of the available screening tools should be used. In 2008, the AHA recommended systematic screening using PHQ-2 for depression in all CHD patients [6]. However, this guideline is challenged because there remains a paucity of evidence that systematic screening for depression is helpful to improve the outcomes of CHD patients [38–40]. Patients with false-positive results in the screening are at increased risk for receiving unnecessary anti-depressive treatment. After effective screening and accurate diagnosis, appropriate referral is also important. Studies have demonstrated that although primary providers can provide effective therapies without referral for up to 75% of patients with depression, most patients are unrecognized or inappropriately treated [41]. After appropriate referral, psychosocial interventions may improve the physiological function [42] and decrease the cardiovascular morbidity and mortality in CHD patients [43,44].

There are several limitations in this systematic review. First, meta-analysis was not performed to assess the pool-statistic such as sensitivity, specificity, and predictive values because of the small number of each tool in studies that met the inclusion criteria. Second, studies included in this review only targeted the identification of MDD rather than minor depression, depression syndrome, and depression of different severities. Apart from MDD, minor depression also has an influence on the morbidity and mortality of CHD patients. Third, there are different semi-structured methods used to determine the interview-based diagnosis, including CIS-R, MINI, ACID, and DIS, all of which have different diagnostic accuracies. Another limitation is the lack of cost-effectiveness analysis in the identification of MDD, and thus whether the cost also influences the false-positives of screening tools is still unclear.

Conclusions

In the absence of systematic screening to recognize (and thus treat) CHD is difficult for the front-line clinicians in the in-patient and primary case settings. To the best of our knowledge, this is the first systematic review in which various depression screening tools were compared with the diagnostic criteria in identification of MDD among CHD patients. In our study, PHQ-9 (≥10), BDI-II (≥14 or ≥16), and HADS-D (≥5 or ≥4) are widely used to screen MDD in CHD patients. When the performance of a screening tool is evaluated, the high sensitivity and NPV should be balanced with the high specificity and PPV, which will provide useful guidance for the application of appropriate tools and optimal cut-off value in the identification of depression in CHD patients. Taking into account psychometric properties and ease of use, effective screening tools should be integrated into clinical care. After screening, further diagnosis, appropriate management, and necessary referral may also improve cardiovascular outcomes.