Validation of Two Depression Screening Instruments in a Sleep Disorders Clinic

Abstract

Study Objectives:

Depression is a commonly diagnosed comorbidity in sleep disorder clinics. However, screening instruments for major depressive episode (MDE) have not been validated in this setting. We aimed to validate the Hospital Anxiety and Depression Scale (HADS) and the Beck Depression Inventory – Fast Screen (BDI-FS) with the Mini International Neuropsychiatric Interview (MINI) in patients with suspected obstructive sleep apnea (OSA).

Design:

Cross-sectional study.

Setting:

Academic center.

Participants:

One hundred one new patients with a clinical suspicion of OSA, as assessed by a sleep physician.

Measurements:

MDE, generalized anxiety disorder (GAD), and panic disorder (PD) were assessed by (1) a diagnostic interview utilizing the MINI and (2) by two self-report questionnaires: HADS and BDI-FS. A receiver operating characteristic (ROC) analysis was undertaken to assess which HADS and BDI-FS threshold yielded the highest correlation for a diagnosis of MDE and/or GAD/PD as assessed with an interview conducted using the MINI.

Results:

A HADS-Depression score ≥ 8 gave optimal sensitivity (83.1%) and specificity (83.3%) with an area under the ROC curve (AUC) 0.851 for predicting the diagnosis of MDE. A HADS-Anxiety score ≥ 11 gave an optimal sensitivity (93.1%) and specificity (84.7%) with an AUC 0.911 for predicting the diagnosis of GAD/PD. A BDI-FS threshold ≥ 6 gave optimal sensitivity (86.7%) and specificity (82.9%) with an AUC 0.897 for MDE.

Conclusion:

The HADS and BDI-FS are accurate screening instruments with high concurrent validity for identifying the probability of a patient having MDE and—in the case of HADS—GAD and PD disorder in a sleep disorders clinic.

Citation:

Law M, Naughton MT, Dhar A, Barton D, Dabscheck E. Validation of two depression screening instruments in a sleep disorders clinic. J Clin Sleep Med 2014;10(6):683-688.

In a sleep disorders clinic, both depressive and anxiety disorders are important to correctly diagnose as they are common, disabling, and their symptoms include fatigue and sleepiness, which can complicate the specific symptom of “excessive daytime sleepiness” and thus the diagnosis of the sleep apnea “syndrome.”^1,2 Moreover, anxiety is an important cause of insomnia and often coexists with obstructive sleep apnea (OSA).³

A large epidemiological survey of the US population recently reported a strong association between sleep apnea and probable depression.⁴ Furthermore, Douglas et al. found that 25% of patients referred with possible sleep apnea had an MDE, and an additional 25% may be at risk while not having a formal diagnosis.⁵ The reported lifetime prevalence rates of major depression and anxiety disorders in sleep clinics vary widely, with major depression in 7% to 63%, and anxiety disorders in 11% to 70%.³ This is compared with a 16.6% lifetime prevalence of MDD and 28.8% lifetime prevalence of any anxiety disorder in the general community.⁶

The wide range of reported prevalence in sleep clinics may be attributed to (1) the type of screening instrument used; (2) the threshold of the instrument and whether it makes a validated diagnosis of MDE, an anxiety disorder, or just measures cross-sectional severity; and (3) the clinic population and bias of referral base.

BRIEF SUMMARY

Current Knowledge/Study Rationale: This study as performed be cause depression and anxiety are frequently diagnosed comorbidities in sleep disorder clinics. Despite this, commonly used screening instruments for depression and anxiety have not been validated in this setting.

Study Impact: This study found that the HADS and BDI-FS were robust screening instruments with high concurrent validity. These devices may be rapidly administered to ensure more frequent and accurate detection of psychopathology and help identifying at-risk patients who require further psychiatric evaluation and treatment.

Two commonly used depression screening instruments that are self-administered and sensitive to change are the Hospital Anxiety and Depression Scale (HADS) and Beck Depression Inventory – Fast Screen (BDI-FS).^7,8 They have both been validated in a range of illness groups including primary care, adolescent, geriatric, multiple sclerosis, and oncology to be accurate screening instruments.^9–13 However, neither has been validated in a sleep clinic population. The purpose of this study was to validate both the HADS and BDI against the MINI in patients referred with possible sleep apnea.

METHOD

Sample Selection

Patients were consecutively recruited from the Alfred Sleep Disorders Service outpatient clinic. All new referrals with a clinical suspicion of OSA who were ≥ 18 years of age and who were cognitively able to complete the questionnaires were eligible for inclusion. All patients were assessed by a sleep physician accredited by the Royal Australasian College of Physicians. Patients were subsequently booked for an attended in-laboratory polysomnogram (PSG) or unattended home oximetry study. Patients who could not speak English and those who had previously had treatment for OSA were excluded from this study.

One-hundred forty-four consecutive patients were approached, with 101 patients completing the assessment. Thirty patients declined to participate, 10 were from non-English speaking backgrounds, and 3 had cognitive impairment. In total, 77.1% of eligible patients agreed to participate in this study. This study was approved by the institution's human research ethics committee.

Sleep Studies

Full channel in-laboratory PSGs were conducted using Compumedics E series equipment (Melbourne, Australia). The apnea-hypopnea index (AHI) was computed in accordance with the American Academy of Sleep Medicine alternative criteria.¹⁴ Overnight home oximetry was conducted using the Masimo Rad 5 oximeter (Masimo, Irvine, CA) with a 2-sec averaging time and 0.5-Hz signal acquisition. This center has previously been involved in a study where diagnosis and management of OSA by oximetry has been found to be similar to in-hospital PSG.¹⁵ A sleep nurse counselled patients as to the correct use of oximetry. Oxygen desaturation was defined as a decrease ≥ 3% from baseline arterial saturation,¹⁶ and the oxygen desaturation index (ODI; oxygen desaturations/h of recording time) was calculated (Download 2000, Stowood, UK). The results of attended PSGs and oximetry studies were scored by sleep scientists and reviewed and reported by sleep physicians. PSGs and oximetry studies were diagnostic for OSA if the AHI or ODI was > 5 per hour.¹⁴

Mini International Neuropsychiatric Interview (MINI)

The MINI was used for the diagnosis of major depression.¹⁷ It is a brief, structured, diagnostic interview for the major Axis I psychiatric disorders administered by a trained researcher to obtain diagnoses based on the diagnostic criteria from the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV). The MINI has been validated as a diagnostic interview^18,19 and has been used in validation studies across a variety of outpatient settings.^10,12,20 The interview sections for major depressive episode, panic disorder, and generalized anxiety disorder were administered. For the purposes of this study anxiety refers to the presence of generalized anxiety disorder (GAD), panic disorder (PD), or both.

Hospital Anxiety and Depression Scale (HADS)

The HADS is a self-report 14-item depression and anxiety screening instrument.⁷ Both subscales are designed to rate the severity of symptoms in medically ill patients, assessing psychological and cognitive symptoms of depression and anxiety while excluding somatic symptoms potentially attributable to comorbid medical conditions. Each question is scored from 0-3, with a total score up to 21 for each subscale.

Beck Depression Inventory – Fast Screen (BDI-FS)

The 7-item BDI-FS retains items from the affective subscale of the original Beck Depression Inventory (BDI).⁸ Similarly, the instrument only samples psychological and cognitive symptoms of depression. Each question is scored from 0-3, with a total score of up to 21.

Questionnaire and Interview Administration

All patients completed the HADS and BDI-FS screening instruments. Patients were then assessed with the MINI by the trained researcher, who was blinded to the subjects' HADS and BDI-FS scores. This occurred prior to any diagnostic sleep testing.

Statistical Analysis

A receiver operating characteristic (ROC) curve was performed for each scale (HADS-A, HADS-D, and BDI-FS) to determine the concurrent validity of the HADS and BDI-FS by calculating an optimal cutoff score.

All data have been reported as mean ± standard deviation (SD), or median (interquartile range [IQR]) as appropriate to tests of normality. Statistical significance was defined as p < 0.05 using two-tailed tests. Statistical analysis was performed using IBM SPSS Statistics Version 20.0 (IBM Corporation, Armonk, NY).

RESULTS

Sample Characteristics

Across the sample, participants were middle-aged (49.2 ± 14.5), obese (BMI 33.0 ± 8.5), and mostly male (70.3%) (Table 1). The majority reported a history of snoring (92.1%) and mild subjective sleepiness, with a mean ESS score of 9.6 ± 5.5.

Table 1.

Sample characteristics (n = 101).

Ninety-six percent (n = 97) of recruited patients completed their scheduled diagnostic studies. Attended polysomnography was completed by 74% (n = 72) with a median AHI of 14.5 (7.1-32.8); 26% underwent home oximetry with a median ODI of 47.0 (24.0-76.1). There were no significant differences in baseline characteristics with the exception of the number of respiratory events per hour (AHI or ODI), and BMI (Table 2). Eighty-eight patients (91%) were diagnosed with OSA (AHI or ODI ≥ 5). Nine patients were diagnosed with simple snoring based upon polysomnography (AHI 0-5). Overall, this sample is representative of moderate-to-severe OSA. Four patients did not attend their diagnostic studies and were subsequently lost to follow-up.

Table 2.

Presence of major depressive episode, generalized anxiety disorder, and panic disorder, as defined by the HADS and BDI-FS, across OSA severity groups.

Psychiatric Data

Psychiatric data is summarized in Table 1. The prevalence of MDE and GAD or PD diagnoses were 29.7% and 28.7%, respectively. Fifty-five percent of those who had MDE also had an anxiety disorder diagnosis. GAD was present in 21.8% and panic disorder in 16.8% of participants, indicating these conditions were frequently comorbid. A past psychiatric history (including previous mood and anxiety disorder diagnoses, as well as psychotic disorders) was present in 55.4%, with 24.8% of all patients currently taking an antidepressant.

Of the 4 patients who did not return to complete their diagnostic testing, 3 had a history of depression. Two patients fulfilled the DSM-IV criteria for MDE, and the remaining 2 had both GAD and PD.

The percentage of patients diagnosed with MDE, GAD, and PD did not differ across the 2 groups who underwent home oximetry or attended in-laboratory PSG (p = 0.91 and p = 0.31, respectively). Table 2 displays the presence of MDE, GAD, and PD across OSA severity groups.

Internal Consistency and Correlation

The internal consistency was high for both scales. Cronbach α was 0.89 for the entire HADS scale, 0.85 for the HADS-A subscale, 0.80 for the HADS-D subscale, and 0.84 for the BDI-FS. All questions in the HADS A and D were worthy of retention, with no increase in α from deleting certain items. All items correlated with the total scale to at least a good degree (lowest r = 0.45, please see supplemental material). Similarly, all questions were worthy of retention in the BDI-FS questionnaire with all items correlating to at least a good degree (lowest r = 0.38). The question with the lower correlation pertained to suicidality and should be included for its clinical importance.

There was a close correlation between total HADS scores, individual HADS-D, HADS-A, and BDI-FS scores. The Pearson correlation between total HADS score and BDI FS was 0.78 (p = 0.01) and between HADS-D and BDI-FS was 0.73 (p = 0.01).

Psychometric Evaluation

Hospital Anxiety and Depression Scale

A threshold of HADS-Depression ≥ 8 gave optimal sensitivity (83.1%) and specificity (83.3%; Tables 3 and 4), with an area under the ROC curve of 0.851. A logistic regression analysis for HADS-Depression ≥ 8 as a predictor correctly identified the presence of MDE in 83.2% of cases.

Table 3.

Sensitivity and specificity from ROC analysis for HADS-Depression.

Table 4.

Sensitivity and specificity from ROC analysis for HADS-Anxiety.

For the anxiety diagnoses of GAD and PD, a threshold of HADS-Anxiety ≥ 11 gave a sensitivity and specificity of 93.1% and 84.7%, respectively, with an area under the ROC curve of 0.911. A logistic regression analysis for HADS-Anxiety ≥ 11 correctly identified 87.1% of GAD/PD cases.

For GAD on its own, a HADS-Anxiety ≥ 12 optimal gives a sensitivity 90.9% and specificity 87.3% with an AUC of 92.9. For PD a threshold of HADS-Anxiety ≥ 11 gives a sensitivity 94.1% and specificity 73.8% with an AUC of 83.7.

Beck Depression Inventory – Fast Screen

A threshold of BDI-FS ≥ 6 was shown in Table 6 to give a sensitivity of 86.7% and specificity of 82.9%. This optimum cutoff score correctly identified depression in 84.0% of cases in a logistic regression analysis with BDI-FS ≥ 6 as a sole predictor, with an area under the ROC curve of 0.897 (Table 5).

Table 6.

Performance of the scales by OSA severity groups.

Table 5.

Sensitivity and specificity from ROC analysis for BDI-FS.

The performance characteristics of both scales were analyzed as a function of sleep apnea severity. The AUC for HADS A and D was lowest in the subgroup with mild OSA (Table 6). The BDI-FS had a reduced AUC in the subgroup with moderate OSA. The small numbers in each group make further interpretation of this difficult.

Predictive Value

A HADS-D score ≥ 8 has a positive predictive value (PPV) of 67.57% CI (50.21-81.97) and a negative predictive value (NPV) of 92.19% CI (82.69 - 97.39). Similarly, a HADS-A score ≥ 11 has a PPV of 71.05% CI (54.09-84.56) and a NPV of 96.83% CI (88.98-99.52). A BDI-FS score ≥ 6 has a PPV of 68.42% CI (51.35-82.48) and NPV of 93.55% CI (84.28-98.17). Both the HADS and the BDI-FS questionnaires perform very well in excluding MDE, GAD, and PD and reasonably well in confirming the presence of these diagnoses.

DISCUSSION

This study, with a sample size comparable to similar studies in specialist clinics,^20–22 found that the HADS and BDI-FS may be useful screening instruments in an OSA sample. These instruments demonstrated a high concurrent validity for identifying the probability of the presence of MDE, GAD, and PD. MDE and GAD and/or PD were found to be common with a prevalence of 29.7% and 28.7%, respectively.

A number of widely accepted self-rating depression and anxiety scales are available; however, many sample somatic symptoms of depression and may spuriously inflate scores. For instance, sleep disturbance, fatigue, impaired concentration, and poor libido may be attributed to depression, sleep disorders, or both. The HADS and BDI-FS do not sample somatic confounders.

The HADS and BDI-FS satisfy the World Health Organization requirements for a “good screening test.” Wilson et al. stipulated that the condition should be of public health importance and that treatment should be available.²³ Both of these are true for depression and anxiety. Furthermore, the screening instruments are quick, cheap, acceptable to patients, and easy to administer.

The prevalence of MDE or GAD/PD was approximately 30%, far lower than the 55.4% reporting a past psychiatric history. This distinction is likely to be multifactorial, with the high prevalence of obesity a potential factor and the possibility of misclassification and recall bias.

We accept that by using criteria for a “major depressive episode” as opposed to “mild depressive symptoms” may have in fact raised the BDI-FS cutoff score to 6. Unfortunately the MINI does not include severity ratings for MDE; nonetheless, rating the severity of depression could have been completed and this would have added to the data.

Hospital Anxiety and Depression Scale (HADS)

Our data indicate that a HADS-Depression score ≥ 8 is the most effective threshold for identifying the high probability of the presence of MDE with good sensitivity and specificity, which is comparable to those found in other illness groups.^10,21,22

We validated a HADS-Anxiety score ≥ 11 to be the optimal threshold to screen for anxiety disorders including GAD and panic disorder. Again, sensitivity and specificity were comparable to other clinical populations.^24–26

The cutoff score of ≥ 11 is higher than recommended in the original HADS article. This may be because data on social phobia, obsessive-compulsive disorder, and posttraumatic stress disorder diagnoses were not collected due to interview time constraints.

Beck Depression Inventory – Fast Screen (BDI-FS)

The BDI-FS performed as well as the HADS-Depression subscale, with similar discriminating abilities in identifying MDE. We identified a BDI-FS score ≥ 6 which is a higher threshold score than ≥ 4, which was recommended by Beck et al. in the original manuscript.⁸ Some have validated the BDI-FS at a score of 4 in specific subgroups.^27,28 Our results find that using a cutoff of 4 reduces the specificity to 67.1%.

Overnight Oximetry

Twenty-five percent of subjects in this study underwent home oximetry as a diagnostic modality. We do not believe this to be a significant limitation. The diagnosis and management of OSA by oximetry has been found to be comparable to attended in-laboratory PSG.¹⁵ Subjects who underwent home oximetry had higher BMIs and more severe sleep apnea, as this modality is reserved for patients with a high pre-test probability of moderate to severe OSA. Furthermore, HADS-Anxiety, HADS-Depression, BDI-FS scores, and the percentage of MINI diagnosed MDE, GAD, or PD did not differ between these two groups.

Limitations

Due to time constraints during the sleep clinic we were only able to administer some sections of the MINI. Accordingly, we do not have data on the full range of anxiety diagnoses from the MINI psychiatric interview. Therefore our results regarding the validity of the HADS-Anxiety subscale are confined to GAD and PD. Moreover, we were only able to make a diagnosis of MDE and not MDD, as major depressive disorder requires two or more major depressive episodes.

Further validation studies including all sections of the MINI would provide clearer evidence; however, our results remain relevant for the two most common anxiety disorders and MDE. Four patients did not return to complete their diagnostic studies. Patients with psychiatric illnesses are often lost to follow-up in the community and may explain non-attendance.³⁰ Two of the non-attendees were diagnosed with a current episode of MDE, one with GAD, and another with panic disorder. We chose to include their data in final analyses so as to avoid selection bias.

The MINI includes sleep and fatigue related symptoms for the diagnosis of MDE and GAD. These questions could not be omitted, as the MINI is only valid when sections are administered in full. Thus the MINI may overestimate the frequency of MDE and GAD in this sample.

In conclusion, this study found that the HADS and BDI-FS were robust screening instruments, with high concurrent validity, for identifying the probable presence of MDE, GAD, and PD in an OSA sample. While self-reporting questionnaires such as the HADS and BDI-FS cannot generate psychiatric diagnoses, they are simple devices, acceptable to the population, and may be rapidly administered to ensure more frequent and accurate detection of psychopathology. We suggest that the use of these screening instruments is valuable, and may be used to optimize management in these patients, aiding clinicians in identifying at-risk patients who require further psychiatric evaluation and treatment.

DISCLOSURE STATEMENT

This was not an industry supported study. The work was performed at Alfred Hospital, Prahran, Victoria Australia. The authors have indicated no financial conflicts of interest.

SUPPLEMENTAL DATA

HADS Combined

Item-Total Statistics

Inter-Item Correlation Matrix

HADS Depression

Item-Total Statistics

Inter-Item Correlation Matrix

HADS Anxiety

Item-Total Statistics

Inter-Item Correlation Matrix

BDI-FS

Inter-Item Correlation Matrix

Item-Total Statistics