Examining brief and ultra-brief anxiety and depression screening methods in a real-world epilepsy clinic sample

Abstract

Objective:

Recent epilepsy quality measure recommendations for depression and anxiety screening endorse ultra-brief screeners, the Patient Health Questionnaire-2 (PHQ-2) and Generalized Anxiety Disorder-2 (GAD-2). Thus, it is important to assess how symptom detection may be affected by using ultra-brief screeners compared with slightly longer, well-validated instruments: Neurological Disorders Depression Inventory-Epilepsy (NDDI-E) and Generalized Anxiety Disorder-7 (GAD-7). The objective was to compare symptom detection by brief versus ultra-brief depression and anxiety screeners in a large real-world epilepsy clinic sample.

Methods:

This was a prospective, cross-sectional assessment of consecutive patients in an adult tertiary epilepsy practice who completed the GAD-7 and NDDI-E with embedded ultra-brief scales (GAD-2; GAD-Single Item: GAD-SI; NDDI-E 2 item: NDDIE-2) on a tablet and had clinic staff administered ultra-brief PHQ-2 (yes/no version) documented in the medical record at the same visit. Prevalences of positive anxiety and depression screens were calculated for each instrument overall, and by epilepsy status. Concordance correlation coefficients (CCC) were calculated comparing the ultra-brief with brief anxiety and depression instruments, and receiver operating curves (ROC) were calculated using the longer instruments as alternative standards.

Results:

Among N=422 individuals the prevalence of positive anxiety screen by GAD-7 was 24% and positive depression screen by NDDI-E was 20%. Positive anxiety and depression screens were significantly less prevalent among seizure free individuals than those with continued seizures. The verbally administered yes/no PHQ-2 had only 1 positive screen (0.2%). Other than poor concordance between the PHQ-2 and NDDI-E, the screener pairs had acceptable concordance (CCC 0.79 to 0.92). Areas under the ROC curves were acceptable for the NDDIE-2, GAD-2 and GAD-SI (0.96, 0.98, 0.89 respectively).

Significance:

In this sample, clinic staff interview-administered yes/no PHQ-2 had exceedingly low sensitivity compared with the NDDI-E self-reported on a tablet. Further investigation is warranted to assess if poor detection is due to characteristics of this PHQ-2 in epilepsy samples, or method of administration in this clinic. The other ultra-brief anxiety and depression instruments demonstrated good concordance with the longer, well-validated instruments and may be useful in clinical practice.

1. Introduction

Anxiety and depression in epilepsy are associated with numerous poor outcomes, yet they are under-detected and undertreated[1]. Recently published epilepsy quality measures recommend screening for depression and anxiety at every epilepsy visit and specifically endorse use of the ultra-brief screeners Patient Health Questionnaire-2 (PHQ-2) and Generalized Anxiety Disorder-2 (GAD-2)[2]. The US Preventive Services Task Force also recommends routine screening for depression with instruments such as the PHQ-2[3]. In light of these recommendations, it is important to assess how symptom detection among people with epilepsy may be affected by use of ultra-brief screeners compared with slightly longer instruments that have been better-validated in epilepsy, such as the Neurological Disorders Depression Inventory-Epilepsy (NDDI-E) and Generalized Anxiety Disorder-7 (GAD-7)[1, 4, 5]. Also, other ultra-brief anxiety and depression screening instruments have recently been validated in epilepsy, the Neurological Disorders Depression Inventory Epilepsy-2 (NDDIE-2) and the single-item GAD (GAD-SI), but symptom detection with these instruments has not been compared with longer instruments in a real world epilepsy clinic sample[6]. If ultra-brief screeners yield similar anxiety and depression symptom detection compared with the longer screeners, use of ultra-brief screeners would be particularly advantageous to address time constraint barriers to screening in routine epilepsy clinic care.

The primary objective of this study was to compare detection rates of brief (NDDI-E; GAD-7) versus ultra-brief (PHQ-2, NDDIE-2; GAD-2, GAD-SI) depression and anxiety screeners administered in a large real-world epilepsy clinic sample.

2. Methods

Data is from a prospective, cross-sectional assessment of consecutive patients in the clinics of three adult epileptologists from April 30, 2018 to June 6, 2019. Patients completed tablet-based anxiety and depression instruments as part of a combined process for clinical care and eligibility assessment for a pragmatic trial of anxiety and depression treatment in epilepsy[7]. Inclusion criteria for this analysis were the following completed instruments for the first time in the study period [interval-valued score range, from less to more severe]: 1. Generalized Anxiety Disorder-7 (GAD-7; [0–21]) and Neurological Disorders Depression Inventory-Epilepsy (NDDI-E; [6–24]) by self-report on a tablet and 2. Clinic staff administered Patient Health Questionnaire-2(PHQ-2) results documented in the medical chart from the same visit date (yes/no response version; [0–2])[8]. The NDDI-E and GAD-7 (along with the embedded ultra-brief scales, the NDDIE-2 [2–8], GAD-2 [0–6], GAD-SI [0–3]) were completed by patients on a tablet at routine epilepsy clinic visits[7], and the PHQ-2 was administered verbally at the same visit by nursing staff and documented in discrete data fields in the electronic health record as a routine care process. Demographics, epilepsy diagnosis, epilepsy type[9], and whether participants were seizure free for the prior 6 months were collected by chart abstraction. Data collection via medical chart abstraction and prospective collection of the GAD-7 and NDDI-E as part of pragmatic trial screening (NCT03464383) were approved by the institutional review board, with waiver of informed consent for the clinical care data.

2.1. Analyses

Based on the original validation literature for each instrument, the following scores were considered a positive anxiety screen: GAD-7≥10, GAD-2>2, GAD-SI>1[6, 10]. Positive depression screens were the following: NDDI-E>15, NDDIE-2>4, PHQ-2>0 (response of yes to either PHQ-2 question)[6, 8, 11]. Prevalence of positive screens and corresponding 95% confidence intervals were calculated for each instrument in the study sample overall and stratified by gender, epilepsy diagnosis status, and seizure freedom status. Differences in prevalence between seizure-free and non-seizure- free groups were tested for statistical significance using Fisher’s exact test; p values ≤ 0.05 were considered statistically significant. To compare brief with ultra-brief anxiety and depression instruments, the concordance correlation coefficient (CCC) was calculated between anxiety scores, and between depression scores, after rescaling each score to the possible range of 0 to 100 through addition and multiplication by constants. The CCC is a measure of association for non-binary measures that ranges from −1 (perfect negative association) to 1 (perfect positive association), similar to commonly-used correlation coefficients (such as Pearson correlation coefficient), but the CCC has an additional advantage: it evaluates both precision and accuracy, by measuring the distance of observations from a 45° line of perfect concordance[12].

Receiver operating characteristic curves were also calculated for each ultra-brief screening instrument, with the longer instrument (GAD-7 and NDDI-E for anxiety and depression, respectively) used as the alternative standard in the absence of an observed “gold standard” in the study. Prior studies have used well-validated scales such as the GAD-7 and NDDI-E as alternatives to a true gold standard psychiatric interview for this type of analysis[13–15].

3. Results

3.1. Epilepsy and demographic characteristics of sample

A total of 422 individuals met inclusion criteria, including 232 women (55%), and the mean age of the sample was 45.0 years (SD 17.5, range 17–101). The sample had the following race/ethnicity distribution: 80.8% (N=341) non-Hispanic/Latino Caucasian, 12.6% (N=53) non-Hispanic/Latino Black, 4.7% (N=20) Hispanic/Latino, and 1.9% (N=8) other or unknown. Most of the sample had a diagnosis of epilepsy (86.3%, N=364), though 9.2% had an alternative primary diagnosis (N=39), and for some the diagnosis was uncertain (4.5%, N=19). Among the 364 individuals with epilepsy, most had only focal epilepsy (72.5%, N=264), while 19.2% had only generalized epilepsy, 7.4% had unknown type, and 0.8% had both focal and generalized epilepsy (N=70, 27, and 3 respectively). About half of those with epilepsy were seizure free over the 6 months prior to completing the screening measures (51.6%, N=188), and seizure status over the prior 6 months was uncertain for 4.1% (N=15).

3.2. Depression and anxiety scores by instrument

The mean GAD-7 score was 5.87 (SD 5.85) and the mean NDDI-E was 11.6 (SD 4.4). Nearly one-third (31.8%, N=134) of the overall sample screened positive for either anxiety or depression based on the GAD-7 and NDDI-E (13.0% both anxiety and depression, 11.4% anxiety alone, 7.3% depression alone). Means and standard deviations for the ultra-brief instruments were the following: PHQ-2 0.0047, SD 0.0974, range 0–2; NDDIE-2 3.27, SD 1.41; GAD-2 1.83, SD 1.93; GAD-SI 0.893, SD 1.04. The maximal NDDI-E score of 24 was not present in the sample, and there were no scores of 1 on the yes/no version of the PHQ-2, but otherwise the full range of potential scores was observed for all instruments. Based upon original validation cut points, the prevalence of positive anxiety screens in the study sample ranged from 23% to 31% across the various instruments (Table 1), and both the NDDI-E and NDDIE-2 scores demonstrated 20% prevalence of positive depression screens in the sample. Among the 422 individuals, only one positive screen (score of 2) was documented via the yes/no PHQ-2 (0.2%).

Table 1:

Prevalence estimates for positive anxiety and depression screens & 95% CI by instruments^* and epilepsy status

		Diagnosis of Epilepsy		Seizure Free in Past 6 Months**
Instrument (positive values)	Overall (N = 422)	No/Uncertain (N = 58)	Yes (N = 364)	Uncertain (N = 15)	Yes (N = 188)	No (N = 161)	P Value***
GAD-7 (≥ 10)	0.24 0.21–0.29	0.22 0.14–0.35	0.25 0.21–0.30	0.40 0.20–0.64	0.17 0.12–0.23	0.32 0.26–0.40	0.0011
GAD-2 (> 2)	0.31 0.27–0.35	0.35 0.24–0.47	0.30 0.26–0.35	0.47 0.25–0.70	0.20 0.15–0.27	0.40 0.33–0.48	<0.0001
GAD-SI (≥ 2)	0.23 0.19–0.27	0.31 0.21–0.44	0.21 0.18–0.26	0.33 0.15–0.58	0.18 0.13–0.24	0.25 0.19–0.32	0.11
NDDI-E (≥ 16)	0.20 0.17–0.25	0.26 0.16–0.38	0.20 0.16–0.24	0.33 0.15–0.58	0.14 0.096–0.20	0.25 0.19–0.32	0.0094
NDDIE-2 (> 4)	0.20 0.16–0.24	0.28 0.18–0.40	0.18 0.15–0.23	0.40 0.20–0.64	0.11 0.070–0.16	0.26 0.19–0.33	0.0004

Confidence interval, CI; Generalized Anxiety Disorder-7, GAD-7; Generalized Anxiety Disorder-2, GAD-2; Single-item GAD, GAD-SI; Neurological Disorders Depression Inventory-Epilepsy, NDDI-E; Neurological Disorders Depression Inventory Epilepsy-2, NDDIE-2

^*PHQ-2(Patient Health Questionnaire-2) omitted, as there was only 1 positive screen across 422 individuals and thus inclusion in this table would be uninformative.

^**Among those with diagnosis of epilepsy (N=364)

^***P values based on Fisher exact test for comparison of seizure free and non-seizure free subgroups; there were no significant differences when those with a diagnosis of epilepsy were compared to those without an epilepsy diagnosis

3.3. Prevalence of positive depression and anxiety screens by demographic and clinical characteristics

Table 1 shows prevalence and 95% CI for positive anxiety or depression screens by instrument, epilepsy diagnosis, and seizure free status. The prevalence of positive anxiety and depression screens for the GAD-7, GAD-2, NDDI-E and NDDIE-2 instruments was significantly higher among those with persistent seizures compared with seizure free individuals. Prevalence of positive anxiety or depression screen did not demonstrate any notable differences when calculated for each instrument by sex and race/ethnicity categories.

3.4. Examining instrument correlations

Table 2 demonstrates the concordance correlation coefficients for each pair of anxiety and depression screening instruments. The concordance coefficients were acceptable (ranging from 0.72 to 0.92) for the anxiety instruments and the NDDI-E and NDDIE-2 instruments. Correlation of the verbally administered yes/no PHQ-2 with the other two depression instruments was exceptionally poor (CCC<0.01). Coefficients and 95% confidence intervals were also examined by epilepsy diagnosis and seizure free status; the coefficients were similar regardless of epilepsy or seizure status.

Table 2.

Anxiety and depression screener concordance correlation coefficients & 95% confidence intervals

Instruments	Concordance coefficient	95% Confidence Interval
GAD-7, GAD-2	0.92	0.91–0.94
GAD-7, GAD-SI	0.81	0.78–0.84
GAD-2, GAD-SI	0.72	0.67–0.76
NDDI-E, NDDIE-2	0.79	0.76–0.82
NDDI-E, PHQ-2	0.001	−0.014–0.015
NDDIE-2, PHQ-2	−0.01	−0.031–0.012

Generalized Anxiety Disorder-7, GAD-7; Generalized Anxiety Disorder-2, GAD-2; Single-item GAD, GAD-SI; Neurological Disorders Depression Inventory-Epilepsy, NDDI-E; Neurological Disorders Depression Inventory Epilepsy-2, NDDIE-2; Patient Health Questionnaire-2 (yes/no response version), PHQ-2

3.5. Receiver operating curves (ROC) of ultra-brief instruments compared to alternative standard (brief instrument)

Figure 1 demonstrates ROC curves for the GAD-2, GAD-SI, and NDDIE-2 relative to the alternative standards (GAD-7 and NDDI-E). As specifically outlined in Table 3, the GAD-2, GAD-SI, and NDDIE-2 at the original validated cutpoints had acceptable sensitivity and specificity relative to the alternative standard. For the NDDIE-2, an alternative cutpoint of >3 had area under the curve equivalent to the original validation cutpoint of >4 (Table 3). Sensitivities, specificities, and the corresponding 95% confidence intervals were examined by sex, epilepsy versus no epilepsy, and seizure free status: there were no notable differences in accuracy of the ultra-brief instruments relative to alternative standard by these factors.

Figure 1:

Receiver operating characteristic curves for ultra-brief anxiety (A) and depression (B) instruments

Table 3.

Anxiety and depression screening sensitivity, specificity & 95% CI estimates by instrument (n=422)

Instrument, positive values (reference)	AUC	Sensitivity	Sensitivity 95% CI*	Specificity	Specificity 95% CI*
GAD-2, > 2 (GAD-7, ≥ 10)	0.98	0.95	0.89–0.98	0.90	0.86–0.93
GAD-SI, ≥ 2 (GAD-7, ≥ 10)	0.89	0.72	0.62–0.80	0.93	0.90–0.95
NDDIE-2, > 4 (NDDI-E, ≥ 16)	0.96	0.78	0.68–0.85	0.95	0.92–0.97
NDDIE-2, >3 (NDDI-E, ≥ 16)	0.96	1.0	0.96–1.0	0.80	0.76–0.84
PHQ-2, >0 (NDDI-E, ≥ 16)	0.50	0.0	0.0–0.043	1.0	0.98–1.0

Confidence interval, CI; Generalized Anxiety Disorder −7, GAD-7; Generalized Anxiety Disorder-2, GAD-2; Single-item GAD, GAD-SI; Neurological Disorders Depression Inventory-Epilepsy, NDDI-E; Neurological Disorders Depression Inventory Epilepsy-2, NDDIE-2

^*Confidence intervals are calculated on the basis of inverting the score test for a binomial proportion.

4. Discussion

In this large real-world epilepsy clinic sample, nearly one third of patients demonstrated positive anxiety or depression screens via standard, brief, validated self-report instruments, the NDDI-E and GAD-7. There was similar overall symptom detection via the embedded ultra-brief instruments (GAD-2, GAD-SI and NDDIE-2) as with the NDDI-E and GAD-7, and symptom detection was similar by sex, race/ethnicity groups, and among those with and without a diagnosis of epilepsy. Similar to prior studies, this investigation demonstrated significantly lower prevalence of positive anxiety and depression screens among those who were seizure free versus not seizure free, across most instruments examined (Table 1)[16].

Perhaps the most striking finding in this analysis is the exceptionally low rate of positive depression screens (0.2%) by the nursing staff verbally-administered, electronic health record-documented yes/no PHQ-2 in this sample, despite the 20% positive screens by tablet self-reported NDDI-E in this same population on the same date. Margrove et al. found similar rates of positive PHQ-2 (35.5 %) and NDDI-E (39.5%) scores in patients with epilepsy in primary care who completed mailed paper instruments, though this study used the PHQ-2 version with 4 responses per question, not the yes/no response version[17]. Differences in collection modalities, PHQ-2 version, or different study sample characteristics may account for the distinct findings in our study. A validation study in epilepsy of the 4 response per question PHQ-2 demonstrated only 42% sensitivity using the gold standard Structured Clinical Interview for the DSM-IV (SCID)[18], literature on the PHQ-2 yes/no version in epilepsy is scarce or nonexistent, and general population literature has demonstrated limitations in the sensitivity of the PHQ-2 [19]. A primary care study at our institution demonstrated an increase in depression detection using the yes/no PHQ-2 instrument from 1% to 14% when administration method changed from staff-initiated verbal screening (method used in our current study for PHQ-2 administration) to tablet self-report[20]. It is possible that patients may be more willing to report symptoms via tablet or paper self-report than nursing staff interview, or there may potentially be validity problems in the method of real-world administration by staff in clinical practice, such as failure to verbally deliver exact question wording in busy clinics, or other inconsistencies in instrument administration.

4.1. Strengths and limitations

While the most novel aspects of this study include examination of symptom detection by specific ultra-brief instruments rarely assessed in the epilepsy literature, and comparison of these detection rates with robustly validated brief instruments in a large real-world practice sample (rather than a more selected research sample), there are notable limitations. The most significant is the lack of a true gold standard and use of the GAD-7 and NDDI-E as alternative standard in the ROC analyses. Prior studies have used a validated scale as an alternative standard when it was not possible to conduct a gold standard evaluation, including studies examining the PHQ-2 and using the related PHQ-9 as reference standard, when assessing the performance of the ultra-brief test at different cut points in a new clinical population[13–15]. Our analysis achieves a similar goal of assessing the detection of symptoms by ultra-brief instruments in a new type of population (real world consecutive clinic sample) and comparing with well-vetted validated instruments that are feasible to obtain in this type of sample (GAD-7 and NDDI-E). Also related to the lack of gold standards, we acknowledge our chosen association measure, the concordance correlation coefficient (CCC), though it has several desirable characteristics, does not account for a level of similarity as expected between instruments having items in common. Gold standard psychiatric diagnostic interviews are impractical to obtain in an unselected routine epilepsy clinic sample. Another limitation is that the yes/no PHQ-2 results were collected based on verbal instrument administration by nursing staff and the related data entry by nursing staff into discrete fields of the electronic health record (EHR). As only the EHR-based results documented by nursing staff were available for analysis in this study, the exact method of interview administration (eg. whether the PHQ-2 questions were administered verbatim in the nursing staff interview consistently in this sample) is unknown. Nevertheless, the PHQ-2 results documented in the EHR in this sample reflect the real-world depression detection via the routine care PHQ-2 screening methods in this clinic, and thus they are clinically relevant to care practices in this setting, and potentially more broadly.

4.2. Conclusions and future directions

This study demonstrated similar anxiety and depression symptom detection by the ultra-brief and brief anxiety and depression instruments examined, except for the nursing staff verbally administered yes/no version of the PHQ-2, which demonstrated extremely poor depression detection compared with the tablet-administered NDDI-E. Further research is warranted to assess the validity of this PHQ-2 version in epilepsy and potential impact of administration method in routine care settings. The results suggest the other ultra-brief instruments may be suitable for efficient screening in routine practice.

Highlights.

• Ultra-brief anxiety and depression screeners were compared to longer instruments

• For depression screening, NDDIE-2 demonstrated good concordance with NDDI-E

• GAD-2 and GAD-SI demonstrated good concordance with GAD-7 for anxiety screening

• Compared to tablet-report NDDI-E, nursing interview PHQ-2 yes/no had poor detection