Screening Tools for Anxiety in People with Multiple Sclerosis

Abstract

Background: Anxiety is prevalent in people with multiple sclerosis (MS). Screening measures are used to identify symptoms of anxiety, but the optimal measure to screen for anxiety disorders in MS has not been established.

Methods: We searched the MEDLINE, Embase, PsycINFO, PsycARTICLES Full Text, Cumulative Index to Nursing and Allied Health Literature, Web of Science, and Scopus databases from database inception until August 7, 2015. Two independent reviewers screened abstracts and full-text reports for study inclusion, extracted data, and assessed risk of bias. We included studies that evaluated the criterion validity of anxiety screening tools when measuring anxiety in individuals with well-documented MS, as measured by sensitivity, specificity, and positive and negative predictive values.

Results: Of the 3181 abstracts screened, 18 articles were reviewed in full text, of which 4 met the inclusion criteria. The criterion validity of three screening tools was assessed: the Hospital Anxiety and Depression Scale–Anxiety (HADS-A), Beck Anxiety Inventory (BAI), and 7-item Generalized Anxiety Disorder Scale (GAD-7). The HADS-A was validated against the Structured Clinical Interview for DSM-IV, the Schedules for Clinical Assessment in Neuropsychiatry (SCAN) interview, and the BAI. The BAI was validated against the SCAN, and the GAD-7 was validated against the HADS-A. The HADS-A had higher measures of sensitivity and specificity than did the BAI and the GAD-7.

Conclusions: Based on this small sample, the HADS-A shows promise as an applicable measure for people with MS. Screening scales used to identify anxiety in MS must be validated against appropriate reference standards.

The rates of anxiety disorders are higher in people with a chronic medical condition compared with the general population,1 and levels of disability are higher in people with comorbid medical conditions and anxiety disorders.1,2 Anxiety is common in individuals with multiple sclerosis (MS); a recent systematic review estimated the population-based prevalence of anxiety to be 21.9%.3 Furthermore, the prevalence of generalized anxiety disorder (GAD),4,5 specific phobia,4 panic disorder,5 and obsessive-compulsive disorder4,5 are higher than in the general population. However, of those reporting anxiety symptoms, it is estimated that only 11.1% receive any form of treatment.6 Anxiety in MS populations is associated with social dysfunction,5 somatic complaints,5 chronic pain,6 fatigue,6 excessive alcohol consumption,5 and suicidal ideation.5,7 In addition, anxiety disorders may also reduce adherence to disease-modifying therapies.8 This can lead to increased morbidity and poorer quality of life.9 Comorbid anxiety can affect how well people with MS respond to treatment for depression, especially those with comorbid GAD.10 Fear disorders (panic disorder, agoraphobia, social phobia, and specific phobia) may also affect the maintenance of therapeutic gains after the cessation of depression treatment. Therefore, it is important to identify symptoms of anxiety so that treatment may target both depression and anxiety to provide long-term positive effects.

Symptoms of anxiety can be measured in numerous ways, including patient self-report, clinician interviews, medical records, and screening tools. Screening tools are typically used to identify elevated symptoms of a disorder that may require further evaluation for confirmation of a diagnosis. They are brief, standardized, and less resource-intensive than instruments that require health professionals for administration, scoring, or interpretation. Several screening tools for identifying anxiety have been well validated in the general population. However, their applicability has not been specifically assessed in chronic disease populations such as MS, and there is reason to believe that these tools may not perform as intended. Frequent symptoms of MS, such as tingling and dizziness, may overlap with the somatic symptoms of anxiety that are assessed in such screening tools,11 resulting in misclassification and consequent overestimates of anxiety presentations. This criterion contamination has been identified with depression screening tools in MS.12,13 Therefore, it is critical that screening tools are validated in the population in which they are administered. In this systematic review, we aimed to synthesize and appraise the existing literature regarding the concurrent criterion validity of screening tools for anxiety in people with MS.

Methods

This review was conducted according to an a priori published protocol14 following the approach detailed in the Cochrane Handbook for Systematic Reviews.15 The findings are reported according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) criteria.16 The primary research question was “What is the criterion validity of anxiety screening tools for use in persons with MS?” We defined criterion validity as a measure of the candidate measure, in this case an anxiety screening tool, against an external reference standard that accurately defines the presence or absence of the condition of interest, in this case, anxiety disorders. We measured criterion validity using the reported sensitivity, specificity, and positive and negative predictive values.

Study Population

Included studies 1) validated an anxiety tool against a reference standard and 2) were conducted in a population of individuals with MS, diagnosed according to the prevailing criteria used when the study was conducted. To maximize the number of studies available, there were no prespecified criteria about the study design, the anxiety tool validated, or the reference standard used.

Search Strategy

The search strategy was developed by the primary and senior authors (BL, KF, RAM) based on their collective experience in conducting systematic reviews and on their background knowledge of psychiatric disorders and MS, respectively (Supplementary Appendix 1 ^{(1.9MB, pdf)} , which is published in the online version of this article at ijmsc.org). The Cochrane Database of Systematic Reviews was searched for related systematic reviews, and the following databases were searched for original research: MEDLINE, Embase, PsycARTICLES Full Text, PsycINFO, Web of Science, Cumulative Index to Nursing and Allied Health Literature, and Scopus. Other potential abstracts were identified by reviewing the reference lists of the studies meeting the inclusion criteria for this review. All the databases were searched from database inception until August 7, 2015, with no language or date limits placed on the search.

Study Selection

Titles and abstracts were independently screened by two reviewers (BL and KMF) using EPPI-Reviewer software17 and a two-step process. In step 1, titles and abstracts were reviewed to determine whether they were validation studies in a population with individuals with MS. In step 2, the abstracts were identified as validation studies of anxiety tools. The same reviewers independently reviewed full-text articles included from the two-step abstract review; disagreements were resolved by consensus.

Data Extraction and Management

A data collection tool was developed by the author team and was implemented in EPPI-Reviewer; all data abstraction was completed in duplicate (BL and KMF). Information was extracted on participant inclusion criteria (eg, age range), summary demographic and disease characteristics (eg, sex, age, and disease course), the tool being validated (eg, length, type of anxiety assessed, and method of administration), the reference standard used to determine the presence or absence of the anxiety disorder, cut-points assessed (if any), performance of the tool being assessed (eg, sensitivity, specificity, kappa statistic, and area under the curve), and items related to the study quality assessment.

Study Quality

The two reviewers assessed the quality of included studies using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2)18 instrument. Domains relating to patient selection, the index test (measure), the reference test, and participant flow and timing were assessed for and categorized as either low risk, unclear risk, or high risk of bias. Applicability considerations include whether study participants match the review question, whether inappropriate exclusions were made, and whether application of the index test was consistent with the review question. The patient selection, index test, and reference test domains were also assessed for concerns relating to applicability as low, unclear, or high concern.

Data Analysis

Findings from all included studies were tabulated and summarized using descriptive statistics. Owing to the heterogeneity of included studies (n = 4) with respect to the screening tools validated and the reference standards used, we did not perform meta-analyses.

Results

Results of Search

The search yielded 3181 unique citations, of which 3163 were excluded at the title and abstract level because they did not use an MS population or they did not specifically aim to validate an anxiety screening tool (Figure 1). We retrieved 18 articles for full-text review, of which 14 were excluded. For data abstraction and analysis, we retained four studies19–22 that collectively evaluated three different anxiety instruments (Table 1).

Figure 1.

Study flow diagram

Table 1.

Characteristics of the anxiety instruments evaluated

Description of Studies

Details of the included studies are shown in Table 2. Three studies evaluated the Hospital Anxiety and Depression Scale–Anxiety (HADS-A),19,21,22 two examined the Beck Anxiety Inventory (BAI),21,22 and one evaluated the 7-item Generalized Anxiety Disorder Scale (GAD-7).20 Dates of publication ranged from 2001 to 2015. Study sample sizes ranged from 34 to 513. In each study, the mean ages of participants were similar, and each had a majority of females (Table 2).

Table 2.

Characteristics of the four included studies that evaluated the validity of anxiety tools in MS

Performance Characteristics of Screening Instruments

The results for the performance of the three anxiety measures are found in Table 3.

Table 3.

Performance characteristics of the anxiety instruments

Hospital Anxiety and Depression Scale

The HADS includes depression and anxiety subscales. The HADS-Anxiety (HADS-A) subscale was examined in three19,21,22 of the four studies using three different cut-points compared with two criterion standards: the Schedules for Clinical Assessment in Neuropsychiatry (SCAN)21 and the Structured Clinical Interview for DSM-IV (SCID).19 A further study validated the HADS-A against the BAI, another screening tool.22 The cut-points evaluated were 1) 7 or greater to identify any anxiety disorder,19 2) 8 or greater to identify GAD19 based on previously recommended values in general medical populations,21 and 3) 11 or greater based on optimum values calculated from a receiver operating characteristic (ROC) curve.21

Using a cut-point of 7, the HADS-A had at least 70% sensitivity and specificity compared with the SCID and an area under the curve (AUC) of 0.82119; it correctly classified 81.4% of all patients with anxiety.19 Using a cut-point of 8, sensitivity improved to 88.5% while retaining specificity (80.7%).19 A HADS-A cut-point of 8 had an AUC of 0.913 and correctly classified 88.6% of patients with GAD compared with the SCID.19 When using the recommended cut-point of 8, the HADS-A had a sensitivity of at least 90% and a specificity of 71% compared with the SCAN diagnoses of GAD and specific phobia (combined).21 It had an AUC of 0.94 and correctly identified 47% of all patients with anxiety compared with the SCAN.21 However, this cut-point produced a positive predictive value of 56% and a negative predictive value of 94%.21 Using the cut-point value of 11 calculated by Watson et al.21 as yielding the best balance between sensitivity and specificity, the HADS-A had a sensitivity of 90% and a specificity of 92% compared with the SCAN.21 This cut-point produced a positive predictive value of 82%, a negative predictive value of 96%, and an AUC of 0.94 for any anxiety disorder.21 Compared with the BAI (cut-point ≥16), the HADS-A with a cut-point of 7 had a sensitivity of 71% and a specificity of 63%, which was the optimal cut-point according to ROC analyses.22

Beck Anxiety Inventory

One study examined the BAI against SCAN21 diagnoses of GAD and specific phobia combined using two cut-point values: 1) 16 or greater based on previously recommended cut-point values in the general population21 and 2) 10 or greater based on optimum values calculated from a ROC curve.21 A second study examined the BAI against the HADS-A22 using a cut-point of at least 16 based on previously recommended values in the general population.22 With a cut-point value of 16 or greater, the BAI had a sensitivity of 70% and a specificity of 79% against the SCAN, with a positive predictive value of 58% and a negative predictive value of 86%.21 Its AUC was 0.77, and the number of cases it correctly identified was 12 (35%).21 A cut-point of 10 produced a higher sensitivity (80%) but a lower specificity (46%), a lower positive predictive value (38%), and a similar negative predictive value (85%).21 Although the AUC was statistically significant (0.77), correspondence between measures was low, with a kappa statistic of 0.20.21 Against the HADS-A, the BAI (cut-point of 16) had a sensitivity of 39%, a specificity of 90%, and a kappa of 0.33.22

7-Item Generalized Anxiety Disorder Scale

The version of the GAD-7 examined was a module from the full Patient Health Questionnaire,20 using the HADS-A (cut-point ≥8) as the reference standard (per the authors, although this was not a true criterion measure). It was assessed only for internal consistency (Cronbach α = 0.75) and correlation with the HADS-A (r = 0.70). Using a cut-point of 8 or more on the HADS-A, 41 participants (8%) were identified as having GAD. However, the overlap between GAD-7 and HADS-A was incomplete, with only 28 participants (25%) who were classified as having GAD by the GAD-7 (n = 41) also meeting cut-point criteria on the HADS-A (n = 112).

Risk of Bias Assessment

We rated the study conducted by Terrill et al.20 as having a low risk of bias across all the domains (Table 4). We rated the study by Honarmand and Feinstein19 as having a high risk of bias in patient selection because the sample comprised merged databases that included a nonrandom, nonconsecutive sample of patients with MS. We rated the remaining three domains as having a low risk of bias. The study by Nicholl et al.22 was rated as having a high risk of bias in the reference standard domain because they used another self-report measure to define anxiety, which could have led to misclassification. The study by Watson et al.21 was rated as having an unclear risk of bias in the patient selection domain due to lack of clarity regarding how patients were selected for participation in the database that was used for recruitment. In the flow and timing domain, we rated this study as having a high risk of bias due to the potential for participants to complete the interview and self-report anxiety instrument questions over a more extended timeframe, which may have allowed some patients' anxiety to change during the intervening period. All the studies had low applicability concerns, meaning that inclusion of participants, conduct of the index test, and application of the reference standard chosen were seen as appropriate to assessment of the criterion validity of anxiety screening tools for use in people with MS, except for the two studies that used other self-report scales as the reference standard.20,22

Table 4.

Risk of bias assessment

Discussion

In this systematic review, we identified four studies that assessed the validity of anxiety screening tools for identifying anxiety disorders in people with MS. Based on these studies, the HADS-A may be considered a potential screening tool in MS populations, although additional investigation is warranted. The BAI was found to have lower sensitivity and specificity than either of the other instruments and a lower positive predictive value (38%–58%). The poorer performance of this tool may reflect the emphasis of the BAI on physical symptoms of anxiety, such as numbness or tingling, dizziness, and hand trembling, which can overlap with the symptoms of MS. Thus, the BAI is not recommended as a suitable anxiety screening tool for people with MS based on available data to date. The GAD-7 was validated against another screening tool rather than a gold standard clinical interview (and was, therefore, an assessment of construct rather than criterion validity), and there is insufficient evidence to assess its validity for individuals with MS at this point. Note that anxiety is a prominent symptom in many mental disorders. In addition to anxiety disorders, mood disorders (eg, major depression with anxious distress), somatic symptom and related disorders, trauma- and stressor-related disorders, and psychotic disorders all may include anxiety as an important symptom.23 A high score on an anxiety screening tool indicates that further assessment is warranted.24 The diagnosis and treatment recommendations will depend on this assessment.

The four studies that we identified evaluated study populations of similar ages, with samples that were drawn from three different countries (Canada, the United States, and two from the United Kingdom). The anxiety disorders examined differed across studies. The performance of the BAI and HADS-A was considered with respect to any anxiety disorder, including GAD, specific phobia, panic disorder, obsessive-compulsive disorder, social phobia, and posttraumatic stress disorder, whereas the GAD-7, and the post hoc analysis of the BAI and the HADS-A, focused primarily on GAD. The two studies investigating the HADS-A had reasonable consistency in their results, supporting its use for anxiety screening in people with MS. The typically recommended cut-points for the HADS-A were found to be appropriate in the MS population as well. A cut-point of 8 was optimal for identifying GAD (a sensitivity of 88.5% and a specificity of 80.7%), and a cut-point of 11 optimized sensitivity (90%) and specificity (92%) for anxiety disorders overall. The HADS-A has previously been validated using similar cut-points (8–11) in a variety of medical populations, such as patients with cancer,25–29 gynecological disorders,30 and stroke.31 The sensitivity and specificity of the HADS-A in each of these populations was similar to that observed in the MS population, except in the stroke population, in which specificity was lower (56% at a cut-point of 7). However, some of those studies focused on the overall HADS score rather than on separate anxiety and depression subscales. Meanwhile, the GAD-7 has been validated in general,32 psychiatric,33 and geriatric populations.34

Valid instruments are needed to identify anxiety in clinical practice and to assess outcomes in treatment studies. In a recent review, three anxiety instruments were used in 20 trials: the HADS, GAD-7, and Hamilton Anxiety Rating Scale (HARS),35 with the latter not yet assessed for validity in an MS population. For anxiety screening instruments to be useful, they must not only have good criterion validity but also be internally reliable, have good test-retest reliability, be clinically relevant, be sensitive to change, be feasible to administer, and be acceptable to patients.

Two of the measures identified in this review (the HADS-A and the BAI) were designed as dimensional measures of anxiety to be used in various ways, including identifying people with high levels of anxiety, assessing anxiety over time, and assessing changes with intervention. Although they were not designed specifically as diagnostic screening instruments, they are sometimes used in this manner, and this practice is supported by the results of this study. It is important to recognize that individuals can meet the diagnostic criteria for an anxiety disorder but not often be experiencing intense symptoms. However, an advantage of dimensional screening tools is that they can efficiently identify individuals with higher levels of symptoms that warrant assessment and possible treatment. This is a clinically relevant consideration. On the other hand, Kessler et al.36 argued that it is best not to eliminate individuals with milder disorders who do not meet screening thresholds because long-term follow-up indicates that they are at higher risk for future more serious disorders. Long-term follow-up of individuals with MS and anxiety disorders is needed to understand how significant these risks are and what the effects of those disorders are on health outcomes. Alternatives to unidimensional symptom measures are scales designed as screening instruments for general psychological distress, such as the Kessler Distress Scales,37,38 which do not differentiate between anxiety and mood disorders but include items that may be influenced by chronic medical conditions (eg, “tired out for no good reason” and “feeling that everything was an effort”). The Overall Anxiety Severity and Impairment Scale is another measure that was developed for screening for anxiety disorders in primary-care settings.39,40 Neither has been tested in the MS population.

This review was guided by a carefully designed standardized protocol. The quality evaluation of the included studies determined that there was a high risk of bias for three19,21,22 of the four studies based on the QUADAS-2. The risk of bias was primarily due to uncertainty of patient selection and the period between administration of the screening tool being validated and the reference standard for an undisclosed number of participants. Future studies can prevent such biases by randomly selecting participants from the MS population or using other sampling strategies to ensure that the study population is representative of the general MS population and by using concurrent measurements of anxiety.

Further investigation of potential anxiety screening tools that can be used in MS is needed because anxiety is common in this population, affects outcomes such as quality of life,41 and may complicate treatment by affecting treatment decisions.42 The results of the present study indicate that the HADS-A is the best available candidate for detecting symptoms of anxiety in individuals with MS, although further validation is required, including assessing its test-retest reliability. The GAD-7 requires further validation, including examining its criterion validity against an appropriate reference standard capable of accurately determining the presence or absence of the anxiety disorder. Additional research assessing the validity of the HARS should be conducted if it continues to be used in clinical trials. Screening alone is not sufficient to diagnose a patient with anxiety, and any patient who meets the cut-points should be clinically evaluated to confirm the diagnosis and determine whether treatment is required.

PracticePoints

• Few anxiety screening tools have been validated in people with MS.

• Of those validated tools, the Hospital Anxiety and Depression Scale–Anxiety had the highest sensitivity and specificity.

• It is essential for screening tools to be validated against appropriate reference standards.