Abstract
Objective
To determine the validity of standardized screening assessments of cognitive functioning to detect neuropsychological impairment evaluated using a comprehensive battery in systemic lupus erythematosus (SLE) and rheumatoid arthritis (RA).
Methods
This is a cross-sectional study using a combined cohort of 139 persons with SLE and 82 persons with RA. Screening cut points were empirically derived using receiver operating characteristic curves and threshold selection methods. Screening measures included the Hopkins Verbal Learning Test-Revised (HVLT-R) learning and delayed recall indices and phonemic fluency, a composite measure of the 3 cognitive screening tests, and the Perceived Deficits Questionnaire-Short Form (PDQ-SF), a self-report measure of cognitive symptoms. A comprehensive neuropsychological battery was administered as the “gold standard” index of neuropsychological impairment.
Results
Rates of neuropsychological impairment were 27% and 15% for the SLE and RA cohorts, respectively. Optimal threshold estimations were derived for 5 screening techniques. The HVLT-R learning and phonemic fluency indices yielded the greatest sensitivity at 81%. The PDQ-SF yielded the lowest sensitivity at 52%. All measures were significantly associated with neuropsychological impairment after controlling for relevant sociodemographic covariates and depression.
Conclusion
These results suggest that telephone-administered screening techniques may be useful measures to identify persons with neuropsychological impairment. Specifically, measures of phonemic fluency and verbal learning appeared to be most sensitive and least likely to misclassify impaired individuals as cognitively intact. Self-reported questionnaires may have relatively decreased sensitivity compared to standardized interviewer-administered cognitive measures.
Introduction
Neuropsychological impairment is among the most common neuropsychiatric manifestations of systemic lupus erythematosus (SLE), with prevalence rates reaching as high as 81% of patients (1). Although individuals with rheumatoid arthritis (RA) are considered to be less likely to develop neuropsychological impairment, recent studies have highlighted the burden of such impairment in RA, with prevalence rates ranging from 30% to 50% (2,3). For most, cognitive dysfunction represents a decline in comparison to their previous level of functioning and may increase the risk of impairment in activities of daily living. In the clinic, patients commonly report cognitive dysfunction that is perhaps milder in severity, yet very troublesome to the patient and detrimental to their daily function. While a routine neuropsychological evaluation would be beneficial for most individuals with suspected neuropsychological impairment, there are many reasons why this option is often not readily available, including lack of access to neuropsychological services, costs of comprehensive evaluations, and time constraints. To promote rapid identification of patients who may benefit from a comprehensive neuropsychological evaluation or to identify patients in large-scale research studies, pragmatic screening approaches to identify patients with cognitive impairment are necessary.
A number of very brief bedside and telephone screening approaches have been developed and validated for use primarily with older adults, and the majority of these instruments are modifications of mental status examinations. Although these approaches are brief, they lack sensitivity in detecting the kinds of impairments characteristic of rheumatic conditions (4). Additionally, subjective symptoms of cognitive decline or measures based on perceptions of cognitive functioning are often not confirmed by neuropsychological testing and are influenced by psychiatric states, including depression (5).
The purpose of this study was to determine the utility of telephone cognitive screening approaches and self-report assessments of cognitive symptoms in detecting neuropsychological impairment for individuals with SLE and RA. While a comprehensive neuropsychological evaluation remains the “gold standard” for research and practice, brief screening approaches hold value in that they may rapidly identify those at greatest need for services or facilitate large-scale research to study cognitive compromise in rheumatic diseases and other chronic medical conditions.
Subjects and Methods
Subjects and data collection method
Two hundred twenty-one individuals (139 with SLE and 82 with RA) residing in the San Francisco Bay area were recruited for a comprehensive clinical study including in-person assessments of cognition, psychiatric status, body composition, and physical function at the University of California, San Francisco (UCSF) Clinical and Translational Science Institutes Clinical Research Center (CRC). Exclusion criteria for this study included non–English speaking, age <18 years, daily dose of 50 mg or greater of oral prednisone, pregnancy, uncorrected vision problems interfering with reading ability, and joint replacement within the past year. SLE participants were drawn from the UCSF Lupus Outcomes Study, a prospective study of 957 individuals with diagnostically confirmed SLE. Details about enrollment and data collection for this study have been reported previously (6). Briefly, subjects were recruited through academic medical centers (25%), community rheumatology offices (11%), nonclinical sources, including support groups and conferences (26%), and the media (38%). Diagnostically confirmed RA participants were drawn from the UCSF RA panel, which has also been described previously (7). Briefly, the RA panel began in 1982 with 822 patients, supplemented with 4 additional enrollment periods between 1989 and 2003. Individuals with RA were recruited from a random sample of board-certified rheumatologists practicing in northern California. The primary data collection method for both cohorts is through annual telephone interviews, including screening measures of cognitive impairment. Telephone interviews and CRC visits were separated by a mean ± SD interval of 1.7 ± 2.5 months.
Evaluators at the CRC were trained by a licensed clinical psychologist (LJJ) over the course of 6 weeks to conduct evaluations using standardized procedures, including assessments observed by the trainer. This research protocol was approved by the UCSF Committee on Human Research. All of the participants gave their informed consent to participate.
Sociodemographic factors, disease characteristics, and depression measures
Sociodemographic and disease characteristics were collected through the telephone interview to describe the sample and included age, sex, race/ethnicity, education, household income, and disease duration (years). The presence versus absence of a diagnosis of major depressive disorder was determined at the in-person assessment through the use of the Mini International Neuropsychiatric Interview (8), a structured interview to determine diagnoses for the major axis I psychiatric disorders in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition and the International Statistical Classification of Diseases and Related Health Problems, Tenth Edition that has been deemed reliable and valid (8).
Measures of neuropsychological impairment
To determine the presence of neuropsychological impairment during the in-person assessment, a battery of tests was modified from the American College of Rheumatology–recommended 1-hour battery for SLE and represented our “gold standard” approach to determine the presence of cognitive impairment (9). This battery has been previously determined to be reliable and valid in SLE (10). To accommodate possible RA-related hand motor impairment, the battery was modified to minimize hand motor demands or control for hand motor speed. The battery and indices used for analyses included: 1) verbal learning and recall: the California Verbal Learning Test-II (CVLT-II) (11) learning trials 1 through 5, short delay free recall, long delay free recall, and recognition indices; 2) nonverbal learning and recall: the Rey Complex Figure Test copy trial (12), immediate delay, and long delay; 3) fluency: the Controlled Oral Word Association Test (COWAT) (13) total correct on phonemic fluency (letters) and semantic fluency (animals) and the Delis-Kaplan Executive Function Test (DKEFS) design fluency test; 4) executive functioning: the DKEFS Color–Word Inhibition Test (inhibition condition), DKEFS Card-Sorting Test (total correct, set 1), and the DKEFS Trail-Making Test (shifting condition controlling for sequencing speed); 5) working memory and speeded processing: Symbol Digit Modalities Test oral version (total correct) (14) and Digit Span Backwards (15); and 6) visuospatial functioning: the Judgment of Line Orientation test short form (total correct) (16).
A total of 16 indices of neuropsychological functioning were generated, and impairment was assigned if performance fell below –1 SD of the population normative data. Using a conventional approach, patients were classified as having neuropsychological impairment if they were impaired on at least 5 of the 16 indices (10).
Screening measures of neuropsychological impairment
Screening measures were selected based on the presence of an analogous measure in the “gold standard” battery, feasibility of use over the telephone (e.g., no visual stimuli), sensitivity for deficits characteristic of rheumatic conditions, and brevity. The screening measures required approximately 12–15 minutes of administration time and included: 1) verbal learning and recall: the Hopkins Verbal Learning Test-Revised (HVLT-R) (17) learning condition and delayed recall condition, and 2) fluency: the COWAT phonemic fluency. Alternative letters were used for the telephone COWAT to minimize practice effects. An analogous impairment index was created in addition to the individual indices that consisted of impairment on any of the 3 cognitive measures at less than or equal to –1 SD below the population norms. Respondents were asked to participate in the telephone interview in a room free from distractions and interruptions and were instructed not to prepare or write anything down during the testing. Test examiners were survey interviewers who received training and supervision by a licensed clinical psychologist (LJJ), including practice administrations, observed administrations, and reliability checks.
Cognitive symptoms were assessed with the Perceived Deficits Questionnaire-Short Form (PDQ-SF) (18), previously used in SLE (19). The PDQ-SF consists of 5 questions covering 4 categories: attention/concentration, retrospective memory, prospective memory, and planning/organization. Respondents rate difficulties using a 4-point Likert scale (0 = never to 4 = almost always). Total scores range from 0 to 20, with higher scores reflecting increasing symptoms.
Statistical analyses
Descriptive statistics were used to characterize participant sociodemographics, disease duration, and rates of neuropsychological impairment. Zero-order correlation coefficients were calculated to evaluate associations among screening measures and the comprehensive test battery. To determine optimal screening cut points, receiver operating characteristic (ROC) curves were estimated and the Youden threshold selection method was utilized. Briefly, the Youden Index has been used as a measure of diagnostic test accuracy in clinical epidemiology and determines the maximum vertical distance from the ROC curve to the diagonal reference or “chance” line, i.e., the “optimal” cut point corresponds to the point on the ROC curve farthest from the reference line (20). Finally, multivariate linear regressions were conducted using screening measures to determine the degree to which each screening test can predict the number of neuropsychological indices impaired after controlling for relevant sociodemographics and depression.
Results
Patient and neuropsychological performance characteristics of the 139 SLE and 82 RA patients are shown in Table 1. Impairment rates on any single index ranged from 8% (digit span) to 32% (Rey Complex Figure Test copy). SLE patients demonstrated impairment on a mean ± SD of 3.5 ± 3.1 indices compared to a mean ± SD of 2.4 ± 2.2 indices for the RA cohort. Overall neuropsychological impairment was present in 25% and 10% of SLE and RA patients, respectively. Rates of impairment on the screening battery ranged from 20% and 14% (HVLT-R learning index) to 46% and 38% (impairment on at least 1 of 3 indices) for the SLE and RA cohorts, respectively. Mean ± SD scores for the PDQ-SF were 8.6 ± 4.3 and 5.6 ± 3.7 for SLE patients and RA patients, respectively. Inter-correlations among screening tests and the neuropsychological battery are shown in Table 2, with significant correlations among screening tests and their analogous tests in the comprehensive battery (i.e., HVLT-R learning and CVLT-II learning: r = 0.52, P < 0.0001; HVLT-R delay and CVLT-II long delay: r = 0.48, P < 0.0001; COWAT letter fluency: r = 0.78, P < 0.0001), as well as among screening tests and neuropsychological impairment indices (screening impairment index and neuropsychological impairment index: r = 0.29, P < 0.0001). The PDQ-SF was significantly but less robustly associated with neuropsychological impairment (r = 0.15, P < 0.05).
Table 1.
Patient and neuropsychological performance characteristics of SLE and RA patients*
| Total (n = 221) | SLE (n = 139) | RA (n = 82) | |
|---|---|---|---|
| Age, mean ± SD years | 51.8 ± 12.2 | 47.7 ± 12.4 | 57.8 ± 11.2 |
| High school education or greater | 86 (190) | 87 (121) | 83 (68) |
| Women | 91 (202) | 95 (132) | 85 (70) |
| Income | |||
| <$20,000 | 16 (35) | 22 (30) | 6 (5) |
| $20,000–39,999 | 12 (27) | 12 (16) | 13 (11) |
| $40,000–59,999 | 17 (38) | 19 (26) | 15 (12) |
| $60,000–79,999 | 13 (29) | 14 (20) | 11 (9) |
| $80,000–99,999 | 13 (29) | 9 (13) | 18 (15) |
| ≥$100,000 | 29 (63) | 25 (34) | 37 (30) |
| Race/ethnicity | |||
| Hispanic/Latino | 10 (21) | 11 (15) | 7 (6) |
| White | 66 (146) | 56 (78) | 83 (68) |
| African American | 8 (17) | 11 (15) | 2 (2) |
| Asian | 12 (27) | 15 (21) | 7 (6) |
| Other | 5 (11) | 7 (10) | 9 (7) |
| Disease duration, mean ± SD years | 18.6 ± 10.9 | 15.6 ± 9.4 | 21.1 ± 11.6 |
| Major depressive disorder | 15 (32) | 18 (25) | 9 (7) |
| Neuropsychological functioning (CRC visit) | |||
| Verbal learning and memory impairment | |||
| CVLT-II learning | 12 (27) | 15 (21) | 9 (7) |
| CVLT-II short delay free recall | 19 (42) | 20 (28) | 18 (15) |
| CVLT-II long delay free recall | 24 (53) | 28 (39) | 17 (14) |
| CVLT-II recognition task | 23 (51) | 25 (35) | 19 (16) |
| Visuospatial learning and memory impairment | |||
| Rey Complex Figure Test copy | 32 (71) | 33 (46) | 31 (25) |
| Rey immediate delay | 24 (53) | 30 (42) | 14 (12) |
| Rey long delay | 24 (53) | 31 (43) | 14 (11) |
| Executive functioning impairment | |||
| Color–Word Inhibition | 20 (44) | 23 (32) | 14 (12) |
| Card Sorting | 17 (38) | 19 (26) | 14 (12) |
| Trail Making (shifting) | 18 (40) | 19 (26) | 17 (14) |
| Fluency impairment | |||
| Phonemic fluency | 28 (62) | 31 (43) | 24 (20) |
| Semantic fluency | 9 (20) | 8 (11) | 10 (8) |
| Design fluency | 25 (55) | 22 (31) | 30 (25) |
| Working memory/speeded processing impairment | |||
| Symbol Digit Modalities Test | 23 (51) | 29 (40) | 13 (11) |
| Digit span (backward) | 8 (18) | 7 (10) | 9 (7) |
| Visuospatial impairment | |||
| Judgment of Line Orientation | 15 (33) | 18 (25) | 8 (7) |
| Total number of cognitive tasks impaired, mean ± SD | 3.0 ± 2.7 | 3.45 ± 3.10 | 2.4 ± 2.2 |
| At least one-third of cognitive tasks impaired | 19 (42) | 25 (34) | 10 (8) |
| Neuropsychological function (screening) | |||
| HVLT-R learning | 18 (40) | 20 (28) | 14 (12) |
| HVLT-R delayed recall | 20 (44) | 21 (29) | 20 (16) |
| Phonemic fluency | 28 (62) | 31 (43) | 25 (21) |
| At least one-third of screening tasks impaired | 43 (95) | 46 (64) | 38 (31) |
| Self-reported cognitive functioning | |||
| Perceived Deficits Questionnaire-Short Form | 7.5 ± 4.3 | 8.6 ± 4.3 | 5.6 ± 3.7 |
Values are the percentage (number) unless otherwise indicated. SLE = systemic lupus erythematosus; RA = rheumatoid arthritis; CRC = Clinical Research Center; CVLT-II = California Verbal Learning Test-II; HVLT-R = Hopkins Verbal Learning Test-Revised.
Table 2.
Correlations among cognitive screening measures and the neuropsychological battery in SLE and RA*
| Total group (n = 222) |
SLE (n = 138) |
RA (n = 84) |
|||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Impairment† | HVLT-R learning | HVLT-R delay | Letter fluency | PDQ-SF | Impairment† | HVLT-R learning | HVLT-R delay | Letter fluency | PDQ-SF | Impairment† | HVLT-R learning | HVLT-R delay | Letter fluency | PDQ-SF | |
| No. of neuropsychological indices impaired | 0.37‡ | –0.49‡ | –0.47‡ | –0.39‡ | 0.17§ | 0.44‡ | –0.50‡ | –0.48‡ | –0.44‡ | 0.10 | 0.37‡ | –0.51‡ | –0.49‡ | –0.33‡ | 0.20 |
| Neuropsychological impairment | 0.29‡ | –0.34‡ | –0.47‡ | –0.34‡ | 0.15§ | 0.34‡ | –0.36‡ | –0.37‡ | –0.37‡ | 0.12 | 0.16 | –0.34‡ | –0.26§ | –0.21 | 0.04 |
| CVLT-II learning index | –0.26 | 0.52‡ | 0.46‡ | 0.29‡ | –0.19 | –0.30‡ | 0.51‡ | 0.41‡ | 0.34‡ | –0.18§ | –0.27§ | 0.54‡ | 0.56‡ | 0.23§ | –0.19 |
| CVLT-II short delay free recall | –0.26‡ | 0.52‡ | 0.49‡ | 0.46‡ | –0.16§ | –0.29‡ | 0.50‡ | 0.46‡ | 0.34‡ | –0.19§ | –0.26§ | 0.55‡ | 0.52*‡ | 0.15 | –0.14 |
| CVLT-II long delay free recall | –0.29‡ | 0.50‡ | 0.48‡ | 0.30‡ | –0.15§ | –0.29‡ | 0.49‡ | 0.47‡ | 0.33‡ | –0.17§ | –0.30‡ | 0.53‡ | 0.50‡ | 0.27§ | –0.11 |
| CVLT-II recognition | –0.26‡ | 0.46‡ | 0.44‡ | 0.24‡ | –0.15§ | –0.28‡ | 0.41‡ | 0.40‡ | 0.25‡ | –0.16 | –0.37‡ | 0.52‡ | 0.52‡ | 0.23§ | –0.15 |
| Rey Complex Figure Test copy | –0.14§ | 0.30‡ | 0.34‡ | 0.18‡ | –0.06 | –0.21§ | 0.24‡ | 0.35‡ | 0.22‡ | –0.02 | –0.21 | 0.44‡ | 0.33‡ | 0.11 | –0.15 |
| Rey Complex Figure Test immediate delay | 0.16§ | 0.27‡ | 0.30‡ | 0.15§ | –0.13 | –0.22‡ | 0.24‡ | 0.30‡ | 0.16 | –0.11 | –0.24§ | 0.35‡ | 0.29‡ | 0.14 | –0.15 |
| Rey Complex Figure Test long delay | –0.19 | 0.31‡ | 0.30‡ | 0.17‡ | –0.07 | –0.26‡ | 0.28‡ | 0.31‡ | 0.21§ | –0.07 | –0.35‡ | 0.36‡ | 0.29‡ | 0.11 | –0.13 |
| Symbol Digit Modalities Test | –0.22 | 0.29‡ | 0.26‡ | 0.31‡ | –0.11 | –0.31‡ | 0.25‡ | 0.25‡ | 0.33‡ | –0.22‡ | –0.26§ | 0.53‡ | 0.50‡ | 0.27§ | 0.02 |
| Digit span (backward) | –0.27‡ | 0.26‡ | 0.21 | 0.28‡ | –0.04 | –0.28‡ | 0.26‡ | 0.24‡ | 0.32‡ | –0.01 | –0.16 | 0.27§ | 0.17 | 0.23§ | 0.10 |
| Color–Word Interference test | –0.19 | 0.26‡ | 0.27‡ | 0.29‡ | 0.03 | –0.23‡ | 0.20§ | 0.18§ | 0.24‡ | 0.19§ | 0.20 | 0.39* | –0.18 | 0.32‡ | 0.05 |
| Card-Sorting test | –0.27‡ | 0.26‡ | 0.27‡ | 0.29‡ | 0.03 | –0.26‡ | 0.39‡ | 0.37‡ | 0.32‡ | –0.06 | –0.11 | 0.06 | 0.09 | 0.27§ | 0.09 |
| Trail-Making test | 0.24‡ | –0.29‡ | –0.32‡ | –0.29‡ | 0.00 | –0.24‡ | 0.16 | 0.15 | 0.18§ | 0.00 | 0.02 | 0.03 | 0.00 | 0.09 | –0.03 |
| Letter fluency | –0.35‡ | 0.35‡ | 0.32‡ | 0.78‡ | –0.11 | –0.51‡ | 0.24‡ | 0.27‡ | 0.77‡ | –0.07 | –0.44‡ | 0.29‡ | 0.28‡ | 0.83‡ | –0.14 |
| Category fluency | –0.14‡ | 0.26‡ | 0.21‡ | 0.40‡ | –0.20 | –0.28‡ | 0.20§ | 0.16 | 0.45‡ | –0.25‡ | –0.31‡ | 0.39‡ | 0.28‡ | 0.34‡ | –0.23§ |
| Design fluency | –0.09 | 0.24‡ | 0.25‡ | 0.09 | 0.00 | –13 | 0.24‡ | 0.27‡ | 0.16 | 0.08 | –0.05 | 0.24§ | 0.23§ | –0.05 | –0.15 |
| Judgment of Line Orientation | –0.15§ | 0.31 | 0.30‡ | 0.14§ | –0.17§ | –0.18§ | 0.38‡ | 0.31‡ | 0.17 | –0.16 | –0.15 | 0.12 | 0.14 | 0.07 | –0.07 |
Nonparametric (Spearman's rho) used as necessary. SLE = systemic lupus erythematosus; RA = rheumatoid arthritis; HVLT-R = Hopkins Verbal Learning Test-Revised; PDQ-SF = Perceived Deficits Questionnaire-Short Form; CVLT-II = California Verbal Learning Test-II.
Screening neuropsychological test impairment on ≥1 of 3 tests administered.
P < 0.01.
P < 0.05.
ROC curves were generated using the entire group (not shown) and empirically derived thresholds were estimated along with the 95% confidence interval estimates of precision (Table 3). The COWAT thresholds yielded the highest sensitivity (79%) and the highest negative predictive value (0.93). The PDQ-SF cut point yielded the lowest sensitivity and negative predictive values of 0.55 and 0.87, respectively. Classification for SLE and RA cohorts are also shown in Table 3.
Table 3.
Empirically derived cut points for cognitive screening measures as compared to impairment on the full neuropsychological battery*
| PDQ-SF, ≥10 | 3 screening cognitive tests, ≥1 impaired at –1 SD | Phonemic fluency, ≤34 | HVLT-R learning, ≤26 | HVLT-R delay, ≤9 | |
|---|---|---|---|---|---|
| Total group | |||||
| Sensitivity (95% CI) | 0.55 (0.38–0.70) | 0.74 (0.58–0.86) | 0.79 (0.63–0.89) | 0.74 (0.58–0.86) | 0.67 (0.50–0.80) |
| Specificity (95% CI) | 0.73 (0.66–0.79) | 0.65 (0.57–0.72) | 0.66 (0.58–0.72) | 0.65 (0.57–0.72) | 0.70 (0.62–0.76) |
| PPV (95% CI) | 0.32 (0.26–0.39) | 0.34 (0.24–0.44) | 0.35 (0.26–0.46) | 0.35 (0.24–0.44) | 0.35 (0.25–0.46) |
| NPV (95% CI) | 0.87 (0.80–0.92) | 0.91 (0.84–0.95) | 0.93 (0.86–0.96) | 0.92 (0.84–0.95) | 0.90 (0.83–0.94) |
| % correctly classified | 69 | 67 | 68 | 67 | 70 |
| SLE | |||||
| Sensitivity (95% CI) | 0.62 (0.44–0.77) | 0.77 (0.58–0.89) | 0.79 (0.62–0.91) | 0.74 (0.55–0.86) | 0.47 (0.30–0.65) |
| Specificity (95% CI) | 0.65 (0.54–0.74) | 0.65 (0.54–0.74) | 0.67 (0.56–0.76) | 0.68 (0.57–0.77) | 0.85 (0.76–0.91) |
| PPV (95% CI) | 0.38 (0.25–0.51) | 0.43 (0.30–0.56) | 0.45 (0.32–0.58) | 0.43 (0.31–0.58) | 0.52 (0.34–0.69) |
| NPV (95% CI) | 0.83 (0.73–0.91) | 0.89 (0.79–0.95) | 0.92 (0.81–0.96) | 0.89 (0.78–0.94) | 0.82 (0.73–0.89) |
| % correctly classified | 64 | 67 | 70 | 69 | 75 |
| RA | |||||
| Sensitivity (95% CI) | 0.25 (0.04–0.64) | 0.63 (0.26–0.90) | 0.86 (0.43–0.99) | 0.75 (0.36–0.96) | 0.75 (0.36–0.96) |
| Specificity (95% CI) | 0.84 (0.73–0.91) | 0.65 (0.53–0.76) | 0.64 (0.52–0.75) | 0.61 (0.49–0.72) | 0.77 (0.66–0.86) |
| PPV (95% CI) | 0.14 (0.03–0.44) | 0.16 (0.06–0.34) | 0.18 (0.07–0.36) | 0.17 (0.07–0.34) | 0.26 (0.11–0.48) |
| NPV (95% CI) | 0.91 (0.81–0.96) | 0.94 (0.83–0.99) | 0.98 (0.88–0.99) | 0.96 (0.85–0.95) | 0.97 (0.87–0.99) |
| % correctly classified | 78 | 65 | 66 | 63 | 77 |
PDQ-SF = Perceived Deficits Questionnaire-Short Form; HVLT-R = Hopkins Verbal Learning Test-Revised; 95% CI = 95% confidence interval; PPV = positive predictive value; NPV = negative predictive value; SLE = systemic lupus erythematosus; RA = rheumatoid arthritis.
Separate multivariate linear regressions were conducted using each screening measure as a predictor of the number of neuropsychological indices impaired controlling for education, ethnicity, sex, income, and the presence of major depressive disorder, followed by a final regression including the entire screening battery (Table 4). The PDQ-SF was the only screening measure that did not significantly predict neuropsychological impairment after controlling for demographics and depression. Phonemic fluency ≤34, HVLT-R learning ≤26, and HVLT-R delay ≤9 were all significant predictors of neuropsychological impairment, accounting for 11%, 13%, and 14% of the variance after accounting for demographics and depression, respectively. The 4 measures as a cognitive battery collectively predicted 26% of the variance independent of depression and demographics.
Table 4.
Separate multivariate linear regression analyses predicting the number of neuropsychological indices impaired*
| Total group | Standardized β | R2 | ΔR2 | Significance |
|---|---|---|---|---|
| Regression 1: PDQ-SF ≥10 | 0.13 | 0.20 | 0.01 | 0.14 |
| Regression 2: phonemic fluency ≤34 | 0.33 | 0.29 | 0.11 | < 0.0001 |
| Regression 3: HVLT-R learning ≤26 | 0.08 | 0.32 | 0.13 | < 0.0001 |
| Regression 4: HVLT-R delay ≤9 | 0.35 | 0.34 | 0.14 | < 0.0001 |
| Regression 5: all cognitive screening measures | 0.46 | 0.26 | < 0.0001 |
Controlling for race/ethnicity, educational level, sex, income, and major depressive disorder. PDQ-SF = Perceived Deficits Questionnaire-Short Form; HVLT-R = Hopkins Verbal Learning Test-Revised.
Discussion
The purpose of this study was to determine the criterion validity of telephone-administered cognitive screening tools in identifying individuals with neuropsychological impairment. Results suggest that telephone screening measures of learning, recall, and fluency were relatively sensitive (sensitivity rates reaching 79%), modestly specific (specificity rates ranged from 60% to 70%), and unlikely to classify an impaired individual as being intact (negative predictive value ranged from 90% to 93%). After controlling for sociodemographics and depression, the neuropsychological screening measures were significant predictors of neuropsychological impairment, but the self-report measure was not an independent predictor of impairment.
The self-reported measure of cognitive symptoms was a less sensitive screening measure for neuropsychological impairment, suggesting slightly decreased validity in detecting neuropsychological impairment. Despite these limitations, self-report measures may have a role in clinical practice and research. These measures provide an evaluation of an individual's perception of their cognitive abilities, and may indicate very mild cognitive changes noticeable to the patient but not yet detectable on the examination. Additionally, comparisons of these measures to informant-based measures and objective neuropsychological testing can provide important information about a patient's insight into their own functioning (21). An additional limitation is related to the precision of estimates based on a smaller sample size of cognitively impaired participants in the disease-specific analyses. In particular, sensitivity point estimates were susceptible to decreased precision as evidenced by the wider confidence intervals, most notably in the RA sample. Further, the relatively low positive predictive values in comparison to the high negative predictive values suggest that these measures may have some advantage in identifying with greater confidence patients who do not have neuropsychological impairment as compared to the relatively reduced confidence in identifying patients who would have neuropsychological impairment. Further, the use of published normative data to classify impairment may be somewhat disadvantageous, and a local matched control group may have alleviated some problems with national normative data. Our cohort may have differed somewhat from national data, particularly with respect to higher education levels, which could have biased our results primarily by underestimating impairment in this study.
The burden of neuropsychological impairment in rheumatic disease is increasingly recognized, and given limited resources, we continue to seek brief but sensitive approaches to detect cognitive impairment for both research endeavors and clinical practice. We present data suggesting that existing standardized screening measures can be feasibly administered by telephone and have adequate utility in identifying patients with neuropsychological impairment. For both clinical and research purposes, these measures hold promise as a means to identify patients who would meet criteria for cognitive impairment, or clinically would be candidates for more comprehensive neuropsychological testing. Estimates of specificity and sensitivity for the single measures were very comparable to existing single-measure screening approaches available in other conditions (22). Screening measures are best interpreted in collaboration with a provider that is competent in neuropsychological assessment to ensure appropriate clinical interpretation. Through the use of such screening techniques, we may facilitate the identification of cognitive impairment in patients and potentially reduce the risk of a range of poor health and functional outcomes.
Significance & Innovations.
● The burden of cognitive impairment is high in systemic lupus erythematosus and increasingly recognized in rheumatoid arthritis.
● Screening measures to rapidly identify individuals with suspected cognitive impairment for large-scale research and clinical purposes are necessary.
● Three standardized screening measures of learning, recall, and fluency demonstrated adequate sensitivity in detecting cognitive impairment.
● Self-reported measures of cognitive symptoms were less sensitive measures to identify individuals with cognitive impairment.
Acknowledgments
The contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.
Supported by the NIH/National Center for Research Resources, University of California, San Francisco Clinical and Translational Science Institute (grant UL1-RR024131), the NIH/National Institute of Arthritis and Musculoskeletal and Skin Diseases (grant P60AR053308), the NIH/National Institute of Mental Health (grant K08MH072724), and the Rosalind Russell Medical Research Center for Arthritis.
Footnotes
AUTHOR CONTRIBUTIONS
All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. Julian had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study conception and design. Julian, Yelin, Katz.
Acquisition of data. Julian, Trupin, Criswell, Yelin, Katz.
Analysis and interpretation of data. Julian, Yazdany, Trupin, Yelin.
REFERENCES
- 1.Ainiala H, Loukkola J, Peltola J, Korpela M, Hietaharju A. The prevalence of neuropsychiatric syndromes in systemic lupus erythematosus. Neurology. 2001;57:496–500. doi: 10.1212/wnl.57.3.496. [DOI] [PubMed] [Google Scholar]
- 2.Appenzeller S, Bertolo MB, Costallat LT. Cognitive impairment in rheumatoid arthritis. Methods Find Exp Clin Pharmacol. 2004;26:339–43. doi: 10.1358/mf.2004.26.5.831324. [DOI] [PubMed] [Google Scholar]
- 3.Bartolini M, Candela M, Brugni M, Catena L, Mari F, Pomponio G, et al. Are behaviour and motor performances of rheumatoid arthritis patients influenced by subclinical cognitive impairments? A clinical and neuroimaging study. Clin Exp Rheumatol. 2002;20:491–7. [PubMed] [Google Scholar]
- 4.Leritz E, Brandt J, Minor M, Reis-Jensen F, Petri M. “Subcortical” cognitive impairment in patients with systemic lupus erythematosus. J Int Neuropsychol Soc. 2000;6:821–5. doi: 10.1017/s1355617700677093. [DOI] [PubMed] [Google Scholar]
- 5.Julian L, Merluzzi NM, Mohr DC. The relationship among depression, subjective cognitive impairment, and neuropsychological performance in multiple sclerosis. Mult Scler. 2007;13:81–6. doi: 10.1177/1352458506070255. [DOI] [PubMed] [Google Scholar]
- 6.Yelin E, Trupin L, Katz P, Criswell L, Yazdany J, Gillis J, et al. Work dynamics among persons with systemic lupus erythematosus. Arthritis Rheum. 2007;57:56–63. doi: 10.1002/art.22481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Katz PP, Morris A, Yelin EH. Prevalence and predictors of disability in valued life activities among individuals with rheumatoid arthritis. Ann Rheum Dis. 2006;65:763–9. doi: 10.1136/ard.2005.044677. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(Suppl):22–57. [PubMed] [Google Scholar]
- 9.ACR Ad Hoc Committee on Neuropsychiatric Lupus Nomenclature The American College of Rheumatology nomenclature and case definitions for neuropsychiatric lupus syndromes. Arthritis Rheum. 1999;42:599–608. doi: 10.1002/1529-0131(199904)42:4<599::AID-ANR2>3.0.CO;2-F. [DOI] [PubMed] [Google Scholar]
- 10.Kozora E, Ellison MC, West S. Reliability and validity of the proposed American College of Rheumatology neuropsychological battery for systemic lupus erythematosus. Arthritis Rheum. 2004;51:810–8. doi: 10.1002/art.20692. [DOI] [PubMed] [Google Scholar]
- 11.Delis D, Kramer J, Kaplan E, Ober B. California Verbal Learning Test manual: second edition, adult version. Psychological Corporation; San Antonio (TX): 2000. [Google Scholar]
- 12.Meyers JE, Meyers KR. Rey Complex Figure Test. Psychological Assessment Resources; Lutz (FL): 1995. [Google Scholar]
- 13.Spreen O, Strauss E. A compendium of neuropsychological tests: administration, norms, and commentary. 2nd ed. Oxford University Press; New York: 1998. [Google Scholar]
- 14.Smith A. Symbol Digit Modalities Test (SDMT): manual (revised) Western Psychological Services; Los Angeles: 1982. [Google Scholar]
- 15.Wechsler D. WISC-IV administrative and scoring manual. Psychological Corporation; San Antonio (TX): 2003. [Google Scholar]
- 16.Vanderploeg RD, LaLone LV, Greblo LV, Schinka JA. Odd-even short forms of the judgment of line orientation test. Appl Neuropsychol. 1997;4:244–6. doi: 10.1207/s15324826an0404_6. [DOI] [PubMed] [Google Scholar]
- 17.Benedict RH, Brandt J. Hopkins Verbal Learning Test-Revised/Brief Visuospatial Memory Test-Revised: professional manual supplement. Psychological Assessment Resources; Lutz (FL): 2007. [Google Scholar]
- 18.Sullivan JJ, Edgley K, Dehoux E. A survey of multiple sclerosis. Part 1. Perceived cognitive problems and compensatory strategy use. Can J Rehabil. 1990;4:99–105. [Google Scholar]
- 19.Vogel A, Bhattacharya S, Larsen J, Jacobsen S. Do subjective cognitive complaints correlate with cognitive impairment in systemic lupus erythematosus? A Danish outpatient study. Lupus. 2011;20:35–43. doi: 10.1177/0961203310382430. [DOI] [PubMed] [Google Scholar]
- 20.Kraemer HC. Evaluating medical tests: objective and quantitative guidelines. Sage Publications; Newbury Park (CA): 1992. [Google Scholar]
- 21.Marrie RA, Chelune GJ, Miller DM, Cohen JA. Subjective cognitive complaints relate to mild impairment of cognition in multiple sclerosis. Mult Scler. 2005;11:69–75. doi: 10.1191/1352458505ms1110oa. [DOI] [PubMed] [Google Scholar]
- 22.Parmenter BA, Weinstock-Guttman B, Garg N, Munschauer F, Benedict RH. Screening for cognitive impairment in multiple sclerosis using the Symbol Digit Modalities Test. Mult Scler. 2007;13:52–7. doi: 10.1177/1352458506070750. [DOI] [PubMed] [Google Scholar]