Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Nov 1.
Published in final edited form as: Rehabil Psychol. 2017 Nov;62(4):425–434. doi: 10.1037/rep0000140

Cognition among Community-Dwelling Individuals with Spinal Cord Injury

Matthew L Cohen 1, David S Tulsky 2, James A Holdnack 3, Noelle E Carlozzi 4, Alex Wong 5, Susan Magasi 6, Robert K Heaton 7, Allen W Heinemann 8
PMCID: PMC5745049  NIHMSID: NIHMS905783  PMID: 29265863

Abstract

Purpose/Objective

To compare the cognitive profiles of a well-characterized sample of adults with and without spinal cord injury (SCI) using the NIH Toolbox – Cognition Battery (NIHTB-CB).

Research Method/Design

156 community-dwelling individuals with SCI were recruited from three academic medical centers. 156 individuals without SCI were selected from the NIHTB-CB normative database. The main outcome measures were the demographically-adjusted NIHTB-CB subtest and composite scores.

Results

Individuals with and without SCI performed equivalently on the NIHTB-CB crystallized composite score, suggesting comparable premorbid functioning. Individuals with SCI produced lower scores on the NIHTB-CB fluid composite score by an average of 4.5 T-score points (Cohen’s d = 0.50; a medium effect size). As a group, individuals with SCI had the most difficulty on tests of processing speed and executive functions, and some difficulty on a test of episodic memory, although effect sizes were small. These differences remained even after accounting for fine motor speed and dexterity. Individuals with tetraplegia produced lower scores than individuals with paraplegia on tests of processing speed and executive functioning.

Conclusions/Implications

Community-dwelling individuals with SCI are at elevated risk of mild cognitive difficulties, particularly on tasks that rely on processing speed and executive functions. The NIHTB-CB is relatively brief, samples important cognitive domains, has good normative data, and is appropriate for some individuals with SCI (those who have functional use of one hand). The battery has standardized accommodations for individuals with minor motor limitations, but timed tests are inaccessible for individuals who are unable to perform a rapid button press.

Keywords: Spinal cord injuries, Cognition, Neuropsychological Tests

Over the last 50 years, spinal cord injury (SCI) medicine has advanced greatly, including the assessment and treatment of secondary conditions (Tulsky & Kisala, 2015). However, despite the fact that cognition affects self-care and community reintegration (Bradbury et al., 2008; Davidoff, Roth, & Richards, 1992), relatively little has been published on the cognitive correlates of SCI. Clinically, cognitive limitations may be misinterpreted as nonadherence to treatment, inability to learn, poor coping, and/or poor motivation (Bradbury et al., 2008; Inoue et al., 2013; Kushner & Alvarez, 2014).

There are several reasons why an individual with SCI may have cognitive deficits. First, individuals may have had premorbid challenges that affect cognition (and that increased risk of SCI), including attention deficit hyperactivity disorder (ADHD) (Lyon, Baker, & Gren, 2009), learning problems (Macciocchi, Seel, & Thompson, 2013) and/or substance misuse (Heinemann, Keen, Donohue, & Schnoll, 1988). Second, individuals with SCI may have sustained concurrent traumatic brain injury (TBI; which may be undiagnosed)(Hagen, Eide, Rekand, Gilhus, & Gronning, 2010; Macciocchi, Seel, Thompson, Byams, & Bowman, 2008; Roth et al., 1989; Sharma, Bradbury, Mikulis, & Green, 2014). Finally, some of the consequences and secondary complications of SCI may affect cognition, such as mood disorder(s)(Bonanno, Kennedy, Galatzer-Levy, Lude, & Elfstrom, 2012; Boyer, Knolls, Kafkalas, & Tollen, 2000; Dryden et al., 2005; Krause, Saunders, & Newman, 2010; Migliorini, Sinclair, Brown, Tonge, & New, 2015), chronic pain (Dijkers, Bryce, & Zanca, 2009; Hassanijirdehi et al., 2015; Masri & Keller, 2012), poor sleep and fatigue (Avluk et al., 2014), and medication side effects (Macciocchi et al., 2013). While there are few published reports that link each of these consequences/complications to cognition in SCI specifically, these conditions have been shown to affect cognition in other medical populations.

The influence of poor cognition, regardless of the cause, on quality of life and rehabilitation after SCI has not been well studied. Concurrent severe TBI (a strong risk factor for cognitive impairment) delays motor recovery after SCI (Inoue et al., 2013; Macciocchi, Seel, Warshowsky, Thompson, & Barlow, 2012). Macciocchi and colleagues reported that patients with concurrent severe TBI have longer acute lengths of stay and worse motor outcomes at discharge, and that delayed motor skill acquisition may be partially explained by impairments in processing speed, language comprehension, memory, and problem solving (Macciocchi, Bowman, Coker, Apple, & Leslie, 2004; Macciocchi et al., 2012). Difficulty performing self-care practices (e.g., medication management), for example, because of poor memory or problem solving (Herrick, Elliott, & Crow, 1994), may place people with SCI at risk for re-hospitalization. Indeed, over 36% of people with SCI are re-hospitalized once in the first year after initial discharge, and over 12% are re-hospitalized twice, frequently because of complications like urinary tract infections and pressure ulcers (DeJong et al., 2013).

Studies of specific cognitive problems after SCI have reported low scores across most major domains of cognition, including attention and concentration, processing speed, episodic memory, and executive functioning (Bradbury et al., 2008; Davidoff, Morris, Roth, & Bleiberg, 1985; Davidoff et al., 1992; Dowler et al., 1997; Dowler et al., 1995; Hess, Marwitz, & Kruetzer, 2003; Lazzaro, Tran, Wijesuriya, & Craig, 2013; Macciocchi et al., 2013; Roth et al., 1989; Wilmot, Cope, Hall, & Acker, 1985). It is not well understood, however, which clinical and injury-related variables predict specific cognitive weaknesses. Individuals with recent injuries have been reported to show problems across most cognitive domains, whereas community dwelling individuals appear most likely to show deficits of processing speed (Dowler et al., 1997; Dowler et al., 1995). Although previous studies have underscored the fact that cognition is important and may be abnormal after SCI, further conclusions are limited by small sample sizes and other methodological characteristics. Most studies have used ad hoc test batteries that may not sample many cognitive domains. Within and across studies, chosen tests reference different normative data that are adjusted for inconsistent factors, or use control groups matched for different factors. Thus, the variable cognitive findings across studies may reflect the fact that normative data or control groups were adjusted or matched for different factors. Finally, the use and interpretation of tests with a motor component has been inconsistent. Indeed, interpretation of cognitive tests with a motor component (which most do)(Cicerone et al., 2011) requires extra attention because these may selectively disadvantage people with decreased upper limb function secondary to SCI (APA, 2014).

The goal of this report is to describe the cognitive profiles of a large, well-characterized sample of community-dwelling adults with SCI using the National Institutes of Health (NIH) Toolbox for the Assessment of Neurological and Behavioral Function – Cognition Battery (NIHTB-CB). The NIHTB-CB is a battery of neuropsychological tests (Gershon et al., 2014) with a list of recommended test administration accommodations to use with individuals who have limited use of their hands (NIH, 2012), and with all tests referencing the same normative data that are calibrated for age, sex, race/ethnicity, and education (Weintraub et al., 2013; Weintraub et al., 2014). Interpreting demographically-adjusted normative data permits greater confidence that group differences are related to SCI rather than socio-demographic factors. We hypothesize that participants with SCI will produce lower scores than control participants on tests of processing speed, working memory, episodic memory, and executive functioning, but with a more attenuated effect size (<1 standard deviation) than previous investigations (Macciocchi et al., 2013; Roth et al., 1989) because the adjusted NIHTB-CB scores will control better for socio-demographic influences on cognitive scores.

Method

Participants

This study is a secondary analysis of data that were collected from a large, multi-site study that aimed to evaluate the validity of new outcome measures, including the NIHTB, in disability populations. Participants were recruited through the institutions’ patient registries and outpatient clinics, as well as from the community through newsletters, flyers, and affiliated hospital referrals. Participants were 18–85 years old, and able to speak and read English at a 5th-grade level, as determined by the Reading subtest of the Wide Range Achievement Test, 4th Ed. (WRAT-IV)(Wilkinson & Robertson, 2006). Participants with SCI were at least one-year since their most recent injury. We characterized SCI either as paraplegia or tetraplegia, and either complete or incomplete according to the International Standards for Neurological Classification of SCI (Kirshblum et al., 2011). Study personnel reviewed medical records to confirm injury-related information. Concomitant TBI was not a focus of the parent study, and was not consistently assessed at each study site.

Study participation required 2 days for most participants, and participants received a stipend of US $90 per day. Some participants required additional days of testing, and received a stipend of US $20 per day for each additional day. Participants were offered rest breaks often. On the first day of the study, participants provided written informed consent in accordance with the local Institutional Review Board. Participants completed the WRAT-IV Reading subtest (as described above) to ensure an adequate reading level for the study tasks, and the Lighthouse Near Visual Acuity Test, 2nd Ed. (Ferris, Kassoff, Bresnick, & Bailey, 1982)(similar to a Snellen chart) to ensure that their vision was at least 20/100.

Participants completed the NIHTB (all batteries) and several self-report measures as part of a comprehensive assessment protocol. The administration the NIHTB-Cognitive Battery took approximately 30 minutes, and administration of all instruments took approximately 10 hours, with breaks scheduled and offered as needed. This report focuses on the results of the NIHTB-CB, but with some analyses including scores from the NIH Toolbox 9-Hole Pegboard Dexterity Test of upper limb function (Reuben et al., 2013).

Measures and Administration

The NIH-Toolbox Cognition Battery (NIHTB-CB) was administered by computer with the assistance of a test administrator. For individuals with limited upper extremity function, test administrators followed the NIH Toolbox Reasonable Accommodations Guide (NIH, 2012). If an individual was not able to complete a test with reasonable accommodations, the test was not administered. A post-hoc audit of accommodations resulted in some scores being discarded because of questionable fidelity to the NIHTB guidelines (Magasi et al., Submitted). The dependent variable for all tests was a T score (mean = 50, SD = 10 in the normative sample) that was adjusted for age, sex, education, and race/ethnicity (Casaletto et al., 2015). Specific measures are described below. The NIHTB-CB composite scores were also demographically adjusted (Casaletto et al., 2015).

The NIHTB Picture Vocabulary Test (Gershon et al., 2013) measures receptive vocabulary. The participant was shown four photographic images on a computer screen, and was asked to click one image that most closely matched the meaning of an aurally-presented word. The test included two practice items with feedback and 25 test items, administered by computer adaptive test (CAT) based on item-response theory (IRT), with initial level of difficulty depending on the participant’s level of education.

The NIHTB Oral Reading Recognition Test (Gershon et al., 2013) measures written word familiarity. The participant was asked to read aloud a word that was shown on the computer screen while the examiner recorded whether or not the word was pronounced correctly. The difficulty of the initial item was determined by the participant’s education, with subsequent items being presented by CAT.

The NIHTB Pattern Comparison (Carlozzi et al., 2014) assesses psychomotor speed. The participant was asked to discern whether two side-by-side figures were the same or not, and to respond as quickly as possible with a key press. Participants were asked to use only their dominant index finger to press the right or left keyboard buttons (indicating ‘same’ or ‘different’). The total score reflected the number of correct items in 90 seconds.

The NIHTB Picture Sequence Memory Test (Dikmen et al., 2014) assesses visuospatial episodic memory. The participant was shown a sequence of pictures, and then asked to reproduce the sequence from memory. The sequence length varied from 6–18 pictures depending on the participant’s age. A point was awarded for each correctly matched adjacent pairs (e.g., the maximum raw score for an 18-picture sequence is 17).

The NIHTB Dimensional Change Card Sorting Test (Zelazo et al., 2014) measures concept formation and cognitive flexibility (executive functions). The participant was shown two target pictures and a series of test pictures. Each test picture matched the target picture on one of two dimensions, color or shape. The words “color” or “shape” (i.e., the sorting dimension) was displayed on the screen before each trial to instruct the participant how to sort the target shape on that trial. The participant was first shown a block of color-sort trials, followed by a block of shape-sort trials. Then, the participant was shown a block of trials that have shape and color sorting trials randomly intermixed. The participant indicated his or her response by using his or her dominant index finger to press the left or right keyboard arrow buttons. The total score, the one that is demographically adjusted, was produced by a formula that accounts for both speed and accuracy.

The NIHTB Flanker Test (Zelazo et al., 2013) measures sustained visual attention and cognitive inhibition. Participants were shown a string of arrows and asked to indicate which direction the central arrow, the target arrow, is facing (left or right), and press the corresponding keyboard arrow key with their dominant index finger. When the target arrow pointed in the same direction and as the flanking arrows, the trial was said to be “congruent.” When the target arrow pointed in the opposite direction as the flanking arrows, the trial was said to be “incongruent.” Incongruent trials required inhibition processes because the participant’s automatic response is in the direction of the flanking arrows. The scoring was based on a combination of accuracy and reaction time. For both the Flanker Test and the Dimensional Change Card Sorting Task, the only score that is demographically adjusted is the one that combines speed and accuracy.

The NIHTB crystallized cognition composite score (Slotkin et al., 2012) is comprised of the picture vocabulary test and the oral reading recognition test and reflects past learning experiences (Akshoomoff et al., 2013). Word knowledge and vocabulary are relatively resistant to acquired brain dysfunction and are considered a reasonable proxy for premorbid functioning (Blair & Spreen, 1989; Strauss, Sherman, & Spreen, 2006). The NIHTB fluid cognition composite score is comprised of the flanker, dimensional card sort, picture sequence memory, list sorting, and pattern comparison, and reflects one’s capacity for new learning and information processing in novel situations (Akshoomoff et al., 2013).

NIH Toolbox 9-Hole Pegboard Dexterity Test is part of the NIHTB Motor Battery, not the Cognition Battery, but was used as a covariate in some analyses. It is a timed test of participants’ manual dexterity (Reuben et al., 2013). Participants retrieve plastic pegs from a well, place them in a pegboard one at a time, and then return the pegs to the well one at a time. Participants completed one practice trial and one timed trial with each hand.

Data and Analyses

Most analyses excluded participants with incomplete data because we aimed to report on the cognitive profiles of individuals with SCI. Multivariate analysis of variance (MANOVA) and covariance (MANCOVA) compared SCI participants’ subtest and composite scores with a control group that was drawn from the NIHTB validation sample. The matching of control participants was done by parsing the SCI group into combinations of age, sex, race/ethnicity, and education (“cells”), selecting the same number of individuals from the NIHTB validation sample to populate each cell, and then fine-tuning (adding or removing individual control participants) as needed to achieve a control sample with equivalent socio-demographics as the SCI sample.

Results

Missing data

Initially, there were 208 individuals with SCI who completed at least some of the NIHTB-CB tests. Fifty-two (25.0%) of these SCI participants had at least one of the 7 cognitive test scores missing, which was significantly more than 10.1% of the 208 control participants: χ2(1) = 16.0, p<.01. Table 1 displays missing data, stratified by SCI characteristics and NIHTB-CB subtest. A disproportionate amount of missing data was from individuals with tetraplegia, particularly on speed-based tests. This was mainly because it was not possible to make accommodations or because participants were unable to complete the task even with recommended accommodations (Magasi et al., Submitted). After excluding SCI participants with incomplete data, 156 remained, and were re-matched with 156 control participants from the NIH Toolbox validation sample. These participants’ demographic and injury characteristics are displayed in Table 2.

Table 1.

Missing Data

SCI (N = 208) Control (N = 208) Statistic p
Missing ≥1 NIHTB-CB Score: % 25.0 10.1 χ2(1) = 16.0 <.01
 Paraplegia only (N= 102) 9.8 --
 Tetraplegia only (N= 106) 39.6 -- χ2(1) = 24.6 <.01
 Complete only (N=102) 32.4 --
 Incomplete only (N=105) 18.1 -- χ2(1) =5.6 =.02
Number Miss. Scores: mean (SD) 0.70 (1.4) 0.18 (0.7) F(1,414) = 24.3 < .01
 Paraplegia only (N= 102) 0.22 (0.9) --
 Tetraplegia only (N= 106) 1.16 (1.6) -- F(1,206) = 27.8 <.01
 Complete only (N=102) 0.89 (1.4) --
 Incomplete only (N=105) 0.52 (1.3) -- F(1,205) = 3.8 =.05
% Participants with Miss Scores
 Picture Vocabulary 1.4 7.7 χ2(1) = 7.9 < .01
 Oral Reading 1.9 0.5 χ2(1) = 0.8 N.S.
 Picture Sequence Memory 3.8 7.2 χ2(1) = 2.25 N.S.
 List Sorting 1.0 1.0 χ2(1) = 0.0 N.S.
 Pattern Comparison 19.7 1.0 χ2(1) = 37.4 <.01
 Flanker 20.7 0.5 χ2(1) = 42.7 <.01
 Dimensional Change Card Sorting 21.6 0.5 χ2(1) = 45.2 <.01
Open in a new tab

N.S. = not significant (p > .05)

Table 2.

Participant demographics and injury characteristics

Variable SCI (N = 156) Control (N = 156)
Age in years: mean (SD) 45.7 (13.8) 45.5 (13.9)
Sex (%male) 78.2 76.3
Race (%)
 White/Caucasian 62.8 68.6
 Black/African American 26.3 26.9
 Other 10.9 4.5
Ethnicity (%)
 Not Hispanic or Latino 90.9 89.9
 Hispanic or Latino 8.7 9.7
 Missing/not provided 0.5 0.5
Education: mean (SD) 13.9 (2.4) 14.0 (2.5)
 <12 years (%) 10.9 12.2
 12 years (%) 18.6 17.9
 13–15 years (%) 37.2 36.5
 16+ years (%) 33.3 33.3
Work status (%)
 Full-time 19.9 --
 Part-time 19.2 --
 Volunteer 0.0 --
 Not employed 57.7 --
 Unknown 3.2 100.0
Time since injury (years) 12.1 (10.2) --
Injury type (%)
 Paraplegia complete 32.1 --
 Paraplegia incomplete 26.3 --
 Paraplegia unknown 0.6 --
 Tetraplegia complete 12.2 --
 Tetraplegia incomplete 28.8 --
Cause of SCI (%)
 Motor vehicle accident 44.9 --
 Fall 16.7 --
 Diving 7.1 --
 Other sports 3.2 --
 Violence 22.4 --
 Other 5.1 --
 Unknown/missing 0.6 --
Open in a new tab

NIH Toolbox – Cognitive Battery

As shown in Table 3, SCI and control participants performed similarly on the NIHTB-CB crystallized composite score. SCI participants produced lower scores than controls on the fluid composite score with a medium effect size (Cohen’s d = 0.50). A MANOVA of the subtest scores supported this finding: F(7, 303) = 3266, p<.001, partial eta2 = .08. Follow-up univariate analyses with these participants (i.e., those with complete data) indicated that this effect was driven primarily by the flanker and dimensional card sorting tests. Table 3 also displays the proportion of group members who scored below 1 and 2 standard deviations of the NIHTB-CB normative sample for each subtest and composite score. Nearly every score fell within three standard deviations of the NIHTB-CB normative sample mean (T=50). One individual with SCI produced a score of T=17 on the picture sequence memory test. However, we did not treat this datum as an outlier because several other participants, both with and without SCI, produced scores in the T=20–30 range on this test and others.

Table 3.

NIH Toolbox Cognitive Battery Scores: SCI vs. Control Participants (Complete Data)

Composite Scores SCI Control F p d % <1 SD % <2 SD
Mean (SD) Mean (SD) SCI Control SCI Control
 Fluid 46.3 (8.6) 50.8 (10.0) 17.45 < .001 0.50 25.0 15.4 4.5 1.9
 Crystallized 50.7 (10.1) 50.8 (9.7) 0.01 .92 0.01 15.4 12.8 0.0 0.6
Subtest Scores
 Picture Vocabulary 51.3 (10.2) 50.4 (9.6) 0.56 .45 0.09 16.0 13.5 1.3 1.9
 Oral Reading Recognition 50.1 (10.2) 51.2 (10.3) 0.96 .34 0.11 16.0 14.8 0.6 1.3
 Picture Sequence Memory 47.4 (9.5) 50.5 (10.1) 7.35 < .01 0.32 21.8 15.5 3.2 0.6
 Pattern Comparison 48.6 (8.4) 51.4 (9.7) 7.14 < .01 0.31 14.1 11.0 1.3 1.3
 List Sorting 49.5 (9.3) 50.7 (10.0) 1.33 .25 0.12 13.5 18.1 1.9 1.9
 Flanker 46.4 (9.2) 50.2 (9.7) 12.91 < .001 0.40 24.4 16.1 4.5 2.6
 Dimensional Change Card Sort 46.5 (9.1) 50.6 (9.5) 15.00 < .001 0.44 26.3 10.3 1.9 3.2
Open in a new tab

Note: %<1 SD indicates the percentage of each group that produced scores below 1 standard deviation (T<40) of the entire NIHTB-CB normative sample.

By focusing analysis on individuals with complete data, we excluded participants who were not able to produce a key press with any finger. However, it is possible that the sample with complete data was biased in other ways, for example, if the excluded participants produced lower scores than the included participants. To test this possibility, we performed univariate ANOVAs for each of the 7 NIHTB-CB subtests with all participants (from the samples of 208) with NIHTB-CB data. Because of inconsistently missing data, the number of participants included in each ANOVA is unique. The results of these analyses (Table 4) closely mirror the results of the univariate analyses of cases with complete data (Table 3), suggesting that there was not a systematic bias in the subset of participants with complete cognitive data other than the exclusion of participants without sufficient hand functioning.

Table 4.

NIH Toolbox Cognitive Battery Scores: SCI vs. Control Participants (Full Sample)

SCI Control F p d
Subtest Scores N Mean (SD) N Mean (SD)
 Picture Vocabulary 204 50.5 (10.3) 191 50.2 (9.4) 0.11 .74 0.01
 Oral Reading Recognition 203 49.7 (10.2) 206 50.6 (10.1) 0.75 .39 0.09
 Picture Sequence Memory 176 47.0 (9.7) 192 50.1 (9.6) 9.52 <.01 0.32
 Pattern Comparison 167 48.8 (8.6) 206 50.9 (9.3) 6.62 <.01 0.23
 List Sorting 205 49.1 (9.6) 205 50.0 (10.1) 0.92 .34 0.09
 Flanker 165 46.1 (9.4) 206 50.2 (9.4) 15.29 <.001 0.44
 Dimensional Change Card Sort 163 46.2 (9.1) 206 50.2 (9.1) 16.54 <.001 0.44
Open in a new tab

Differences Among Subgroups of Individuals with SCI

To investigate the relationship between the cause of SCI (Table 2) and cognition, we conducted a Kruskal-Wallis test. This analysis revealed no effect of SCI cause on the fluid cognition composite score: X2(5) = 7.6, p = .18.

A MANOVA of subtest scores revealed that individuals with tetraplegia (n = 64) and paraplegia (n = 92) produced significantly different scores: F(7,148) = 1858, p <.001. Univariate ANOVAs (Table 5) showed that these participants produced crystallized composite scores that were equivalent to one another and generally equal to the mean and standard deviation of the NIHTB normative data. However, the group with tetraplegia produced significantly lower scores on the fluid composite score than the group with paraplegia. Specifically, the group with tetraplegia scored lower on the pattern comparison, flanker, and dimensional change card sorting tests.

Table 5.

NIH Toolbox Cognitive Battery Scores: Paraplegia vs. Tetraplegia

Tetra (N=64) Para (N=92) F p d
Composite Scores
 Fluid 43.0 (8.8) 48.6 (7.8) 17.3 < .001 .67
 Crystallized 51.9 (10.0) 50.5 (10.2) 0.1 .78 .14
Subtest Scores
 Picture Vocabulary 51.2(10.0) 51.3(10.4) <0.01 .95 .01
 Oral Reading Recognition 50.7(10.4) 49.7(10.1) 0.36 .55 .09
 Picture Sequence Memory 47.0(10.6) 47.8(8.6) 0.26 .61 .08
 Pattern Comparison 45.5(7.8) 50.8(8.1) 16.53 < .001 .67
 List Sorting 49.3(10.1) 49.6(8.7) 0.04 .85 .03
 Flanker 42.3(8.9) 49.2(8.4) 24.15 < .001 .80
 Dimensional Change Card Sorting 43.6(8.9) 48.5(8.8) 11.61 < .01 .55
Open in a new tab

Covarying for Manual Dexterity

Cognitive scores were only recorded and analyzed if participants could complete the tasks without accommodations or with the recommended accommodations. Yet, three of the four cognitive subtests with group differences (control vs. all-SCI), Pattern Comparison, Flanker, and Dimensional Change Card Sort, were subtests that required a timed motor response. To further investigate whether group differences in cognitive subtests were affected by differences in manual speed and dexterity, we applied participants’ NIH Toolbox 9-Hole Pegboard Dexterity Test scores as a covariate. Because reasonable accommodations on the cognitive tasks would have permitted use of the dominant or nondominant hand, we applied whichever NIH Toolbox 9-Hole Pegboard Dexterity Test score (left or right hand) was the fastest (SCI mean time = 23.3 seconds, SD = 26.4; Control mean = 18.3 seconds, SD = 5.6).

Of participants with complete cognitive data, Pegboard data were missing for 10 (6.4%) control participants and 32 (20.5%) SCI participants (χ2 (1) =13.32, p<.001). Although data were missing for different reasons, the main reason they were missing for SCI individuals was because their hand function was not sufficient to complete the Pegboard test but was good enough to press a button on the cognitive tests. Indeed, 24 (75%) of the 32 individuals with SCI who were missing Pegboard scores had tetraplegia rather than paraplegia, which was a significant asymmetry: (χ2(1)=19.2, p<.001). SCI participants without Pegboard scores produced lower scores on these three timed cognitive tests than SCI participants with Pegboard scores (F (3,152) = 7.45, p<.001, partial eta2 = .13), which may be due to cognitive and/or motor deficits. However, SCI participants without Pegboard data also scored lower on the Picture Sequence Memory test (t (52) = 2.56, p<.05), which did not require a timed motor response and could only be caused by cognitive deficits. Overall, it seems that participants with SCI who were included in the covariance analyses (i.e., those with Pegboard scores) may have had better cognitive and motor functioning than the SCI sample of 156 reported above (e.g., Table 3), possibly because they had worse injuries that may have more seriously affected brain function. Still, the covariance analysis remains helpful to test group differences in cognition independent of upper extremity function.

Individuals with SCI were compared to individuals without SCI on the Flanker, Card Sorting, and Pattern Comparison subtests using MANCOVA, covarying for fastest grooved pegboard scores. The group differences persisted: F(3,306) = 5.49, p=.001, partial eta2 = .051. Specifically, Flanker F(1,308) = 10.74, p=.001, partial eta2 = .034; Card Sort F(1,308)=14.11, p<.001, partial eta2 = .044; Picture Comparison F(1, 308) = 6.52, p=.011., partial eta2 = .021. This finding supports the hypothesis that the SCI group’s lower performance on the Flanker,Card Sort, and Picture Comparison tests was due to true differences in cognition rather than differences in upper extremity function and dexterity.

Discussion

Cognitive deficits after SCI are not the topic of many scientific investigations and may be clinically misinterpreted as nonadherence to treatment, inability to learn, poor coping, and/or poor motivation (Bradbury et al., 2008; Inoue et al., 2013; Kushner & Alvarez, 2014). This relative neglect and misunderstanding is unfortunate because cognitive limitations may affect some individuals’ rehabilitation, community reintegration, and/or quality of life (Bradbury et al., 2008; Davidoff et al., 1992). Studies of cognition after SCI are difficult to synthesize and apply to clinical practice because of variations in the chronicity of patients’ symptoms, the cognitive tests used, what accommodations were made for motor limitations, and in the normative data or comparison group. Here, we describe the performance of a large sample of community-dwelling individuals with SCI who completed the NIHTB-CB, a battery of neuropsychological tests with standardized accommodations for individuals with limited use of their hands (e.g., use of a response finger other than the dominant index finger) and with high quality normative data that are adjusted for age, sex, race, and education.

Individuals with and without SCI performed nearly identically on the crystallized cognition composite score, suggesting comparable premorbid functioning (Crawford, Parker, & Besson, 1988; Franzen, Burgess, & Smith-Seemiller, 1997). These group means were also similar to the mean of the total normative reference group for the NIHTB-CB (T = 50), which enhances confidence that the findings are not influenced by the samples’ demographic characteristics or differences in premorbid cognitive abilities.

Individuals with SCI produced lower scores than control participants on the fluid cognition composite score by an average of 4.5 T-score points. This corresponds with 25.0% of participants with SCI (versus 15.4% of participants without SCI) scoring below one standard deviation of the NIHTB-CB total normative sample mean on the fluid cognition composite score. Specifically, individuals with SCI performed worse than controls on tests of processing speed (Pattern Comparison) and executive function (Flanker and Card Sorting tests) and episodic memory (Picture Sequence Memory), but not working memory (List Sorting). These differences persisted even after statistically controlling for participants’ manual speed and dexterity.

These findings are consistent with those of Dowler et al. (1995) who performed a discriminant function analysis and found that processing speed differentiated a group with chronic SCI from a control group. Contrary to the findings of Bradbury et al. (2008) and Roth et al. (1989), however, we observed only small group differences on a test of episodic memory. The discrepancy may result from sample differences: those authors tested participants during acute rehabilitation, whereas our participants were community dwelling. Our results cannot inform the cognitive challenges faced during acute recovery, but suggest that cognitive screening with community-dwelling individuals with SCI might target processing speed and executive functioning.

Findings may be affected by excluding participants with incomplete cognitive data (e.g., for motor reasons or other unknown factors). However, control participants with incomplete data were also excluded, and when all cases available for a specific variable were analyzed, the results were nearly identical to the results of the complete-data cases. Similar findings between the two sets of analyses enhances our confidence that the subset of participants with complete data was not systematically biased for reasons other than excluding individuals with insufficient hand functioning. However, a limitation of this study is that we did not include an in-depth debriefing and reporting for causes of each missing datum and for specific accommodations that were made. It is possible that despite offering frequent rest breaks SCI participants’ cognitive test scores were more influenced by fatigue than the control participants. However, it is also possible that our results underestimate the magnitude of cognitive deficits in this population. Our sample with tetraplegia produced the lowest cognitive scores, and they may be a relatively high functioning group of persons with tetraplegia; they had sufficient hand functioning to rapidly press a button and were able to overcome the challenges associated with participating in a research study.

Reported rates of concurrent SCI and TBI vary widely in the literature, from 16% to 74% (Davidoff et al., 1988; Ghobrial et al., 2014; Kushner & Alvarez, 2014; Macciocchi et al., 2008). One limitation of this study is that we do not know the rate of known concurrent TBI in this sample, a challenge shared by other studies of cognition in community-dwelling individuals with SCI (Dowler et al., 1997; Dowler et al., 1995). While concurrent TBI is a main risk factor for cognitive deficits after SCI, cognitive deficits and TBI do not overlap perfectly, and the discussion of cognitive deficits after SCI is more complicated than presence or absence of TBI. The influence of concurrent TBI on cognition and rehabilitation after SCI is clouded by several factors, including difficulties in accurately identifying cases, as well as misconceptions and misattributions about TBI (Block et al., 2014; Block, West, & Goldin, 2015; McKinlay, Bishop, & McLellan, 2011). Identifying cases for research can be difficult because TBI may not be diagnosed or diagnosed accurately, especially in acute settings. For example, a 2014 study found that of patients with SCI and TBI who were referred for inpatient rehabilitation, 58.5% did not have TBI diagnosed while in acute care (Sharma et al., 2014). Furthermore, retrospective studies that rely on International Classification of Disease (ICD) codes for TBI status may reflect inconsistent and inaccurate coding of mild TBI (Bazarian, Veazie, Mookerjee, & Lerner, 2006). Furthermore, TBI is only one risk factor for cognitive deficits; two studies convincingly demonstrated no additional cognitive effects of mild TBI among individuals with SCI (Hess et al., 2003; Macciocchi et al., 2013). In summary, the discussion of cognitive deficits after SCI is complicated, particularly regarding potentially contributory risk factors, and requires consideration of factors other than the documented presence or absence of TBI.

Conclusion

Effect sizes associated with cognitive difficulties experienced by community-dwelling individuals with SCI are relatively small, and with the exception of the NIHTB-CB measure of episodic memory, limited to tests of processing speed and executive functioning. The NIHTB-CB nevertheless may be a useful screening tool for many individuals with SCI because it is relatively brief, samples most cognitive domains, has good normative data, and has recommended accommodations for individuals with motor limitations. However, it is not a “motor free” battery and is not appropriate for all persons and contexts. Timed cognitive tests (i.e., Flanker, Dimensional Change Card Sort, and Pattern Comparison) are inaccessible for individuals who cannot performed a rapid button press (NIH, 2012). The results presented here also suggest that individuals who can perform a button press but cannot perform the NIH Toolbox 9-Hole Pegboard Dexterity Test are likely to score low on the timed cognitive tests, but it is unclear if it is for cognitive or motor reasons. The minimum level of upper extremity function needed to complete the timed cognitive subtests (Flanker, Dimensional Change Card Sort, and Pattern Comparison) deserves future investigation.

One of the main reasons why cognitive assessment is important in this population is that it may help clarify and predict functional challenges faced by these individuals every day. Future studies could examine the consequences of cognitive deficits on the everyday functioning (e.g., self-care) of individuals with SCI, and further investigate the influence of preinjury and medical variables on cognition (Macciocchi et al., 2013). Concurrent administration of tests measuring “everyday cognition” and functional limitations, for example, the Executive Function Performance Test (Baum et al., 2008) and the Texas Functional Living Scale (Cullum et al., 2001), would also be valuable.

Impact.

  • Previous research on cognition after SCI has produced variable results. This variability is partially caused by normative data or control groups that are adjusted or matched for different factors. This study uses a cognitive battery where each score is adjusted for the same demographic factors.

  • This study supports previous findings that community-dwelling individuals with SCI have subtle deficits on tests of processing speed and executive functioning.

  • Our results suggest that cognitive screening with community-dwelling individuals with SCI should target processing speed and executive functioning.

Acknowledgments

This research was supported by grant #H133B090024 (PI: Heinemann) from the National Institute on Disability, Independent Living, and Research Rehabilitation (NIDILRR), as well as Psychosocial Research Postdoctoral Fellowship Grants from the Craig H. Neilsen Foundation (#290474; PI: Wong; #364962; PI: Cohen). Salary support for Dr. Cohen was also provided by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number U54-GM104941 (PI: Binder-Macleod). The authors would like to thank their talented and dedicated research staff: Ana Miskovic, Arielle Goldsmith, Sara Jerousek, Sue Tucker, Jessica Dashner, Cathy Crawley, Ben Aziz, Julie Grech, Livia Vanden Belt, and Angela Miciura.

Footnotes

An abstract based on earlier data was accepted for presentation as a poster at the 2016 Annual Conference of the International Spinal Cord Society. These data were also presented at the 2017 Annual Conference of APA Division 22 (Rehabilitation Psychology).

Contributor Information

Matthew L. Cohen, University of Delaware

David S. Tulsky, University of Delaware and Kessler Foundation

James A. Holdnack, University of Delaware and Pearson Assessment

Noelle E. Carlozzi, University of Michigan

Alex Wong, Washington University.

Susan Magasi, University of Illinois at Chicago.

Robert K. Heaton, University of California San Diego

Allen W. Heinemann, Rehabilitation Institute of Chicago and Northwestern University

References

  1. Akshoomoff N, Beaumont JL, Bauer PJ, Dikmen SS, Gershon RC, Mungas D, … Heaton RK. NIH Toolbox Cognition Battery (CB): composite scores of crystallized, fluid, and overall cognition. Monogr Soc Res Child Dev. 2013;78(4):119–132. doi: 10.1111/mono.12038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. APA. Standards for educational and psychological testing. Washington, DC: American Psychological Association; 2014. [Google Scholar]
  3. Avluk OC, Gurcay E, Gurcay AG, Karaahmet OZ, Tamkan U, Cakci A. Effects of chronic pain on function, depression, and sleep among patients with traumatic spinal cord injury. Annals of Saudi Medicine. 2014;34(3):211–216. doi: 10.5144/0256-4947.2014.211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Baum CM, Connor LT, Morrison T, Hahn M, Dromerick AW, Edwards DF. Reliability, validity, and clinical utility of the Executive Function Performance Test: A measure of executive function in a sample of people with stroke. American Journal of Occupational Therapy. 2008;62(4):446–455. doi: 10.5014/ajot.62.4.446. [DOI] [PubMed] [Google Scholar]
  5. Bazarian JJ, Veazie P, Mookerjee S, Lerner EB. Accuracy of mild traumatic brain injury case ascertainment using ICD-9 codes. Academic emergency medicine. 2006;13(1):31–38. doi: 10.1197/j.aem.2005.07.038. [DOI] [PubMed] [Google Scholar]
  6. Blair JR, Spreen O. Predicting premorbid IQ: a revision of the National Adult Reading Test. Clin Neuropsychol. 1989;3(2):129–136. [Google Scholar]
  7. Block CK, Fabrizio K, Bagley B, Hannah J, Camp S, Mindingall N, … Lokken K. Assessment of Veteran and caregiver knowledge about mild traumatic brain injury in a VA medical center. Journal of Head Trauma Rehabilitation. 2014;29(1):76–88. doi: 10.1097/HTR.0b013e3182886d78. [DOI] [PubMed] [Google Scholar]
  8. Block CK, West SE, Goldin Y. Misconceptions and misattributions about traumatic brain injury: An integrated conceptual framework. PM&R. 2015 doi: 10.1016/j.pmrj.2015.05.022. [DOI] [PubMed] [Google Scholar]
  9. Bonanno GA, Kennedy P, Galatzer-Levy IR, Lude P, Elfstrom ML. Trajectories of resilience, depression, and anxiety following spinal cord injury. Rehabilitation Psychology. 2012;57(3):236–247. doi: 10.1037/a0029256. [DOI] [PubMed] [Google Scholar]
  10. Boyer B, Knolls M, Kafkalas C, Tollen L. Prevalence of posttraumatic stress disorder in patients with pediatric spinal cord injury: Relationship to functional independence. Topics in Spinal Cord Injury Rehabilitation. 2000;6(Supplement 1):125–133. [Google Scholar]
  11. Bradbury CL, Wodchis WP, Mikulis DJ, Pano EG, Hitzig SL, McGillivray CF, … Green RE. Traumatic brain injury in patients with traumatic spinal cord injury: clinical and economic consequences. Archives of Physical Medicine & Rehabilitation. 2008;89(12 Suppl):S77–84. doi: 10.1016/j.apmr.2008.07.008. [DOI] [PubMed] [Google Scholar]
  12. Carlozzi NE, Tulsky DS, Chiaravalloti ND, Beaumont JL, Weintraub S, Conway K, Gershon RC. NIH Toolbox Cognitive Battery (NIHTB-CB): the NIHTB Pattern Comparison Processing Speed Test. Journal of the International Neuropsychological Society. 2014;20(6):630–641. doi: 10.1017/S1355617714000319. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Casaletto KB, Umlauf A, Beaumont J, Gershon R, Slotkin J, Akshoomoff N, Heaton RK. Demographically corrected normative standards for the English version of the NIH Toolbox Cognition Battery. Journal of the International Neuropsychological Society. 2015;21(05):378–391. doi: 10.1017/S1355617715000351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Cicerone KD, Langenbahn DM, Braden C, Malec JF, Kalmar K, Fraas M, … Ashman T. Evidence-based cognitive rehabilitation: updated review of the literature from 2003 through 2008. Archives of physical medicine and rehabilitation. 2011;92(4):519–530. doi: 10.1016/j.apmr.2010.11.015. [DOI] [PubMed] [Google Scholar]
  15. Crawford JR, Parker DM, Besson JA. Estimation of premorbid intelligence in organic conditions. The British Journal of Psychiatry. 1988;153(2):178–181. doi: 10.1192/bjp.153.2.178. [DOI] [PubMed] [Google Scholar]
  16. Cullum CM, Saine K, Chan LD, Martin-Cook K, Gray KF, Weiner MF. Performance-based instrument to assess functional capacity in dementia: The Texas Functional Living Scale. Cognitive and Behavioral Neurology. 2001;14(2):103–108. [PubMed] [Google Scholar]
  17. Davidoff G, Morris J, Roth E, Bleiberg J. Cognitive dysfunction and mild closed head injury in traumatic spinal cord injury. Archives of physical medicine and rehabilitation. 1985;66(8):489–491. [PubMed] [Google Scholar]
  18. Davidoff G, Roth E, Richards J. Cognitive deficits in spinal cord injury: epidemiology and outcome. Archives of physical medicine and rehabilitation. 1992;73(3):275–284. [PubMed] [Google Scholar]
  19. Davidoff G, Thomas P, Johnson M, Berent S, Dijkers M, Doljanac R. Closed head injury in acute traumatic spinal cord injury: incidence and risk factors. Archives of physical medicine and rehabilitation. 1988;69(10):869–872. [PubMed] [Google Scholar]
  20. DeJong G, Tian W, Hsieh CH, Junn C, Karam C, Ballard PH, … Backus D. Rehospitalization in the first year of traumatic spinal cord injury after discharge from medical rehabilitation. Archives of physical medicine and rehabilitation. 2013;94(4 Suppl):S87–97. doi: 10.1016/j.apmr.2012.10.037. [DOI] [PubMed] [Google Scholar]
  21. Dijkers M, Bryce T, Zanca J. Prevalence of chronic pain after traumatic spinal cord injury: a systematic review. Journal of Rehabilitation Research and Development. 2009;46(1):13–29. [PubMed] [Google Scholar]
  22. Dikmen SS, Bauer PJ, Weintraub S, Mungas D, Slotkin J, Beaumont JL, … Heaton RK. Measuring episodic memory across the lifespan: NIH Toolbox Picture Sequence Memory Test. Journal of the International Neuropsychological Society. 2014;20(6):611–619. doi: 10.1017/S1355617714000460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Dowler RN, Harrington DL, Haaland KY, Swanda RM, Fee F, Fiedler K. Profiles of cognitive functioning in chronic spinal cord injury and the role of moderating variables. Journal of the International Neuropsychological Society. 1997;3(5):464–472. [PubMed] [Google Scholar]
  24. Dowler RN, O’Brien SA, Haaland KY, Harrington DL, Feel F, Fiedler K. Neuropsychological functioning following a spinal cord injury. Appl Neuropsychol. 1995;2(3–4):124–129. doi: 10.1080/09084282.1995.9645349. [DOI] [PubMed] [Google Scholar]
  25. Dryden DM, Saunders LD, Rowe BH, May LA, Yiannakoulias N, Svenson LW, … Voaklander DC. Depression following traumatic spinal cord injury. Neuroepidemiology. 2005;25(2):55–61. doi: 10.1159/000086284. [DOI] [PubMed] [Google Scholar]
  26. Ferris FL, 3rd, Kassoff A, Bresnick GH, Bailey I. New visual acuity charts for clinical research. Am J Ophthalmol. 1982;94(1):91–96. [PubMed] [Google Scholar]
  27. Franzen MD, Burgess EJ, Smith-Seemiller L. Methods of estimating premorbid functioning. Archives of Clinical Neuropsychology. 1997;12(8):711–738. [PubMed] [Google Scholar]
  28. Gershon RC, Cook KF, Mungas D, Manly JJ, Slotkin J, Beaumont JL, Weintraub S. Language measures of the NIH Toolbox Cognition Battery. J Int Neuropsychol Soc. 2014;20(6):642–651. doi: 10.1017/S1355617714000411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Gershon RC, Slotkin J, Manly JJ, Blitz DL, Beaumont JL, Schnipke D, … Weintraub S. NIH Toolbox Cognition Battery (CB): measuring language (vocabulary comprehension and reading decoding) Monogr Soc Res Child Dev. 2013;78(4):49–69. doi: 10.1111/mono.12034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Ghobrial GM, Amenta PS, Maltenfort M, Williams KA, Jr, Harrop JS, Sharan A, … Prasad S. Longitudinal incidence and concurrence rates for traumatic brain injury and spine injury - a twenty year analysis. Clinical Neurology and Neurosurgery. 2014;123:174–180. doi: 10.1016/j.clineuro.2014.05.013. [DOI] [PubMed] [Google Scholar]
  31. Hagen EM, Eide GE, Rekand T, Gilhus NE, Gronning M. Traumatic spinal cord injury and concomitant brain injury: a cohort study. Acta Neurologica Scandinavica. 2010;(190):51–57. doi: 10.1111/j.1600-0404.2010.01376.x. [DOI] [PubMed] [Google Scholar]
  32. Hassanijirdehi M, Khak M, Afshari-Mirak S, Holakouie-Naieni K, Saadat S, Taheri T, Rahimi-Movaghar V. Evaluation of pain and its effect on quality of life and functioning in men with spinal cord injury. Korean Journal of Pain. 2015;28(2):129–136. doi: 10.3344/kjp.2015.28.2.129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Heinemann AW, Keen M, Donohue R, Schnoll S. Alcohol use by persons with recent spinal cord injury. Archives of physical medicine and rehabilitation. 1988;69(8):619–624. [PubMed] [Google Scholar]
  34. Herrick S, Elliott TR, Crow F. Self-appraised problem-solving skills and the prediction of secondary complications among persons with spinal cord injuries. Journal of Clinical Psychology in Medical Settings. 1994;1(3):269–283. doi: 10.1007/BF01989628. [DOI] [PubMed] [Google Scholar]
  35. Hess DW, Marwitz JH, Kruetzer JS. Neuropsychological impairments after spinal cord injury: A comparative study with mild traumatic brain injury. Rehabilitation Psychology. 2003;48(3):151–156. [Google Scholar]
  36. Inoue T, Lin A, Ma X, McKenna SL, Creasey GH, Manley GT, … Beattie MS. Combined SCI and TBI: recovery of forelimb function after unilateral cervical spinal cord injury (SCI) is retarded by contralateral traumatic brain injury (TBI), and ipsilateral TBI balances the effects of SCI on paw placement. Experimental Neurology. 2013;248:136–147. doi: 10.1016/j.expneurol.2013.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Kirshblum SC, Burns SP, Biering-Sorensen F, Donovan W, Graves DE, Jha A, … Waring W. International standards for neurological classification of spinal cord injury (revised 2011) Journal of Spinal Cord Medicine. 2011;34(6):535–546. doi: 10.1179/204577211X13207446293695. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Krause JS, Saunders LL, Newman S. Posttraumatic stress disorder and spinal cord injury. Archives of physical medicine and rehabilitation. 2010;91(8):1182–1187. doi: 10.1016/j.apmr.2010.05.012. [DOI] [PubMed] [Google Scholar]
  39. Kushner DS, Alvarez G. Dual diagnosis: traumatic brain injury with spinal cord injury. Physical Medicine and Rehabilitation Clinics of North America. 2014;25(3):681–696. ix–x. doi: 10.1016/j.pmr.2014.04.005. [DOI] [PubMed] [Google Scholar]
  40. Lazzaro I, Tran Y, Wijesuriya N, Craig A. Central correlates of impaired information processing in people with spinal cord injury. Journal of Clinical Neurophysiology. 2013;30(1):59–65. doi: 10.1097/WNP.0b013e31827edb0c. [DOI] [PubMed] [Google Scholar]
  41. Lyon JL, Baker RK, Gren LH. Attention deficit hyperactivity disorder and increased risk of injury. Advances in medical sciences. 2009;54(1):20–26. doi: 10.2478/v10039-009-0022-7. [DOI] [PubMed] [Google Scholar]
  42. Macciocchi S, Bowman B, Coker J, Apple D, Leslie D. Effect of co-morbid traumatic brain injury on functional outcome of persons with spinal cord injuries. American Journal of Physical Medicine & Rehabilitation. 2004;83(1):22–26. doi: 10.1097/01.PHM.0000104661.86307.91. [DOI] [PubMed] [Google Scholar]
  43. Macciocchi S, Seel RT, Thompson N, Byams R, Bowman B. Spinal cord injury and co-occurring traumatic brain injury: assessment and incidence. Archives of physical medicine and rehabilitation. 2008;89(7):1350–1357. doi: 10.1016/j.apmr.2007.11.055. [DOI] [PubMed] [Google Scholar]
  44. Macciocchi S, Seel RT, Warshowsky A, Thompson N, Barlow K. Co-occurring traumatic brain injury and acute spinal cord injury rehabilitation outcomes. Archives of physical medicine and rehabilitation. 2012;93(10):1788–1794. doi: 10.1016/j.apmr.2012.01.022. [DOI] [PubMed] [Google Scholar]
  45. Macciocchi S, Seel R, Thompson N. The impact of mild traumatic brain injury on cognitive functioning following co-occurring spinal cord injury. Archives of Clinical Neuropsychology. 2013;28(7):684–691. doi: 10.1093/arclin/act049. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Magasi S, Harniss M, Tulsky DS, Cohen ML, Heaton RK, Heinemann A. Frequency of Test Accomodations for Individuald with Neurological Conditions Completing the NIH Toolbox - Cognition Battery. doi: 10.1037/rep0000191. (Submitted) [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Masri R, Keller A. Chronic pain following spinal cord injury. Advances in Experimental Medicine and Biology. 2012;760:74–88. doi: 10.1007/978-1-4614-4090-1_5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. McKinlay A, Bishop A, McLellan T. Public knowledge of ‘concussion’ and the different terminology used to communicate about mild traumatic brain injury (MTBI) Brain injury. 2011;25(7–8):761–766. doi: 10.3109/02699052.2011.579935. [DOI] [PubMed] [Google Scholar]
  49. Migliorini C, Sinclair A, Brown D, Tonge B, New P. Prevalence of mood disturbance in Australian adults with chronic spinal cord injury. Internal Medicine Journal. 2015 doi: 10.1111/imj.12825. [DOI] [PubMed] [Google Scholar]
  50. NIH. NIH Toolbox Reasonable Accomodations Guidelines. 2012 Retrieved from http://www.nihtoolbox.org/HowDoI/reasonableacommodations/Accessibility documents/NIH Toolbox Reasonable Accommodations Guidelines.pdf.
  51. Reuben DB, Magasi S, McCreath HE, Bohannon RW, Wang YC, Bubela DJ, … Gershon RC. Motor assessment using the NIH Toolbox. Neurology. 2013;80(11 Suppl 3):S65–75. doi: 10.1212/WNL.0b013e3182872e01. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Roth E, Davidoff G, Thomas P, Doljanac R, Dijkers M, Berent S, … Yarkony G. A controlled study of neuropsychological deficits in acute spinal cord injury patients. Paraplegia. 1989;27(6):480–489. doi: 10.1038/sc.1989.75. [DOI] [PubMed] [Google Scholar]
  53. Sharma B, Bradbury C, Mikulis D, Green R. Missed diagnosis of traumatic brain injury in patients with traumatic spinal cord injury. Journal of Rehabilitation Medicine. 2014;46(4):370–373. doi: 10.2340/16501977-1261. [DOI] [PubMed] [Google Scholar]
  54. Slotkin J, Kallen M, Griffith J, Magasi S, Salsman J, Nowinski C, Gershon R. Technical Manual of the NIH Toolbox - Cognition Battery. 2012 Retrieved from http://www.nihtoolbox.org/HowDoI/TechnicalManual/Technical Manual sections/Toolbox Cognition Battery Composite Scores Technical Manual.pdf.
  55. Strauss E, Sherman EMS, Spreen O. A Compendium of Neuropsychological Tests. 3. Oxford: Oxford University Press; 2006. [Google Scholar]
  56. Tulsky DS, Kisala PA. The Spinal Cord Injury - Quality of Life (SCI-QOL) measurement system: Development, psychometrics, and item bank calibration. Journal of Spinal Cord Medicine. 2015;38(3):251–256. doi: 10.1179/2045772315Y.0000000035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Weintraub S, Dikmen SS, Heaton RK, Tulsky DS, Zelazo PD, Bauer PJ, … Gershon RC. Cognition assessment using the NIH Toolbox. Neurology. 2013;80(11 Suppl 3):S54–64. doi: 10.1212/WNL.0b013e3182872ded. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Weintraub S, Dikmen SS, Heaton RK, Tulsky DS, Zelazo PD, Slotkin J, … Gershon R. The cognition battery of the NIH toolbox for assessment of neurological and behavioral function: validation in an adult sample. Journal of the International Neuropsychological Society. 2014;20(6):567–578. doi: 10.1017/S1355617714000320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Wilkinson GS, Robertson GJ. Wide Range Achievement Test (WRAT4) Lutz, FL: Psychological Assessment Resources; 2006. [Google Scholar]
  60. Wilmot CB, Cope DN, Hall K, Acker M. Occult head injury: its incidence in spinal cord injury. Archives of physical medicine and rehabilitation. 1985;66(4):227–231. doi: 10.1016/0003-9993(85)90148-0. [DOI] [PubMed] [Google Scholar]
  61. Zelazo PD, Anderson JE, Richler J, Wallner-Allen K, Beaumont JL, Conway KP, … Weintraub S. NIH Toolbox Cognition Battery (CB): validation of executive function measures in adults. Journal of the International Neuropsychological Society. 2014;20(6):620–629. doi: 10.1017/S1355617714000472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Zelazo PD, Anderson JE, Richler J, Wallner-Allen K, Beaumont JL, Weintraub S. NIH Toolbox Cognition Battery (CB): measuring executive function and attention. Monogr Soc Res Child Dev. 2013;78(4):16–33. doi: 10.1111/mono.12032. [DOI] [PubMed] [Google Scholar]

RESOURCES