Validation of the NIH Toolbox in Individuals with Neurologic Disorders

Abstract

Objective

Individuals with spinal cord injury (SCI), traumatic brain injury (TBI), and stroke experience a variety of neurologically related deficits across multiple domains of function. The NIH Toolbox for the Assessment of Neurological and Behavioral Function (NIHTB) examines motor, sensation, cognition, and emotional functioning. The purpose of this paper is to establish the validity of the NIHTB in individuals with neurologic conditions.

Methods

Community-dwelling individuals with SCI (n = 209), TBI (n = 184), or stroke (n = 211) completed the NIHTB. Relative risks for impaired performance were examined relative to a matched control groups.

Results

The largest group differences were observed on the Motor domain and for the Fluid Cognition measures. All groups were at increased risk for motor impairment relative to normative standards and matched controls. Fluid cognitive abilities varied across groups such that individuals with stroke and TBI performed more poorly than individuals with SCI; increased relative risks for impaired fluid cognition were seen for individuals in the stroke and TBI groups, but not for those in the SCI group. All three neurologic groups performed normally on most measures in the Sensation Battery, although TBI participants evidenced increased risk for impaired odor identification and the stroke group showed more vision difficulties. On the Emotion Battery, participants in all three groups showed comparably poor psychological well-being, social satisfaction, and self-efficacy, whereas the TBI group also evidenced slightly increased negative affect.

Conclusions

Data provide support for the validity of the NIHTB in individuals with neurologic conditions.

Introduction

Neuropsychologists typically rely on standardized cognitive assessments that evaluate brain functioning in order to aid in providing clinical diagnoses or to make recommendations about an individual's relative strengths and weaknesses following the diagnosis of a neurological illness/disease or traumatic insult to the brain (Lezak, Howieson, & Loring, 2004). Such relative strengths and weaknesses are determined by examining measures of (a) “crystallized cognition,” or those measures capture experience- or learning-based abilities that develop rapidly in childhood, then stabilize or even slightly improve with age and are relatively insensitive to effects of acquired brain dysfunction in adulthood (to determine relative strengths) and (b) “fluid cognition,” or those components of cognition that are considered to be more reflective of biological processes that typically change throughout the lifespan (peaking in the mid-20s–30s and then steadily declining), and are sensitive to acquired brain injury/disease (to determine areas of weakness). The NIH Toolbox (NIHTB) for the Assessment of Neurological Behavior and Function is designed to briefly and efficiently capture multiple domains of function including motor, sensory, cognitive, and emotional functioning (Gershon, Cella, Fox, Havlik, Hendrie, & Wagster, 2010). Although the NIHTB has great potential as a multimodal assessment  because it provides standardized measures of several domains of functioning, validity data are not yet available for individuals with neurologic conditions. Thus, this paper is focused on establishing validity data on the NIHTB in individuals with spinal cord injury (SCI), traumatic brain injury (TBI), and stroke.

There are approximately 280,000 individuals in the USA living with SCI (Singh, Tetreault, Kalsi-Ryan, Nouri, & Fehlings, 2014). These individuals frequently experience a number of physical complications including impaired mobility, loss of muscle tone, respiratory problems, bowel and bladder alterations, and circulatory dysregulation (Dudley-Javoroski & Shields, 2006; Haisma, Bussmann, Stam, Sluis, Bergen, Post et al., 2007; Li et al., 2012). With regard to sensation, chronic pain (Alschuler, Jensen, Sullivan-Singh, Borson, Smith, & Molton, 2013; Dudley-Javoroski & Shields, 2006; Haisma et al., 2007; Li et al., 2012; Murray et al., 2007) and loss of skin sensation (Gefen, 2014; Recio, Felter, Schneider, & McDonald, 2012; Westermann, Krumova, Pennekamp, Horch, Baron, & Maier, 2012) are common, and there is evidence to suggest that some individuals experience vestibular dysfunction (Ribaric-Jankes, Cobeljic, Svetel, & Pesic, 2009). In addition, it is estimated that as many as 40%–50% have cognitive difficulties (Davidoff, Roth, & Richards, 1992a; Murray et al., 2007; Richards, Brown, Hagglund, Bua, & Reeder, 1988) which often go undiagnosed (Davidoff et al., 1992a; Dowler, Harrington, Haaland, Swanda, Fee, & Fiedler, 1997; Narayan, Gokaslan, Bontke, & Berrol, 1990). Difficulties often include fluid cognition problems: problems with processing speed, attention, episodic memory, and executive functioning (Bradbury et al., 2008; Davidoff, Morris, Roth, & Bleiberg, 1985; Davidoff, Roth, & Richards, 1992b; Dowler et al., 1997; Dowler, O'Brien, Haaland, Harrington, Feel, & Fiedler, 1995; Hess, Zhan, Foo, & Yalla, 2003; Lazzaro, Tran, Wijesuriya, & Craig, 2013; Macciocchi, Seel, & Thompson, 2013; Roth et al., 1989; Wilmot, Cope, Hall, & Acker, 1985). Depression and anxiety are more prevalent than rates found in the general population (Alschuler et al., 2013; Dudley-Javoroski & Shields, 2006; Huang et al., 2015; Krueger, Noonan, Williams, Trenaman, & Rivers, 2013; Murray et al., 2007).

There are approximately 5.3 million individuals in the USA who are living with a complicated mild, moderate, or severe TBI (Thurman, Alverson, Dunn, Guerrero, & Sniezek, 1999). TBI can be associated with several disabling physical effects, including balance and motor coordination problems, fatigue, headache, sleep disturbances, seizures, sensory impairments, slurred speech, spasticity and tremors, problems in urinary control, dizziness and vestibular dysfunction, and weakness (Whyte, Hart, Laborde, & Rosenthal, 2005). Furthermore, sensory difficulties after TBI can include pain and olfactory dysfunction (Frasnelli et al., 2016; Schofield, Moore, & Gardner, 2014; Sigurdardottir et al., 2016). Difficulties with cognition often include the following aspects of fluid cognition: memory, attention, processing speed, verbal fluency, and executive function deficits (Cicerone et al., 1996, 2000, 2005, 2011; Dikmen, Corrigan, Levin, Machamer, Stiers, & Weisskopf, 2009; Palacios et al., 2012; West, Curtis, Greve, & Bianchini, 2011). Emotional problems can include apathy, irritability, denial and unawareness of deficits, as well as impulsivity, sexual disturbances, and childlike behavior (Grafman, Schwab, Warden, Pridgen, Brown, & Salazar, 1996; Kim, 2002), plus depression and anxiety (Brown, Gordon, & Spielman, 2003; Hibbard, Uysal, Kepler, Bogdany, & Silver, 1998).

There are approximately 6.6 million stroke survivors in the USA (Mozaffarian et al., 2016). Common physical impairments for stroke survivors include muscle weakness, or paralysis (Barrett & Muzaffar, 2014; Droste, Safo, Metz, & Osada, 2014; Soekadar, Birbaumer, Slutzky, & Cohen, 2014), as well as difficulty talking (Droste et al., 2014) or swallowing (Rothwell, Boaden, Bamford, & Tyrrell, 2013). In addition, stroke survivors often experience a multitude of cognitive difficulties in fluid cognition including memory (Droste et al., 2014), attention (Barker-Collo, Feigin, Lawes, Senior, & Parag, 2010; Rothwell et al., 2013), executive functioning (Dancause, Ptito, & Levin, 2002; Morrison et al., 2013), as well as slower processing speed (Loranger, Lussier, Pepin, Hopps, & Senecal, 2000; Sachdev et al., 2004; Su, Wuang, Lin, & Su, 2015). With regard to sensation, pain (Droste et al., 2014; Rothwell et al., 2013), numbness (Droste et al., 2014), and temperature sensitivity are common. Emotional difficulties can include depression (Lightbody, Auton et al., 2007; Lightbody, Baldwin et al., 2007; Rothwell et al., 2013) or emotional lability (Jones, O'Keeffe, Kingston, & Carroll, 2013).

Given the multitude of problems across all domains of functioning in SCI, TBI, and stroke survivors, it is important to evaluate functioning across all of these domains. In order to establish validity for the NIHTB in these different neurologic conditions, we need to establish that the pattern of findings across the different domains of functioning. Specifically, this paper will describe the different profiles of functioning across the three groups. Furthermore, this paper will describe clinical impairment rates (including the risk for impairment) for the three different clinical groups (TBI, SCI, and stroke) across four domains of functioning on the NIHTB (motor, sensation, cognition, and emotion). With regard to motor function, we anticipated that all three groups would demonstrate elevated risk for clinical impairment (relative to normative standards and demographic controls), with the SCI group demonstrating the highest relative risk among the three groups. With regard to cognition, we anticipated that all three groups would demonstrate a higher relative risk of clinical impairment on measures of fluid cognition (relative to normative standards and controls), with both the TBI and stroke groups demonstrating the highest levels. With regard to sensory function, we hypothesized that individuals with TBI will demonstrate vestibular dysfunction and olfactory impairments, whereas individuals with SCI will demonstrate vestibular dysfunction and that both SCI and TBI would have elevated relative risk for clinical impairment in these domains. Furthermore, we hypothesized that all three clinical groups would have elevated rates of pain interference. With regard to emotion, we expected all three groups to evidence elevated rates of distress for sadness, fear, anger, stress, perceived hostility and rejection, and lower satisfaction (i.e., more loneliness, less life satisfaction, less friendship). Such data are important for establishing validity data for the NIHTB in individuals with neurologic impairments. In addition, these data will provide clinicians with a better understanding of the different motor, emotional, cognitive, and sensory problems that individuals in each of these clinical groups may experience, and promote the utility of assessments, like the NIHTB, that provide information across multiple domains of function.

Materials and Methods

Participants

We recruited 604 individuals with medically documented neurological conditions: 209 SCI, 184 TBI, and 211 stroke participants. Participants were administered the NIHTB in English as part of a larger battery. Recruitment occurred at three facilities: Rehabilitation Institute of Chicago, Washington University in St. Louis, and the University of Michigan. Participants were at least 18 years old, able to comprehend and speak English at a fifth grade level, able to provide informed consent, and willing and able to return for follow-up testing. As the primary goal of this study was to examine long-term outcomes for individuals with neurological conditions (“Consensus conference. Rehabilitation of persons with traumatic brain injury. NIH Consensus Development Panel on Rehabilitation of Persons With Traumatic Brain Injury,” 1999), at least one year must have passed since their most recent injury or stroke to be considered eligible; thus, potential participants who were still in the acute stages of recovery (i.e., less than one year post-injury) were not eligible (Burns, Marino, Flanders, & Flett, 2012; Dikmen et al., 2009; Dikmen, Machamer, Winn, & Temkin, 1995; Dikmen, Mclean, & Temkin, 1986; Dikmen, Reitan, & Temkin, 1983; Ditunno, Stover, Freed, & Ahn, 1992; Jorgensen, Nakayama, Raaschou, Vive-Larsen, Stoier, & Olsen, 1995a, 1995b; Waters, Adkins, Yakura, & Sie, 1993, 1994, 1998; Waters, Yakura, Adkins, & Sie, 1992). Data were collected in accordance with and approval of the local institutional review boards.

With regard to SCI, individuals had a medically documented acute traumatic lesion of neural elements in the spinal canal, resulting in either temporary or permanent sensory or motor deficits (Kirshblum, Burns et al., 2011). The International Standards for Neurological Classification of SCI were used to characterize impairment severity (Kirshblum, Waring et al., 2011). Specifically, SCI participants were characterized as paraplegic (impairment or loss of motor and/or sensory function in the thoracic, lumbar, or sacral [but not cervical] segments of the spinal cord secondary to damage of neural elements within the spinal canal), or tetraplegic (impairment or loss of motor and/or sensory function in the cervical segments of the spinal cord due to damage of neural elements within the spinal canal). Furthermore, participants were characterized as either complete (an absence of sensory and motor function in the lowest sacral segment) or incomplete (partial preservation of sensory and/or motor function is found below the neurological level and includes the lowest sacral segment).

With regard to TBI, individuals had a medically confirmed diagnosis of complicated mild (Williams, Levin, & Eisenberg, 1990), moderate (Traumatic Brain Injury Model Systems National Data Center, 2006), or severe TBI (Traumatic Brain Injury Model Systems National Data Center, 2006). Emergency room Glasgow Coma Scale (GCS) scores and neuroimaging results were used to confirm TBI severity; severity was defined according to the lowest GCS score within the first 24 hr after injury (not due to intubation, sedation, or intoxication). Specifically, complicated mild TBI was defined as a GCS score of 13–15 with positive findings on neuroimaging; moderate TBI was defined as a GCS score of 9–12; and severe TBI was defined as a GCS score of 8 or below. In cases where GCS data were unavailable (n = 22), severity was determined based on the detailed description of injury and reviewed by a neuropsychologist on staff (independently of NIHTB results). The two TBI groups (those with GCS scores and those without) generally did not differ on the NIHTB cognition tests; therefore, we report on findings for the entire sample, subsequently.

With regard to stroke, individuals had medically documented, rapidly developing clinical signs of focal or global disturbance of cerebral function, with symptoms lasting more than 24 hr, and with no apparent cause other than that of vascular origin (Bothig, 1989; World Health Organization, 1990). The Modified Rankin Scale (van Swieten, Koudstaal, Visser, Schouten, & van Gijn, 1988) was used to classify participants as having either a mild (scores of 1–2), moderate (score of 3), or severe stroke (score of 4). Stroke participants were classified as having either an ischemic or a hemorrhagic event. Participants with stroke were excluded if they demonstrated significant evidence of aphasia due to the significant language comprehension and expression characteristics of the testing battery (as determined by the Frenchay Aphasia Test; Enderby, Wood, Wade, & Hewer, 1987).

To establish relative risk, a subset of analyses focused on examining each clinical group relative to a comparison group that was drawn from the NIHTB Toolbox Normative Study (Beaumont et al., 2013). Participants in the SCI and TBI groups were matched on race, age, education, and sex to controls from the NIHTB Normative Study. Participants within the stroke clinical group were matched on age, education, and sex but due to insufficient numbers of African Americans in the NIHTB Normative Study, total race-matching was not possible for this group.

Instruments

The NIHTB for the assessment of neurological and behavioral functioning

The NIHTB Motor Functioning Battery (Reuben et al., 2013) comprised five tests, including the 9-Hole Pegboard Dexterity Test (dominant and non-dominant hand; Wang et al., 2011), Grip Strength Dynamometer (dominant and non-dominant hand), Balance Accelerometer Measure (BAM), the 2-Minute Walk Endurance Test, and 4-Meter Walk Test (locomotion—usual pace and fast pace; Table 1). The NIHTB Sensation Battery (Dunn et al., 2013) comprised four tests that yield a total of seven scores: the Odor Identification Test (Dalton et al., 2013), Dynamic Visual Acuity (DVA) Test (Rine et al., 2013), Static Visual Acuity (Varma, McKean-Cowdin, Vitale, Slotkin, & Hays, 2013), Regional Taste Intensity Test (Coldwell et al., 2013), Pain Interference (Cook et al., 2013), and Pain Intensity (Cook et al., 2013). The NIHTB Cognition Battery (Weintraub et al., 2013) comprised seven primary tests. The battery includes two measures of “Crystallized” cognition (Picture Vocabulary Test [Gershon et al., 2013, 2014] and Oral Reading Recognition Test [Gershon et al., 2013, 2014]) and five measures of “Fluid” cognition (Picture Sequence Memory Test [Bauer, Dikmen, Heaton, Mungas, Slotkin, & Beaumont, 2013; Dikmen et al., 2014], Pattern Comparison Processing Speed Test [Carlozzi et al., 2014; Carlozzi, Tulsky, Kail, & Beaumont, 2013], List Sorting Working Memory Test [Tulsky, Carlozzi, Chevalier, Espy, Beaumont, & Mungas, 2013; Tulsky et al., 2014], Flanker Inhibitory Control and Attention Test [Zelazo, Anderson, Richler, Wallner-Allen, Beaumont, & Weintraub, 2013; Zelazo et al., 2014], and Dimensional Change Card Sort Test [Zelazo et al., 2013, 2014]). A Crystallized Composite Score comprised of the two crystallized measures mentioned earlier, as well as a Fluid Composite Score comprised of the five fluid measures mentioned earlier were also examined (Heaton et al., 2014). The NIHTB Emotion Battery (Salsman et al., 2013) comprised 17 self-report measures. This battery includes five Negative Affect measures (Anger-Affect [Pilkonis, Choi, Salsman, Butt, Moore, Lawrence et al., 2013], Anger-Hostility [Pilkonis et al., 2013], Sadness [Pilkonis et al., 2013], Fear-Affect [Pilkonis et al., 2013], and Perceived Stress [Kupst et al., 2015]) that can be combined to create a Negative Affect Composite Score, five Social Satisfaction measures (Friendship [Cyranowski et al., 2013], Loneliness [Cyranowski et al., 2013], Emotional Support [Cyranowski et al., 2013], Instrumental Support [Cyranowski et al., 2013], and Perceived Rejection [Cyranowski et al., 2013]) that can be combined to create a Social Satisfaction Composite Score, three measures of Psychological Well-Being (General Life Satisfaction [Salsman et al., 2014], Meaning and Purpose [Salsman et al., 2013, 2014], and Positive Affect [Salsman et al., 2014]) that can be combined to create a Psychological Well-Being Composite Score, and four additional emotion measures (Perceived Hostility [Cyranowski et al., 2013], Anger-Physical Aggression [Pilkonis et al., 2013], Fear-Somatic Arousal [Pilkonis et al., 2013], and Self-Efficacy [Kupst et al., 2015]).

Table 1.

Summary descriptions of the NIHTB battery

Test name	Test description	Scores
NIHTB motor battery
9-Hole Pegboard Dexterity Test	Assesses fine motor dexterity and the ability to coordinate the fingers and manipulate objects in a timely manner	Scores reflect time to completion (in seconds) for each hand; higher scores indicate better functioning
Grip Strength Dynamometry	Assesses upper extremity strength	Scores reflect the amount of force exerted in pounds of each hand; higher scores indicate better functioning
2-Minute Walk Endurance Test	Assesses endurance	Scores reflect the amount of distance traveled (in feet and inches) during the 2-min time frame; higher scores indicate better functioning
BAM	Assesses postural sway and participants’ vestibule-spinal function	Scores reflect average accelerometer readings of postural sway across all completed poses; higher scores indicate better functioning
NIHTB sensation battery
Odor Identification Test	Assesses the ability to identify various odors using scratch ‘n’ sniff cards. Participants are asked to identify which of four pictures on the screen matches the odor smelled	Scores reflect the proportion of correct responses out of nine possible stimuli; higher scores indicate better functioning
Dynamic Visual Acuity (DVA) Static Visual Acuity	Assesses gaze stability and deficits of the vestibular ocular reflex; there is both a static and dynamic component	Scores based on a logarithmic scale in LogMAR units that reflects overall functional distance vision; higher scores indicate better functioning
Regional Taste Intensity—Water	Assesses the taste perception for different everyday solutions (water and quinine)	Higher scores reflect greater perceived taste intensity averaged across all solutions; higher scores indicate better functioning
Pain intensity	Assesses self-reported pain intensity	Scores range from 1 to 10 with higher scores indicative of greater pain intensity
Pain interference	Assesses the degree to which pain interferes with other activities in life	Higher scores indicated greater pain interference
NIHTB cognition battery
Crystallized Cognition	Assesses experience/learning-based abilities that are relatively insensitive to acquired brain dysfunction	Composite scores comprised performance on Picture Vocabulary and Oral Reading Recognition; higher scores indicate better functioning
Composite
Picture Vocabulary	Assesses receptive vocabulary. Participant is instructed to select the picture that most closely identifies definition of a word	Reflects the number of correct responses using item response theory; higher scores indicate better performance
Oral Reading Recognition Test	Assesses verbal knowledge. Participants are asked to read and pronounce letters and words as accurately as possible	Scores reflect number of letters/words that were correctly pronounced (uses item response theory); higher scores indicate better performance
Fluid Cognition Composite	Assesses abilities that are purported to be more reflective of biologically based brain processes changing throughout the lifespan and are sensitive to potential acquired brain injury/disease	Composite scores comprised performance on Picture Sequence Memory, Pattern Comparison Processing Speed, List Sorting Working Memory, Dimensional Change Card Sort, and Flanker Inhibitory Control and Attention; higher scores indicate better functioning
Picture Sequence Memory	Assesses episodic memory. Participants are asked to reproduce the sequence of pictures which have been demonstrated. Sequence length varies from 6 to 18 pictures depending on the participant age	Scores reflect number of adjacent pairs that are correctly identified over two trials (uses item response theory); higher scores indicate better performance
Pattern Comparison Processing Speed	Assesses processing speed. Participants discern whether two side-by-side pictures are the same (yes/no)	Scores reflect number of correct items (maximum 130) completed in 90 s; higher scores indicate better performance
List Sorting Working Memory	Assesses working memory. Participants must sequence stimuli that are presented visually and auditorily	Scores reflect number correct for the one-and two-list versions (maximum 28); higher scores indicate better performance
Flanker Inhibitory Control and Attention	Assesses executive functioning (inhibitory control) and attention. Requires participant to focus on a given stimulus while inhibiting attention to the stimuli flanking it	Scoring algorithm based on a combination of accuracy and reaction time; higher scores indicate better performance
Dimensional Change Card Sort	Assesses executive functioning (cognitive flexibility). Participants are asked to match a series of bivalent test pictures	Scoring algorithm based on a combination of accuracy and reaction time; higher scores indicate better performance
NIHTB emotion battery
Negative Affect Composite	Assesses unpleasant feelings and emotions	Composite scores are comprised from Anger-Affect, Anger-Hostility, Sadness, Fear-Affect, and Perceived Stress; higher scores indicate worse functioning
Anger-Affect	Assesses feelings of frustration and cynicism	Higher scores indicate worse functioning (uses item response theory)
Anger-Hostility	Assesses feelings of hostility	Higher scores indicate worse functioning (uses item response theory)
Sadness	Assesses feelings of depression	Higher scores indicate worse functioning (uses item response theory)
Fear-Affect	Assesses feeling of anxiety that reflect perceptions of threat	Higher scores indicate worse functioning (uses item response theory)
Perceived Stress	Assesses perceptions about the nature of events and their relationships to the values and coping resources of an individual	Higher scores indicate worse functioning (uses item response theory)
Social Satisfaction Composite	Assesses feelings about social relationships and support	Composite scores are comprised from Friendship, Loneliness, Emotional Support, Instrumental Support, and Perceived Rejection; higher scores indicate worse functioning
Friendship	Assesses perceptions about the availability of friends or companions with whom to interact or affiliate	Higher scores indicate better functioning (uses item response theory)
Loneliness	Assess the extent to which an individual feels alone or socially isolated from other individuals	Higher scores indicate worse functioning (uses item response theory)
Emotional Support	Assesses the perception that people in one's social network are available to listen to one's problems with empathy, caring, and understanding	Higher scores indicate better functioning (uses item response theory)
Instrumental Support	Assesses to the perception that people in one's social network are available to provide information or advice needed to solve problems that arise	Higher scores indicate better functioning (uses item response theory)
Perceived Rejection	Assess perceptions related to others not listening to an individual when they ask for help or being ignored	Higher scores indicate worse functioning (uses item response theory)
Psychological Well-Being Composite	Assesses general feelings of emotional contentment	Composite scores are comprised from General Life Satisfaction, Meaning and Purpose, and Positive Affect; higher scores indicate better functioning
General Life Satisfaction	Assesses feelings of satisfaction with ones owns life, including self-satisfaction and satisfaction with different aspects of life	Higher scores indicate better functioning (uses item response theory)
Meaning and Purpose	Assesses the extent to which individuals feel that their lives matter and or make sense	Higher scores indicate better functioning (uses item response theory)
Positive Affect	Assesses feelings of happiness, joy, excitement, enthusiasm, and contentment	Higher scores indicate better functioning (uses item response theory)
Additional Emotion measures
Perceived Hostility	Assesses perceptions of being criticized or yelled at	Higher scores indicate worse functioning (uses item response theory)
Anger-Physical Aggression	Assesses aggressive behaviors	Higher scores indicate worse functioning (uses item response theory)
Fear-Somatic Arousal	Assesses feeling of anxiety that reflect autonomic arousal	Higher scores indicate worse functioning (uses item response theory)
Self-Efficacy	Assesses a person's feels of control over his/her life, as well as their confidence in being able to manage their own functioning	Higher scores indicate better functioning (uses item response theory)

Notes: NIHTB = NIH toolbox; BAM = Balance Accelerometer Measure.

Normative standards

For the NIHTB cognition and motor batteries, demographically corrected normative standards were developed with neurologically healthy individuals from the NIHTB norming project in order to determine deviations from expected levels of performances. Details regarding these norms are provided in Casaletto and colleagues (2015). We examined relationships between raw NIHTB test scores and demographic characteristics (race, ethnicity, age, education, and sex). In cases where there were substantial effects (i.e., for tests on the NIHTB cognition and motor batteries) of demographic factors on raw scores, fractional polynomial models were created from the raw scores of each test separately for each race/ethnicity (i.e., Non-Hispanic White, African American, Hispanic) and regressed on demographic characteristics (i.e., age, education, sex). The resulting T scores (M = 50, SD = 10) for each test represents an individual's cognitive or motor performance relative to age-, education-, sex-, and race/ethnicity-matched peers. For the NIHTB Sensory battery, this process was the same except that the race/ethnicity groups were combined because no major race/ethnicity effects were apparent on this battery. Finally, for the NIHTB Emotion Battery, given that the relationships between the demographic factors and test scores were negligible (i.e., accounting for less than 5% of the variance), results were not adjusted for demographics; instead, results on this battery reflect how the individual scored in relation to a sample of adults weighted to be representative of the U.S. population as determined by the 2010 U.S. Census (Babakhanyan, McKenna, Casaletto, & Heaton, 2016).

Analysis Plan

Group differences

We used multivariate analyses to examine group differences on the NIHTB measures. NIHTB Motor Battery: the first multivariate analysis examined group differences in profiles (SCI, TBI, and stroke) for the six NIHTB Motor Scores. Post hoc analyses that employed a Bonferroni correction were utilized for follow-up analyses. Due to a large amount of missing data for these subtests for individuals with SCI (due to paralysis or mobility limitations) scores were imputed using Winsor's method. Specifically, in cases where individuals with SCI were unable to complete a test due to their severe motor limitations (e.g., being unable to walk or manipulate pegs), they received a score that was 1 point lower than the lowest recorded SCI score for an individual that was able to complete this test. In addition, 21.7% of TBI and 19.4% of CVA cases were missing 4 or more measures from the NIHTB motor battery and 16.8% of TBI and 16.1% of CVA were missing at least 6 or more of the motor measures. Due to inconsistent record keeping to indicate whether the participants with TBI or stroke were missing data due to paralysis or mobility limitations, or due to other reasons (e.g., refusal, time restrictions), we elected not to Winsorize scores for these participants. NIHTB Sensation Battery: we conducted a multivariate analysis to examine group differences on the seven NIHTB Sensation scores. Post hoc analyses that employed a Bonferroni correction were utilized for follow-up analyses. NIHTB Cognition Battery: Two separate multivariate analyses examined group differences on the seven individual NIHTB Cognition Scores and two NIHTB Cognition Composite scores, respectively. Again, post hoc analysis for significant findings employed a Bonferroni correction. NIHTB Emotion Battery: the final set of multivariate analyses examined clinical group differences on the 3 NIHTB Emotion Composite Scores and the 17 individuals NIHTB Emotion Scores; post hoc analyses employed a Bonferroni correction.

Impairment rates

For all NIHTB scores, we calculated the percentage of individuals showing impairment relative to normative rates (i.e., scores of impairment reflect individuals with scores >1 SD beyond the normative mean in the abnormal direction); we expect 16% of the people to score 1 SD below the mean; impairment rates that exceeded 16% are indicative of elevated rates of impairment (Heaton, Miller, Taylor, & Grant, 2004).

Relative risk

Relative risk for all of the different NIHTB measures was derived using an odds ratio analysis (Ivnik, Smith, Cerhan, Boeve, Tangalos, & Petersen, 2001). These analyses examined a subsample of each clinical group relative to a matched normative control group; the subsample includes participants with good matches on age, education, sex, race, and ethnicity. These analyses do not include Pain Interference or Pain Intensity from the NIHTB Sensation battery, as these measures were not administered as a part of the normative study. In cases with significant findings, we focused the discussion on those findings that indicated that the clinical sample was ≥2 times more likely than the general population to be at risk.

Results

Our sample included 604 participants: n = 184 TBI (36% complicated mild, 9% moderate, 54% severe; 1% unknown severity), n = 211 stroke (28.4% mild, 27% moderate, 44% severe, 1% unknown; 67% ischemic and 27% hemorrhagic; 38% left-sided stroke, 39% right-sided stroke; 4% bilateral; 18% unknown), and n = 209 SCI (27% paraplegia complete, 22% paraplegia incomplete, 22% tetraplegia complete, 29% tetraplegia incomplete, and 1% unknown severity). Participants in the confirmed sample ranged in age from 18 to 84 years (M = 47.34; SD = 16.26) and were predominantly men (64% men). See Table 2 for detailed demographic information by neurologic group; group differences occurred for time since injury (SCI longest, stroke shortest), age (TBI youngest, stroke oldest), gender (SCI and TBI more men), and race (stroke more African Americans), but the groups were comparable for ethnicity and education. The average NIHTB composite and subtest T scores for individuals with SCI, TBI, and Stroke are presented in Table 3, along with the percentage of individuals showing impairment (i.e., individuals with scores ≥ 1 SD beyond the normative mean in the abnormal direction). Table 4 includes detailed information about the subsample of participants that we utilized to evaluate relative risk; this table includes demographic data for each neurological group and its matched control group; there were no group differences for age, education, or sex. Descriptive data for the normative control groups’ performance on the NIHTB tests are provided in Table 5.

Table 2.

Group demographic characteristics by disability group

Variable	SCI	TBI	Stroke	Combined sample
	(N = 209)	(N = 184)	(N = 211)	(N = 604)
Age (years)		F (2, 601) = 65.95, p < .0001a
M (SD)	45.47 (14.06)	39.38 (17.24)	56.13 (12.97)	47.37 (16.26)
Time since injury (years)		F (2, 600) = 102.845, p < .0001b
M (SD)	12.12 (10.01)	5.95 (5.54)	2.74 (2.46)	6.97 (7.86)
Gender (%)		χ² (2, N = 604) = 37.95, p < .0001c
Men	78.5	64.1	49.8	64.1
Women	21.5	35.9	50.2	35.9
Race (%)		χ² (4, N = 604) = 52.60, p < .0001d
Caucasian	61.2	73.4	43.1	58.6
African American	28.7	15.8	49.3	32
Asian	0.5	4.9	1.9	2.3
American Indian or Alaskan Native	0	0.5	0.5	0.3
Native Hawaiian or Other Pacific Islander	0	0.5	0	0.2
More than one race	0.5	1.1	1.9	1.2
Other	9.1	3.8	3.3	5.5
Ethnicity (%)		χ² (4, N = 604) = 2.25, p = .69
Not Hispanic or Latino	90.9	92.4	93.8	92.4
Hispanic or Latino	8.6	7.1	5.2	7
Not provided	0.5	0.5	0.9	0.7
Education		χ² (4, N = 604) = 1.02, p = .91
M (SD)	13.84 (2.51)	13.74 (2.47)	13.68 (2.57)	13.75 (2.52)
		F (2, 599) = 0.22, p = .80

Notes: SCI = spinal cord injury; TBI = traumatic brain injury.

^aWith regard to age, participants with stroke were significantly older than participants with SCI and TBI, and participants with SCI were significantly older than participants with TBI.

^bWith regard to time since injury, SCI was significantly further from injury relative to both TBI and stroke, and participants with TBI were significantly further from injury that individuals with stroke.

^cThere were significantly more men than women for both SCI and TBI.

^dWith regard to race, the stroke group had significantly more African Americans than the SCI and TBI groups.

Table 3.

NIHTB performance for SCI, TBI, and stroke participants

Variable	Multivariate		SCI			TBI			Stroke
	F	Partial η2	N	Mean (SD)	% Impaired	N	Mean (SD)	% Impaired	N	Mean (SD)	% Impaired
NIHTB motor scores
9-Hole Peg DHa,b	20.25	.09	191(59)c	27.97 (20.14)	63.3	111	42.66 (13.75)	36.5	100	41.25 (15.42)	42.4
9-Hole Peg N-DHa,b	22.64	.10	191(67)c	25.09 (22.60)	64.7	111	41.63 (14.59)	36.0	100	43.16 (14.30)	37.1
Grip strength DHa,b	29.83	.13	191(82)c	27.25 (26.36)	52.2	111	49.05 (11.33)	15.7	100	43.98 (12.79)	27.6
Grip strength N-DHa,b	22.40	.10	191(88)c	25.28 (26.12)	55.1	111	45.45 (15.34)	21.9	100	42.1 (14.45)	35.2
Balance accelerometera,b,d	171.94	.46	191(157)c	16.90 (9.68)	87.0	111	43.77 (11.20)	25.8	100	39.29 (13.51)	37.1
2-Minute Walk (endurance)a,b,d	282.59	.59	191(150)c	10.86 (9.35)	90.8	111	43.75 (12.18)	32.0	100	37.41 (11.23)	59.5
4-Meter Walk—Usual Pacea,b,d	280.88	.59	191(149)c	18.71 (8.56)	90.3	111	49.10 (11.37)	22.5	100	41.54 (10.19)	47.6
4-Meter Walk—Fast Pacea,b,d	306.87	.61	191(149)c	15.61 (10.32)	89.4	111	49.60 (10.24)	18.0	100	43.08 (11.64)	49.0
NIHTB sensation scores
Odor Identification Testa,d	13.60	.07	124	50.99 (10.75)	14.5	126	43.38 (14.29)	33.2	116	48.14 (10.66)	19.9
Static Visual Acuity	1.89	.01	124	51.02 (14.09)	19.3	126	51.13 (13.12)	20.1	116	49.06 (11.68)	22.3
DVAa,b	5.40	.03	124	52.4 (11.06)	9.7	126	48.88 (11.51)	19.6	116	47.80 (10.49)	18.0
Toolbox Taste Test—Quinine	2.68	.02	124	49.78 (9.23)	11.1	126	48.62 (9.73)	15.8	116	51.18 (10.58)	11.8
Toolbox Taste Test—Salta,c	14.56	.07	124	55.94 (8.42)	5.8	126	51.57 (9.91)	9.2	116	57.78 (9.29)	4.7
Pain intensitya,b	7.44	.04	124	4.20 (2.70)	—	126	2.59 (2.64)	—	116	2.80 (2.67)	—
Pain interferencea	5.40	.03	124	54.41 (8.91)	25.1	126	50.15 (8.93)	14.6	116	51.08 (8.98)	16.7
NIHTB cognition scores
Crystallized Cognition Composite	1.21	.005	156	50.78 (10.30)	16.0	163	49.79 (9.54)	16.6	176	49.54 (10.85)	19.2
Picture vocabulary	1.40	.005	203	50.57 (10.25)	16.3	177	49.41 (9.60)	15.8	201	49.08 (12.06)	22.3
Oral Reading Recognition	1.56	.005	203	49.67 (10.21)	15.8	177	49.86 (10.82)	16.9	201	48.34 (10.58)	23.3
Fluid Cognition Compositea,b	20.29	.08	156	47.90 (9.30)	20.5	163	41.75 (13.00)	41.1	176	40.51 (11.59)	49.2
Picture memorya,b	10.21	.04	158	47.24 (9.69)	22.2	161	42.35 (10.88)	43.5	175	42.64 (12.38)	43.8
Pattern Comparisona,b	13.19	.05	158	48.69 (8.38)	13.9	161	45.44 (10.86)	28.0	175	43.13 (10.74)	39.2
List Sortinga,b	12.23	.05	158	49.41 (9.31)	13.9	161	44.75 (10.55)	32.3	175	43.98 (9.96)	40.3
Flankera,b	8.38	.03	158	46.29 (9.20)	24.7	161	43.21 (11.10)	38.5	175	42.08 (10.58)	40.3
DCCSb	8.17	.03	158	46.56 (9.09)	25.9	161	44.42 (10.49)	33.5	175	42.46 (9.47)	40.3
NIHTB emotion scores
Negative Affect Composite	0.64	.002	195	51.93 (10.27)	21.0	175	52.77 (11.90)	27.4	202	51.62 (11.20)	24.1
Anger-Affect	1.30	.005	196	50.04 (1.47)	16.8	175	51.46 (11.71)	22.3	203	49.92 (11.04)	18.1
Anger-Hostility	0.47	.002	196	51.49 (10.01)	20.4	175	51.66 (10.29)	22.9	203	51.04 (11.05)	22.1
Sadness	0.19	.001	196	53.04 (10.67)	23.5	175	53.41 (11.73)	26.9	203	52.75 (10.88)	24.5
Fear-Affect	0.29	.001	196	51.82 (9.87)	18.4	175	52.80 (12.13)	25.7	203	52.14 (11.42)	23.5
Perceived Stress	0.44	.002	196	51.24 (9.99)	18.4	175	51.92 (11.36)	22.9	203	50.85 (11.33)	19.1
Social Satisfaction Composite	0.01	.000	195	45.97 (11.45)	30.8	175	46.08 (10.43)	25.1	202	46.29 (12.61)	32.0
Friendship	0.03	.000	195	46.84 (11.16)	26.2	175	46.55 (10.16)	25.7	202	46.75 (11.15)	32.0
Loneliness	0.34	.001	195	54.65 (11.60)	34.9	175	54.49 (11.35)	32.6	202	53.52 (12.63)	28.1
Emotional Support	0.69	.002	195	44.87 (11.42)	34.4	175	46.22 (11.54)	26.9	202	45.15 (11.82)	33.0
Instrumental Support	1.67	.006	195	49.91 (9.20)	14.9	175	48.31 (9.08)	21.7	202	49.24 (10.71)	23.2
Perceived Rejection	0.06	.000	195	51.77 (10.97)	19.5	175	51.28 (10.71)	21.1	2032	51.40 (11.98)	25.6
Psychological Well-Being Composite	1.49	.005	195	44.42 (10.80)	32.3	175	46.22 (11.81)	28.0	202	45.39 (11.48)	29.6
General Life Satisfactiona,d	6.56	.02	196	42.45 (11.24)	46.4	175	46.92 (11.73)	26.9	202	43.98 (11.67)	37.9
Meaning and Purpose	0.42	.001	196	45.92 (9.86)	25.5	175	46.47 (11.39)	26.3	202	45.86 (10.54)	27.6
Positive Affect	0.58	.002	196	47.14 (11.24)	26.5	175	47.06 (11.47)	25.1	202	47.97 (12.60)	24.6
Additional
Perceived Hostility	0.43	.002	195	49.82 (10.41)	2.0	175	50.68 (9.81)	19.4	203	49.91 (11.09)	17.2
Anger-Physical Aggression	3.95	.01	195	52.62 (10.39)	26.7	175	54.68 (12.04)	33.7	203	51.72 (11.16)	27.1
Fear-Somatic Arousal	5.36	.02	195	56.04 (10.95)	34.9	175	54.14 (12.00)	32.0	203	53.60 (11.47)	27.1
Self-Efficacy	1.79	.006	195	48.35 (11.17)	25.1	175	46.45 (10.80)	27.4	203	47.27 (11.66)	30.0

Notes: All scores are reported as T scores (M = 50, SD = 10): T scores for the motor and cognitive domains are demographically adjusted for age, sex, education, and race/ethnicity, T scores for the sensory domain are demographically adjusted for age, sex, and education, and T scores for the emotion domain are compared to the general U.S. Census; % impairment reflects individuals with scores >1 SD beyond the mean on a given test in the negative direction. NIHTB = NIH Toolbox; DH = Dominant Hand; N-DH = Non-Dominant Hand; DCCS = Dimensional Change Card Sort; SCI = spinal cord injury; TBI = traumatic brain injury.

^aSignificant univariate differences between SCI and TBI after Bonferroni correction.

^bSignificant univariate differences between SCI and Stroke after Bonferroni correction.

^cNumber of participants with windsorized scores.

^dSignificant univariate differences between TBI and Stroke after Bonferroni correction.

Table 4.

Demographic information for neurological versus matched control groups

Variable	SCI matched controls	TBI matched controls	Stroke matched controls
	(n = 209)	(n = 184)	(n = 211)
Age—M (SD)	45.99 (15.07)	40.20 (16.68)	55.34 (13.00)
Women (%)	23.0	35.3	50.2
Education—M (SD)	14.03 (1.64)	14.08 (2.568)	14.09 (2.434)
Race (%)
Caucasian	67.8	73.4	77.7
African American	23.1	15.2	22.3
Other	9.1	11.4	0.0
Ethnicity (%)
Not Hispanic or Latino	87.6	87.5
Hispanic or Latino	11.5	12.0	1210.5
Not provided	1.0	0.5	0.5

Notes: SCI = spinal cord injury; TBI = traumatic brain injury.

Table 5.

Normative control group performance on NIHTB tests

Variable	SCI matched controls	TBI matched controls	Stroke matched controls
	(n = 209)	(n = 184)	(n = 211)
	M (SD)	M (SD)	M (SD)
NIHTB motor scores
9-Hole Peg DH	50.05 (9.85)	50.66 (10.09)	50.68 (9.93)
9-Hole Peg N-DH	50.46 (9.87)	51.1 (10.38)	51.17 (9.75)
Grip Strength DH	50.61 (9.04)	49.93 (10.29)	51.69 (8.92)
Grip Strength N-D	50.86 (9.49)	50.52 (9.47)	51.65 (9.13)
Balance Accelerometer	51.18 (9.40)	49.57 (10.39)	50.35 (9.59)
2-Minute Walk (endurance)	50.72 (10.41)	50.15 (9.18)	51.45 (10.03)
4-Meter Walk—Usual Pace	49.76 (9.58)	50.12 (9.79)	50.48 (9.93)
4-Meter Walk—Fast Pace	50.26 (9.55)	49.8 (10.12)	51.49 (9.69)
NIHTB sensation scores
Odor Identification Test	50.60 (10.62)	50.36 (10.35)	49.15 (10.41)
Static Visual Acuity	49.73 (9.58)	48.57 (10.09)	50.03 (9.76)
DVA	50.45 (10.16)	50.19 (10.24)	50.61 (10.53)
Toolbox Taste Test—Quinine	50.92 (10.05)	50.32 (9.47)	49.88 (9.54)
Toolbox Taste Test—Salt	50.00 (9.74)	50.09 (9.57)	49.79 (10.33)
NIHTB cognition scores
Crystallized cognition composite	50.95 (9.96)	50.43 (10.09)	50.65 (9.37)
Picture vocabulary	50.55 (9.43)	50.27 (9.97)	50.42 (8.93)
Oral reading recognition	51.19 (10.29)	50.35 (10.21)	50.50 (9.79)
Fluid cognition composite	50.90 (10.11)	49.82 (10.41)	50.77 (10.08)
Picture memory	49.92 (9.44)	49.15 (9.19)	50.14 (8.81)
Pattern comparison	50.97 (10.20)	50.14 (9.92)	50.92 (9.99)
List sorting	50.52 (10.34)	50.32 (10.41)	50.62 (9.80)
Flanker	50.89 (10.14)	50.08 (10.13)	50.40 (9.57)
DCCS	51.07 (10.01)	49.88 (9.96)	50.89 (10.57)
NIHTB emotion scores
Negative affect composite	49.78 (10.73)	49.65 (10.49)	48.23 (9.59)
Anger-affect	49.54 (11.04)	49.27 (10.96)	48.67 (9.99)
Anger-hostility	50.61 (10.25)	51.34 (9.94)	48.83 (9.94)
Sadness	49.94 (11.01)	49.55 (10.42)	49.54 (9.73)
Fear-affect	49.35 (10.19)	49.49 (10.22)	48.14 (9.97)
Perceived stress	49.52 (10.03)	49.15 (10.56)	47.70 (8.97)
Social satisfaction composite	49.15 (10.50)	50.43 (10.56)	50.31 (10.90)
Friendship	49.56 (10.47)	50.46 (10.69)	50.23 (10.17)
Loneliness	50.55 (11.05)	50.16 (10.59)	49.52 (10.69)
Emotional support	48.56 (9.47)	49.81 (9.84)	49.48 (10.51)
Instrumental support	50.01 (10.51)	50.73 (9.98)	50.74 (10.98)
Perceived rejection	50.71 (10.44)	49.3 (10.42)	49.89 (9.64)
Psychological well-being composite	49.44 (9.96)	50.95 (10.62)	50.07 (10.15)
General life satisfaction	49.15 (9.79)	50.37 (10.75)	49.78 (10.24)
Meaning and purpose	49.47 (10.43)	50.93 (10.76)	49.78 (10.27)
Positive affect	49.9 (10.03)	50.95 (10.27)	50.41 (10.21)
Additional
Perceived hostility	49.88 (9.93)	49.33 (10.28)	48.85 (10.39)
Anger-physical aggression	51.46 (9.98)	52.33 (10.82)	47.05 (7.81)
Fear-somatic arousal	49.36 (9.55)	50.16 (10.00)	49.53 (10.19)
Self-efficacy	51.33 (9.80)	51.21 (10.40)	51.91 (9.91)

Notes: All scores are reported as T scores (M = 50, SD = 10): T scores for the motor and cognitive domains are demographically adjusted for age, sex, education, and race/ethnicity, T scores for the sensory domain are demographically adjusted for age, sex, and education, and T scores for the emotion domain are compared to the general U.S. Census. NIHTB = NIH Toolbox; DH = Dominant Hand; N-DH = Non-Dominant Hand; DCCS = Dimensional Change Card Sort.

NIHTB Motor Battery

Group differences

Findings from the multivariate analysis examining group differences for the six NIHTB Motor Scores is provided in Fig. 1 and Table 3; Pillai's Trace = .68, F(16, 784) = 25.42, p < .0001, partial eta squared = .34. The number of SCI participants with Winsorized scores for each motor test is indicated in Table 3 and ranged from n = 59 (Pegboard dominant hand) to n = 157 (Balance Accelerometer). There were significant group differences for all motor tests. Bonferroni post hoc analyses indicated that, consistent with hypotheses, individuals with SCI scored lower than individuals with TBI and individuals with stroke on all motor function measures (partial eta squared > .09). In addition, individuals with stroke performed worse than those with TBI for the BAM and all three walk tests.

Fig. 1.

The NIHTB motor battery: average T scores for individuals with SCI, TBI, and stroke.

Notes: NIHTB = NIH Toolbox; SCI = spinal cord injury; TBI = traumatic brain injury. Scores earlier are fully corrected for age, sex, race, and education. Scores are standardized T scores with a mean of 50 and SD of 10. Stars indicate significant group differences for a given test.

Impairment rates

Consistent with hypotheses, impairment rates (percentage of participants >1 SD worse than the normative mean) were elevated for all three clinical groups on all measures, with the exception of grip strength dominant hand and locomotion (fast pace) for the TBI group relative to normative standards.

Relative risk

Relative risk indicated that the SCI and stroke groups were at increased risk for motor functioning impairments relative to their matched controls for all measures (Table 6). For the TBI group, elevations were evident for six of the eight motor tests, with only four of these being greater than two times more likely than controls to exhibit risk (Table 6). Relative risk for impairment was generally 4–8 times higher for SCI, 3–5 times higher for stroke, and 2–3 times higher for TBI (on measures with significant impairment) relative to matched controls.

Table 6.

Relative risk for clinical impairment compared to matched controls

	SCIRR (95% CI)	TBIRR (95% CI)	StrokeRR (95% CI)
NIHTB motor scores
9-Hole Peg Dominant Hand	4.06 (2.87, 4.74)*	3.00 (1.95, 4.64)*	3.36 (2.33, 4.85)*
9-Hole Peg Non-Dominant Hand	4.82 (3.31, 7.01)*	3.11 (2.04, 4.74)*	4.31 (2.88, 6.45)*
Grip Strength Dominant Hand	5.18 (3.33, 8.06)*	1.18 (0.73, 1.90)	2.86 (1.84, 4.42)*
Grip Strength Non-Dominant Hand	4.94 (3.25, 7.51)*	2.13 (1.30, 3.49)*	4.02 (2.59, 6.23)*
BAM	8.00 (5.04, 12.68)*	1.97 (1.29, 2.99)*	4.08 (2.73, 6.09)*
2-Minute Walk (endurance)	5.42 (3.96, 7.43)*	2.62 (1.73, 3.97)*	5.51 (3.84, 7.90)*
4-Meter Walk—Usual Pace (locomotion)	6.68 (4.67, 9.53)*	1.93 (1.21, 3.08)*	4.49 (3.08, 6.56)*
4-Meter Walk—Fast Pace (locomotion)	6.87 (4.77, 9.89)*	1.46 (0.91, 2.35)	5.06 (3.42, 7.49)*
NIHTB sensation scores
Odor Identification Test	0.88 (0.57, 1.36)	2.03 (1.41, 2.91)*	1.06 (0.73, 1.55)
Static Visual Acuity	1.82 (1.17, 2.84)*	1.49 (0.95, 2.35)	1.64 (1.10, 2.44)
DVA	0.90 (0.53, 1.52)	1.51 (0.97, 2.36)	1.78 (1.15, 2.74)*
Toolbox Taste Test—Quinine	1.01 (0.60, 1.71)	1.35 (0.82, 2.21)	0.85 (0.53, 1.37)
Toolbox Taste Test—Salt	0.43 (0.24, 0.79)*	0.65 (0.37, 1.14)	0.27 (0.14, 0.53)*
NIHTB cognition scores
Crystallized Intelligence Composite	1.37 (0.84, 2.21)	1.21 (0.75, 1.93)	1.98 (1.26, 3.13)*
Picture Vocabulary	1.55 (0.94, 2.57)	1.16 (0.69, 1.92)	1.83 (1.15, 2.91)*
Oral Reading Recognition	1.12 (0.70, 1.78)	1.06 (0.67, 1.70)	1.47 (0.99, 2.20)
Fluid Intelligence Composite	1.50 (0.94, 2.40)	2.89 (1.91, 4.38)*	3.69 (2.50. 5.43)*
Picture Sequence Memory Test	1.88 (1.21, 2.92)*	2.80 (1.89, 4.14)*	4.12 (2.67, 6.35)*
Pattern Comparison	1.00 (0.62, 1.63)	2.09 (1.38, 3.16)*	2.84 (1.97, 4.09)*
List Sorting	0.90 (0.59, 1.36)	2.03 (1.38, 3.00)*	2.34 (1.66, 3.30)*
Flanker	1.78 (1.16, 2.74)*	3.13 (2.07, 4.73)	2.97 (2.07, 4.27)*
DCCS	2.44 (1.54, 3.87)*	2.50 (1.66, 3.76)*	3.21 (2.19, 4.70)*
NIHTB emotion scores
Negative Affect Composite	1.29 (0.85, 1.96)	1.82 (1.19, 2.78)*	2.68 (1.64, 4.39)*
Anger-Affect	1.23 (0.78, 1.96)	1.61 (1.02, 2.55)	1.66 (1.02, 2.69)
Anger-Hostility	0.97 (0.66, 1.43)	0.92 (0.63, 1.33)	1.36 (0.91, 2.04)
Sadness	1.42 (0.95, 2.11)	2.11 (1.34, 3.33)*	2.15 (1.37, 3.53)*
Fear-Affect	1.79 (1.09, 2.96)	2.33 (1.43, 3.78)*	2.35 (1.46, 3.78)*
Perceived Stress	1.40 (0.89, 2.22)	2.07 (1.26, 3.39)*	3.31 (1.79, 6.15)*
Social satisfaction composite	1.59 (1.12, 2.25)*	1.39 (0.93, 2.07)	1.85 (1.29, 2.65)*
Friendship	1.50 (1.02, 2.19)	1.57 (1.04, 2.37)	2.16 (1.47, 3.16)*
Loneliness	1.71 (1.23, 2.38)*	1.86 (1.27, 2.72)*	1.47 (1.03, 2.09)
Emotional Support	2.02 (1.41, 2.90)*	1.70 (1.12, 2.56)	1.82 (1.28, 2.57)*
Instrumental Support	0.73 (0.47, 1.12)	1.28 (0.84, 1.96)	1.34 (0.91, 1.98)
Perceived Rejection	1.12 (0.74, 1.68)	1.54 (0.97, 2.45)	1.78 (1.18, 2.66)*
Psychological well-being composite	1.89 (1.28, 2.81)*	1.91 (1.21, 3.00)*	1.94 (1.29. 2.91)*
General Life Satisfaction	2.51 (1.81, 3.46)*	1.52 (1.02, 2.26)	2.00 (1.44, 2.80)*
Meaning and Purpose	1.62 (1.06, 2.48)	1.71 (1.09, 2.69)*	2.11 (1.36, 3.29)*
Positive Affect	1.66 (1.12, 2.45)*	1.70 (1.10, 2.61)*	1.55 (1.04, 2.29)
Additional emotion scores
Perceived Hostility	1.65 (1.04, 2.62)	1.61 (0.98, 2.64)	1.24 (0.79, 1.94)
Anger-Physical Aggression	1.16 (0.82, 1.63)	1.33 (0.96, 1.84)	2.83 (1.76, 4.55)*
Fear-Somatic Arousal	2.44 (1.66, 3.60)*	1.99 (1.34, 2.96)*	1.57 (1.08, 2.29)
Self-Efficacy	2.06 (1.33, 31.20)*	2.26 (1.43, 3.57)*	2.87 (1.84, 4.49)*

Notes: All scores are reported as T scores (M = 50, SD = 10): T scores for the motor and cognitive domains are demographically adjusted for age, sex, education, and race/ethnicity, T scores for the sensory domain are demographically adjusted for age, sex, and education, and T scores for the emotion domain are compared to the general U.S. Census; NIHTB = NIH Toolbox; DCCS = Dimensional Change Card Sort; BAM = Balance Accelerometer Measure; the control sample did not include data on NIHTB Pain Interference, thus relative risk was not able to be calculated for this measure.

^*Significant p value after Bonferroni correction employed.

NIHTB Sensation Battery

Group differences

The multivariate analysis examining group differences on the seven NIHTB Sensation scores is reported in Table 3; Pillai's Trace = .22, F(14, 714) = 6.400, p < .0001, partial eta squared = .11. There were significant group differences for all tests except the Quinine Taste Test and Static Visual Acuity. For DVA, individuals with SCI had higher scores than both TBI and stroke individuals. For odor identification, as hypothesized, individuals with TBI had significantly lower scores than stroke and SCI. Individuals with SCI indicated more pain intensity than those with TBI or stroke and more pain interference than those with TBI.

Impairment rates

Consistent with hypotheses, impairment rates were elevated for static and DVA and for olfaction for TBI and stroke. Impairment rates were also elevated for pain interference and static visual acuity for SCI.

Relative risk

The clinical groups were generally not at increased risk for sensory dysfunction; the major exception was for olfaction in TBI (this was the only significant finding that exceeded a priori standards; Table 6).

NIHTB Cognition Battery

Group differences

The multivariate analyses examining group differences on the seven individual NIHTB Cognition Scores and group differences on the two NIHTB Cognition Composite scores can be found in Table 3; Pillai's Trace for the two composite scores = .08, F(4, 982) = 9.83, p < .0001, partial eta squared = .04, Pillai's Trace for the two crystallized subtests = .009, F(4, 1154) = 1.326, p = 1.33, partial eta squared = .005, and Pillai's Trace for the five fluid subtests = .09, F(10, 974) = 4.63, p < .0001, partial eta squared = .05. Univariate analyses indicated significant group differences on all scores except for Picture Vocabulary, Oral Reading Recognition, and Crystallized Cognition Composite scores. Bonferroni post hoc analyses indicated that, as hypothesized, individuals with stroke or TBI demonstrated lower (worse) cognitive scores than individuals with SCI. Specifically, the stroke and TBI groups had lower scores than the SCI group on Picture Sequence Memory, Pattern Comparison, List Sorting, and Flanker (as well as the Fluid Intelligence Composite). The stroke group also had lower scores than the SCI group on the DCCS. See Fig. 2 for group comparisons on each of the NIHTB cognitive tests.

Fig. 2.

The NIHTB cognition battery: average T scores for individuals with SCI, TBI, and stroke.

Notes: NIHTB = NIH Toolbox; SCI = spinal cord injury; TBI = traumatic brain injury; DCCS = Dimensional Change Card Sort. Scores earlier are fully corrected for age, sex, race, and education. Scores are standardized T scores with a mean of 50 and SD of 10. Stars indicate significant group differences for a given test.

Impairment rates

As hypothesized, impairment rates also were elevated compared to normative standards for individuals in the stroke and TBI groups on all measures of Fluid Cognition. Hypotheses for SCI and stroke were partially supported: impairment rates (relative to normative standards) were slightly elevated for stroke on even the Crystallized measures; and impairment rates were slightly elevated for measures of executive functioning and memory for SCI (as opposed to hypothesized elevations on all fluid measures).

Relative risk

Relative risk analyses (Table 6) indicated that individuals with stroke were generally 2–4 times more likely than controls to exhibit impairment on all measures of fluid cognition, and individuals with TBI were generally 2–3 times more likely than controls to be at increased risk for clinical impairment on those measures. Individuals with SCI were only at increased risk for one of the measures of executive functioning (DCCS).

NIHTB Emotion Battery

Group differences

Multivariate analyses examining neurologic group differences on the 3 NIHTB Emotion Composite Scores and the 17 individual NIHTB Emotion Scores are reported in Table 3; Pillai's Trace for the three composite scores = .02, F(6, 1134) = 1.570, p = .15, partial eta squared = .008, Pillai's Trace for the five negative affect subtests = .007, F(10, 1134) = .44, p = .93, partial eta squared = .004, Pillai's Trace for the five social satisfaction subtests = .02, F(10, 1130) = 1.29, p = .23 partial eta squared = .01, and Pillai's Trace for the four additional emotion subtests = .05, F(8, 1132) = 3.62, p < .0001, partial eta squared = .03. There were no significant neurologic group differences for the three composite scores, or the additional four measures of emotion (this single exception was that individuals with TBI reported more General Life Satisfaction than ether SCI or stroke). However, note that while the three neurologic groups generally did not demonstrate significant emotional differences from each other, all three were substantially worse than normative expectations and their matched control groups on most Emotion Battery scores (Tables 3 and 6).

Impairment rates

As just suggested, all three neurologic groups evidenced increased rates of emotional disturbance across most of the measures in the Emotion Battery (exceptions were that rates were not elevated for: the Negative Affect Composite in SCI, and for Anger-Affect, Anger-Hostility, Instrumental Support, and Perceived Hostility in all groups; Table 3).

Relative risk

Relative risk analyses (Table 6) indicated that individuals with SCI were at risk for increased fear-somatic arousal than controls, individuals with stroke were at increased risk for negative affect and anger-physical aggression, and individuals with stroke and TBI were at increased risk for sadness, fear-affect, and perceived stress relative to controls. Interestingly, all three clinical groups evidenced problems across a number of the psychological well-being measures, with all three clinical groups being at high risk for poor self-efficacy relative to normative controls.

Discussion

This study was designed to establish the validity of the NIHTB in individuals with TBI, stroke, and SCI. Although neuropsychological evaluations typically evaluate cognition (and to a lesser degree emotional and motor functioning), the NIHTB provides a co-normed battery of measures across a variety of domains of functioning. Though the battery was designed for research purposes and was developed for use in the general populations, the data presented here provide validation evidence to suggest it may be appropriate for clinical purposes. Specifically, in addition to cognitive function, the NIHTB can help clinicians more broadly evaluate emotion, motor, and sensory function. The results provide support that individuals with disabilities are more likely to score lower on several NIHTB tests. As such, the current paper presents data on the clinical utility of the NIHTB sensory, motor, emotional, and cognitive tests in rehabilitation populations. Such data are important for the neuropsychologist who is attempting to interpret NIHTB scores in clinical practice. Although future work is needed to establish the clinical utility of these measures across repeat assessments, these data provide an important first step in establishing the clinical utility of the NIHTB in individuals with neurologic conditions.

We observed significant group differences on all NIHTB Motor tests. As expected, individuals with stroke and SCI performed more poorly than those with TBI; group-level averages were generally at least 1 SD below the normative mean for all motor tests (and 2 or more SD below the mean for SCI). Furthermore, both impairment rates and associated relative risks were considerably higher than the general population for all three clinical groups on most measures of motor functioning. These data support the validity of the NIHTB motor battery as sensitive to motor dysfunction across the three neurological conditions represented in this study.

The NIHTB provides the neuropsychologist a co-normed set of measures across multiple sensory modalities that can be incorporated into their clinical assessment batteries. Although group average scores were generally within normal limits on the Sensation Battery, when impairment rates were examined, they were consistent with our prior hypotheses: impairment rates were elevated for sense of smell for TBI (relative to what we would expect based on a normal distribution; Heaton et al., 2004). Individuals with SCI also evidenced more pain than either those with TBI or stroke. This is not surprising and is consistent with findings for given frequent physical complications in SCI (Dudley-Javoroski & Shields, 2006; Haisma et al., 2007; Li et al., 2012). Individuals with TBI had more problems with smell than either individuals with SCI or stroke which may reflect the fact that in TBI, the subregion where the olfactory bulbs are located is particularly vulnerable to insult following head injury (Steuer, Schaefer, & Belluscio, 2014). The addition of sensory measures into a clinical battery may assist with differential diagnosis, and in identifying sensory deficits that might not otherwise be identified or targeted for treatment interventions.

For the Cognition Battery, although the crystallized cognitive measures demonstrated comparable and close to normal performance across neurologic groups (Oral Reading and Picture Vocabulary), there were significant group differences for all of the fluid cognitive tests within the NIHTB-CB. Individuals with acquired brain injury are generally 2–4 times more likely than individuals without brain injury to receive an impaired score on tests of fluid cognition. Although individuals with SCI performed within normal limits as a group, the TBI and stroke groups demonstrated significant impairments on all fluid cognition measures. Additional detail examining the cognitive data on the NIHTB for each of these neurological samples, including analyses examining interrelationships between injury severity and performance on the NIHTB Cognition Battery measures, is the focus of other reports (Carlozzi et al., 2017; Cohen et al., in press; Tulsky et al., 2017).

Although these cognitive profiles are consistent with the neuropsychological profiles commonly observed in these clinical samples, overall, our sample appears to be higher functioning than others reported on in the literature. For example, in prior research, the cognitive deficits that are characteristic of stroke and TBI often result in average scores that are at least 1 SD below the mean on standardized tests (Carlozzi, Grech, & Tulsky, 2013). However, our selection criteria that required non-aphasic community-dwelling participants who were at least 12 months post-injury in all three groups almost certainly excluded most or all individuals with the worst outcomes. Although these differences between our sample and neurological samples in the published literature provide a likely explanation for these small effects, it is possible that they suggest a lack of sensitivity for the NIHTB measures in these populations. This may suggest that the NIHTB is appropriate for screening, but may not be useful as a diagnostic tool; this is not surprising given that the NIHTB was not designed as a clinical diagnostic tool (it was designed for research settings), nor was it designed to take the place of longer, more comprehensive clinical assessments. Regardless, with regard to group differences, individuals with stroke and TBI generally performed more poorly than individuals with SCI. However, even for SCI, normative impairment rates were higher than expected in the general population for measures of episodic memory and executive functioning, suggesting the possibility of undetected complicated mild head injury associated with the original spinal injury. This observation is consistent with other studies that suggest that comorbid head injury is commonly overlooked in SCI (Hagen, Eide, Rekand, Gilhus, & Gronning, 2010; Macciocchi, Seel, Thompson, Byams, & Bowman, 2008; Roth et al., 1989; Sharma, Bradbury, Mikulis, & Green, 2014).

Unexpectedly, we generally did not observe group profile differences on the NIHTB Emotion tests, even though impairment rates were elevated across all three groups for most emotion measures. Interestingly, the SCI and stroke groups reported significantly lower levels of psychological well-being than controls, apparently reflecting reduced general life satisfaction and ability to find meaning and purpose in one's life; all three groups also reported feelings of poor self-efficacy. Finally, individuals with stroke were more likely than controls to report physical aggression, and the TBI and stroke groups were more likely than controls to indicate sadness, fear-affect, and perceived stress. Although findings evidenced increased rates of problems across NIHTB emotional measures, a substantial percentage of our sample had scores that were within normative limits on each of these measures, suggesting that there are measures within this battery that can represent relative strengths.

Although these data provide important descriptive information for our sample of individuals with different neurological disorders, it is also important to acknowledge several limitations. First, participants in this study were at least 1-year post-injury; thus, findings are not generalizable to individuals with acute injuries. This limitation is significant because the most substantial gains post-injury occur within the first 12 months for these groups (Burns et al., 2012; Dikmen et al., 1983, 1986, 1995, 2009; Ditunno et al., 1992; Jorgensen et al., 1995a, 1995b; Waters et al., 1992, 1993, 1994, 1998). Inclusion criteria were broad, and participants were not excluded on the basis of comorbid diagnoses and comorbid conditions were not collected. Although findings should be generalizable to other heterogeneous samples of individuals with neurological conditions, future work is needed to understand how comorbid conditions influence performance on the NIHTB. In addition, we did not assess effort nor did we query litigation status. Future work is needed to describe the impact that suboptimal effort or malingering has on the different domains of the NIHTB, although we did not note consistent outliers in performance on this battery. The stroke participants in our sample had less motor impairment than what is typical of individuals seen in inpatient stroke rehabilitation (and we excluded participants with severe aphasia). What is important is that the participants appear to be exhibiting subtle cognitive impairments that may benefit from early screening. This is especially important given the fact that patients who do not show physical limitations are often sent home without consideration of cognitive dysfunction. Therefore, for example, measures such as the NIHTB can help identify individuals with stroke-related executive dysfunction who need cognitive behavioral interventions focused on strategies to help them return to family, work, and community life. We also did not record information about stroke location; this should be a focus of future inquiry. Additional research is needed to better understand how the NIHTB relates to everyday functioning (degree of independence, etc.), and which measures might best predict functional impairments for these different neurological groups. Future work should also examine test–retest reliability and establish norms for change over time for these clinical populations.

Regardless of these limitations, these data provide evidence that may help clinicians interpret and utilize the NIHTB battery with individuals with neurological impairments. In particular, the average test scores per clinical group provide information from a large national sample that can be used to assist clinicians in interpretation and future recommendations for the individuals who are exhibiting difficulties. Given the significant group differences and variable patterns of performances across these batteries within each clinical cohort, assessment of these domains may provide important information with clinical implications potentially for rehabilitation need and daily functioning capacities among these individuals. In summary, the NIHTB demonstrates discriminant validity among clinical cohorts consistent with previous literature, which supports its clinical utility in individuals with TBI, SCI, and stroke.

Funding

This study was supported by the NIDILRR RRTC on Improving Measurement of Medical Rehabilitation Outcomes (Award number: H133B090024, PI: Heinemann). This project was also supported in part with Federal funds from the Blueprint for Neuroscience Research, National Institutes of Health, under Contract No. HHS-N-260-2006-00007-C. Co-Author, Kaitlin Casaletto, was supported in part by NIDA F31-DA035708 and the Foundation for Rehabilitation Psychology Dissertation Award. Co-Author, Alex Wong, was supported in part by the Craig H. Neilsen Foundation (Award number: 290474, PI: Wong).

Conflict of interest

None declared.