Skip to main content
NCBI home page
VA Author Manuscripts logoLink to VA Author Manuscripts
. Author manuscript; available in PMC: 2021 Sep 24.
Published in final edited form as: J Int Neuropsychol Soc. 2020 Sep 24;27(4):305–314. doi: 10.1017/S1355617720000879

Elevated Intra-individual Variability in Executive Functions and Associations with White Matter Microstructure in Veterans with Mild Traumatic Brain Injury

Scott F Sorg 1,2, Victoria C Merritt 1, Alexandra L Clark 1, Madeleine L Werhane 3, Kelsey A Holiday 3, Dawn M Schiehser 1,2,4, Mark Bondi 2,1, Lisa Delano-Wood 1,4,2
PMCID: PMC8462939  NIHMSID: NIHMS1621986  PMID: 32967755

Abstract

Objective:

We examined whether intra-individual variability (IIV) across tests of executive functions (EF-IIV) is elevated in Veterans with a history of mild traumatic brain injury (mTBI) relative to military controls (MCs) without a history of mTBI. We also explored relationships among EF-IIV, white matter microstructure, and post-traumatic stress disorder (PTSD) symptoms.

Method:

77 Veterans (mTBI=43, MCs=34) completed neuropsychological testing, diffusion tensor imaging (DTI), and PTSD symptom ratings. EF-IIV was calculated as the standard deviation across six tests of EF, along with an EF-Mean composite. DSI Studio connectometry analysis identified white matter tracts significantly associated with EF-IIV according to generalized fractional anisotropy (GFA).

Results:

After adjusting for EF-Mean and PTSD symptoms, the mTBI group showed significantly higher EF-IIV than MCs. Groups did not differ on EF-Mean after adjusting for PTSD symptoms. Across groups, PTSD symptoms significantly negatively correlated with EF-Mean, but not with EF-IIV. EF-IIV significantly negatively correlated with GFA in multiple white matter pathways connecting frontal and more posterior regions.

Conclusions:

Veterans with mTBI demonstrated significantly greater IIV across EF tests compared to MCs, even after adjusting for mean group differences on those measures as well as PTSD severity. Findings suggest that, in contrast to analyses that explore effects of mean performance across tests, discrepancy analyses may capture unique variance in neuropsychological performance and more sensitively capture cognitive disruption in Veterans with mTBI histories. Importantly, findings show that EF-IIV is negatively associated with the microstructure of white matter pathways interconnecting cortical regions that mediate executive function and attentional processes.

Keywords: brain concussion, TBI, traumatic brain injury, mild TBI, mTBI, cognition, assessment, patient outcome, diffusion tensor imaging, neuroimaging, executive control

INTRODUCTION

The substantial number of mild traumatic brain injuries (mTBI) sustained by service members amid the US military engagements in Iraq and Afghanistan has contributed to an increased emphasis on understanding and clarifying the acute and chronic effects of mTBI. One of the most commonly reported acute sequelae of mTBI is cognitive dysfunction, and both subjective cognitive complaints and objective cognitive deficits have been well documented in this population (Belanger, Kretzmer, Vanderploeg, & French, 2010; Dolan et al., 2012; French, Lange, & Brickell, 2014). Although recovery is generally anticipated to occur within several days to weeks following mTBI (McCrea et al., 2009; Rohling et al., 2011), some studies have shown that Veterans with a history of mTBI continue to show cognitive and neurobehavioral dysfunction for many months and even years following injury (Delano-Wood et al., 2015; Mac Donald et al., 2017; Sorg et al., 2016; Vanderploeg, Curtiss, & Belanger, 2005). Although a number of neuropsychological and brain imaging studies have been conducted on the Veteran population to improve understanding of the long-term consequences of mTBI, the precise mechanisms underlying this cognitive dysfunction remain unclear and understudied.

To date, neuropsychological studies investigating the effects of mTBI have almost exclusively relied on examinations of central tendency, or the average performance across groups, which has itself been the mainstay of neuropsychological research. However, alternative approaches to detecting abnormalities in neuropsychological functioning such as within-person variability, or intra-individual variability (IIV), are beginning to receive more attention as they may more accurately reflect cognitive impairment, particularly in younger populations with milder head injuries (Cole, Gregory, Arrieux & Haran, 2017; Hill, Rohling, Boettcher, & Meyers, 2013; Merritt et al., 2018). IIV has traditionally been classified as either inconsistency of performances across trials on a particular test (e.g., variable reaction time) or as dispersion, defined as variability of an individual’s performance across a set number of measures.

Studies examining IIV in TBI are generally limited and existing research has primarily focused on inconsistency-related IIV as opposed to dispersion-related IIV (e.g., Cole et al., 2017). One of the studies investigating cognitive dispersion in the context of TBI, Hill, Rohling, Boettcher, & Meyers (2013) examined a large civilian sample and found that greater dispersion (calculated across a broad battery of neuropsychological tests) was (1) inversely related to mean cognitive performance and (2) associated with TBI severity, such that those with more severe TBIs showed greater dispersion than those with milder TBIs. Another study investigated cognitive dispersion in a sample of college athletes who were evaluated at baseline and following sports-related concussion (Rabinowitz & Arnett, 2013). Study authors showed an inverse relationship between dispersion and mean cognitive performance at both baseline and post-concussion. Moreover, results demonstrated that athletes showing high versus low variability at baseline were more likely to exhibit cognitive dysfunction post-injury. More recently, Merritt et al. (2018) investigated cognitive dispersion in military Veterans with and without a history of remote mTBI and found elevated dispersion in Veterans with mTBI relative to military controls (MCs). Additionally, results showed greater dispersion in Veterans with a history of 3 or more mTBIs compared to those with a history of 1–2 mTBIs. Together, these studies suggest that IIV may uniquely contribute to our understanding of cognitive functioning following head injury.

The neural underpinnings of cognitive dispersion are not well established, though emerging findings suggest that dispersion-related IIV is a sensitive marker of cortical integrity and may denote the breakdown of neuronal networks (Hines et al., 2016). For example, in studies of both cognitive aging and HIV, elevations in dispersion have been associated with gray matter atrophy (Bangen et al., 2019, Hines et al., 2016). Within the context of HIV positivity, alterations in white matter microstructure (i.e., lower fractional anisotropy values) in frontal-subcortical regions have been observed (Jones et al., 2018). This latter finding tying dispersion to white matter is relevant in the context of mTBI, as white matter is particularly sensitive to even mild levels of neurotrauma (Morey et al., 2013; Sorg et al., 2016). Examining neural correlates of IIV may provide further evidence for IIV as a valid measure of cerebral dysfunction following mTBI. To our knowledge, no existing studies have examined associations between neuroimaging metrics and cognitive dispersion in TBI.

Given our prior work showing strong associations between executive function decrements and reduced white matter integrity in Veterans with mTBI using diffusion tensor imaging (DTI, Sorg et al., 2014), we focused the present study on (1) examining cognitive dispersion specific to executive functions (hereafter referred to as EF-IIV) and (2) investigating the associations of EF-IIV with white matter anisotropy. We hypothesized that Veterans with a history of mTBI would show greater EF-IIV relative to MCs and that higher EF-IIV would be associated with lower anisotropy. As a secondary analysis, we examined whether the mTBI group demonstrated lower anisotropy within white matter tracts found to be associated with EF-IIV. Finally, because of the high prevalence of posttraumatic stress disorder (PTSD) in Veterans with a history of mTBI (Vasterling & Dikmen, 2012; Yurgil et al., 2014) and demonstrated associations between PTSD symptoms and neuropsychological test performance in this population (Donnelley, Donnelly, Warner, Kittleson, & King, 2018), we examined (as an exploratory aim) whether PTSD symptoms were associated with EF-IIV within the mTBI group.

METHODS

Participants & Procedures

Study participants were 77 military Veterans (mTBI=43, MCs=34) who were recruited from posted recruitment flyers and various outpatient clinics within the Veterans Affairs San Diego Healthcare System (VASDHS). Prior to study participation, written and informed consent in compliance with the institutional review boards of the University of California, San Diego (UC San Diego) and VASDHS was obtained for each participant. Study participation included a TBI interview, neuropsychological testing, completion of self-report psychiatric and postconcussive symptom questionnaires, and brain imaging.

Participants were excluded from the study if they: (1) had previously suffered a moderate or severe TBI (i.e., loss of consciousness [LOC] > 30 minutes, alteration of consciousness [AOC] or posttraumatic amnesia [PTA] > 24 hours; (Traumatic Brain Injury Task Force, 2009); (2) exhibited poor performance on tests of effort as described below; (3) endorsed a history of a learning disability; (4) had a history of a major neurological/medical or psychiatric illness (e.g., epilepsy, multiple sclerosis, chronic fatigue syndrome, bipolar disorder, schizophrenia); (5) met Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) criteria for current substance or alcohol abuse; or (6) demonstrated any contraindications (e.g., shrapnel or claustrophobia) to magnetic resonance imaging (MRI).

TBI Diagnostic Procedure

Per Department of Defense (DoD) guidelines (Traumatic Brain Injury Task Force, 2009), diagnosis of mTBI was based on the following: (1) LOC ≤ 30 minutes; (2) AOC ≤ 24 hours; and/or (3) PTA ≤ 24 hours. A detailed TBI history was obtained using a modified version of the VA Semi-Structured Clinical Interview for TBI (Vanderploeg et al., 2012). Each participant was probed via open-ended questions about any head injuries they may have sustained both during their military service (i.e., military-related) and prior to or after their military service (i.e. non-military). Diagnostic information obtained from this interview included duration of LOC, AOC, and PTA; mechanism of injury (i.e., blast or blunt-force); whether medical attention was received; quantification of the severity and presence of any neurological symptoms (i.e., headache, tinnitus) after each reported head injury; and details regarding blast exposures (i.e., number, distance, and direction [i.e., front, back, left, right] of any blast in which they were exposed).

Neuropsychological Assessment

All participants completed a comprehensive neuropsychological test battery as part of a larger investigation that included assessment of effort, memory, executive functions, and processing speed. For the purpose of the present study, only tests of executive functions and performance validity were examined. Measures of executive functions included the Trail Making Test, Design Fluency and Verbal Fluency subtests of the Delis-Kaplan Executive Function System (D-KEFS; Delis, Kaplan, & Kramer, 2001), and the Wisconsin Card Sorting Test-64 (WCST-64; Kongs, 2000). Measures of performance validity included the Test of Memory Malingering (TOMM; Tombaugh, 1996) Trial 2 and the Forced Choice subtest of the California Verbal Learning Test-II (CVLT-II; Delis, Kramer, Kaplan, & Ober, 2000). One participant was excluded scoring below manual-defined cutoffs on two measures of performance validity (Moore & Donders, 2004; Tombaugh, 1996). Participants also completed the Wide Range Achievement Test 4 (WRAT-4) Word Reading test as an estimate of pre-morbid intellectual functioning (Wilkinson & Robertson, 2006).

Intra-individual Variability.

The primary variable of interest in the current study was an index of IIV, or dispersion, across the six measures of executive function (EF-IIV). First, age-corrected scaled scores for the (1) Trails Number-Letter Switching Total Time, (2) Design Fluency Switching Total Time, (3) Letter Fluency Total, (4) Category Fluency Switching Total, and T-scores adjusted for age and education for the (5) WCST-64 Total Errors and (6) WCST-64 Perseverative Errors were converted to z-scores. Next, the EF-IIV score was calculated for each Veteran by computing the standard deviation of the converted z-scores across the six executive function variables (Morgan et al., 2011). Higher EF-IIV scores are indicative of greater variability across the tests of executive functions. The same process was used to calculate EF-IIV based on raw test scores, with z-scores converted, as appropriate (e.g., Trails Number-Letter Switching Total Time), such that higher scores showed better performance.

Executive Function Composite.

A composite of overall performance on tests of executive function (EF-Mean) was created by averaging each participant’s converted z-scores across the six measures. Higher composite scores are indicative of better performance across the tests. Composite scores were calculated for both the normed data and raw data.

Psychiatric Symptom Ratings

To assess current levels of PTSD symptoms, participants completed the PTSD Checklist—Military Version (PCL-M), which corresponds directly with DSM-IV-TR diagnostic criteria for PTSD (Weathers et al., 1991).

Neuroimaging Protocol and Procedures

Participants underwent structural MRI and DTI on 3T GE Discovery MR750 whole-body scanner using an 8-channel receive-only head coil at UCSD’s Center for Functional MRI.

Structural Scanning.

A sagitally acquired high-resolution 3D T1-weighted anatomical MRI was collected over approximately 8 minutes with the following parameters: FOV = 24 cm, 256 × 256 × 192 matrix, .94 × .94 × 1.25 mm voxels, 172 slices, TR = 20 ms, TE = 3.1 ms, T1 = 550, flip angle 12°.

Diffusion Image Scanning.

DTI images were collected via dual spin echo EPI acquisition (Reese, Heid, Weisskoff, & Wedeen, 2003) over 16 minutes with the following parameters: FOV = 240 mm, slice thickness = 3 mm, matrix size 128 × 128, in-plane resolution = 1.875 × 1.875, TR 8000ms, TE 93 ms. Thirty-four slices were acquired with 61 diffusion directions distributed on the surface of a sphere in conjunction with the electrostatic repulsion model (Jones, Horsfield, & Simmons, 1999) and a b-value of 1500 s/mm2. One T2 weighted image with no diffusion (b = 0) was also collected along with two field map images to be used in magnetic field homogeneity corrections.

Diffusion Imaging Processing.

EPI acquisitions were unwarped with two field maps, and subsequently motion and eddy current corrected using tools from the Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB) Software Library (FSL). Next, all images were visually inspected for quality assurance (no subjects were removed), and any non-brain voxels were removed from these images using the FSL brain extraction tool.

Diffusion data were further processed within DSI Studio (http://dsi-studio.labsolver.org). Diffusion data were reconstructed using q-space diffeomorphic reconstruction to obtain the spin distribution function, and individual anisotropy maps were aligned to MNI space using a non-linear registration to the Human Connectome Project 842 subject template (Yeh & Tseng, 2011). A diffusion sampling length ratio of 1.25 was used, and the output resolution used in subsequent analyses was 2 mm3. All subject registrations were inspected for quality and goodness-of-fit using the R2 statistic between the template image and the warped individual subject diffusion image dataset (Yeh, Tang, & Tseng, 2013).

The primary metric used to describe the quality of the white matter was generalized fractional anisotropy (GFA) and was computed within DSI Studio. Similar to traditional diffusion tensor models of anisotropy including fractional anisotropy (FA), lower GFA values are consistent with undirected, isotropic diffusion, while higher GFA values indicate more unidirectional diffusion, consistent with healthier white matter. However, traditional FA provides poor anisotropy estimates in regions with crossing fibers (De Santis, Drakesmith, Bells, Assaf, & Jones, 2014; Nilsson, Latt, Stahlberg, van Westen, & Hagslatt, 2012; Oouchi et al., 2007). GFA is an extension of FA to high-angular resolution diffusion-weighted image that is capable of measuring anisotropy across multiple diffusion directions (Tuch, 2004).

Statistical Analyses

Group comparisons (i.e., mTBI versus MCs) for demographic and neuropsychological test variables were performed with analysis of variance (ANOVA) and analysis of covariance (ANCOVA) for continuous variables and Χ2 tests for categorical variables. Correlational analyses were conducted using Pearson correlations.

A complete description of the connectometry analysis methods is detailed by Yeh, Badre, & Verstynen (2016). In brief, connectometry analysis examines the local connectome or the degree of connectivity between adjacent voxels within a white matter fascicle. Traditional end-to-end fiber tracking examines the average diffusivity value across the whole of an a priori identified tract (e.g., FA averaged across all the voxels within a tract). In contrast, connectometry employs a “track the difference” paradigm where tracking is performed in the segment of fiber bundle that exhibits significant association with the study variable, identified using a specified T-score threshold. Random permutations of the associations between study variables and the local connectome matrices are performed and tracked to obtain a null distribution of tract lengths if associations to the study variables were due to chance alone. Tracts connected from false findings (i.e., drawn from data permutations) appear as randomly distributed short-distance fragments while true findings can be differentiated by longer trajectories propagating along a common fiber pathway. The false discovery rate (FDR) can be directly calculated from the length histogram obtained from permuted and non-permuted conditions as the ratio of the area under the two distribution curves.

For the present study, following registration to a standard space (e.g., MNI) as described above, multiple regression was used to identify GFA-based local connectomes that showed significant associations with the EF-IIV adjusting for PCL-M score, age and EF-Mean. A T-threshold of 2 was assigned to select the local connectomes which were then tracked along the core pathway using a deterministic fiber tracking (Yeh, Verstynen, Wang, Fernandez-Miranda, & Tseng, 2013) to reveal the subcomponents of the fascicles that have significant associations with the study variables. Topology-informed pruning (Yeh et al., 2018) was conducted with 1 iteration to remove false connections. All tracts generated from bootstrap resampling were included. A length threshold of 20 voxel distance was used to select tracts. The seeding number for each permutation was 10000. To estimatee the FDR, a total of 2000 randomized permutations were applied to obtain the null distribution of the tract length. Mean GFA data across the resultant tract set was extracted for each participant and imported into SPSS Version 26.0 for follow-up analyses.

RESULTS

Demographic comparisons between MCs and Veterans with a history of mTBI are shown in Table 1. While the MC group on average reported having completed approximately one more year of education relative to the mTBI group, groups reported similar levels of academic achievement (as reflected by the highest level of education obtained) and they did not differ on an estimate of pre-morbid intellectual functioning (WRAT-4 Reading). With regard to other demographic variables, there were significantly fewer women and non-White participants in the mTBI group relative to the MC group (both p-values = .002). Additionally, as expected, the mTBI group endorsed significantly greater PTSD symptomatology on the PCL-M relative to the MC group (p < .001). Regarding mTBI characteristics, most Veterans (75%) reported having more than one mTBI with approximately half of the participants reporting that they experienced LOC during their most significant mTBI. Additionally, the majority of participants in the mTBI group reported having experienced a blast-related mTBI. Regarding time since injury, Veterans on average reported that their most recent mTBI was more than five years ago.

Table 1.

Sample characteristics by group.

MCs (n = 34) mTBI (n = 43) p
Age 32.4 (8.1) 32.6 (7.3) .94
Number of Men 21 (62%) 39 (91%) .002
Total Years of Education 15.2 (1.9) 14.3 (1.6) .04
Highest Level of Education .27
 High School Graduate (No College) 3 (8.8%) 3 (7.0%) -
 Some College 16 (47.0%) 28 (65.1%) -
 College Graduate (4 Year Degree) 15 (44.1%) 12 (27.9%) -
Ethnicity
 White 27 (79.4%) 18 (41.8%) .002
 Non-White 7 (20.6%) 25 (58.1%)
WRAT-4 Reading Standard Score 105.5 (8.8) 103.7 (12.0) .46
PTSD Checklist - Military Version 21.5 (8.3) 40.7 (17.8) < .001
TBI Characteristics
 Total Number of mTBIs - 2.6 (1.6) -
  1 mTBI - 11 (25%) -
  2 mTBIs - 14 (33%) -
  3+ mTBIs - 18 (42%) -
 Years since Most Recent mTBI - 5.9 (3.9) -
 Reported Loss of Consciousness - 21 (49%) -
 Blast-Related mTBI - 28 (65%) -
Open in a new tab

MCs = Military Controls; mTBI = mild traumatic brain injury; p = p-value for t-test or chi square; WRAT-4 = Wide Range Achievement Test, 4th edition; PTSD = Post-traumatic stress disorder.

Neuropsychological Findings

An ANOVA revealed that the MC group demonstrated significantly better performance on tasks of executive function (EF-Mean) than the mTBI group (F1,75 = 9.93, p = .002, η2 = 0.12, see Figure 1). However, when the PCL-M total score was added as a covariate, an ANCOVA revealed that this difference in EF-Mean performance was greatly attenuated and was at trend level [F1,74 = 3.11, p = .08, η2 = 0.04; adjusted EF-Mean and standard error: mTBI = −0.10 (0.09), MCs = .155 (0.10)]. Another ANOVA showed that the mTBI group had significantly higher EF-IIV compared to MCs (F1,75 = 5.58, p = .021, η2 = 0.07, see Figure 1), and this finding remained unchanged when including the PCL-M total score as a covariate [F1,74 = 5.39, p = .023, η2 = 0.07, adjusted EF-IIV and standard error: mTBI = 0.92 (0.06), MCs = 0.72 (0.07)]. To evaluate whether this effect was a by-product of lower mean test performance, the EF-Mean score was added to the model as an additional covariate. With the addition of this variable, group differences remained largely unchanged (F1,73 = 4.22, p = .043, η2 = 0.06). A follow-up analysis using EF-IIV generated from raw test score data (i.e., non-normed test scores) correcting for EF-Mean (non-normed), PCL-M scores, and age continued to show increased EF-IIV in the mTBI group relative to MC group (F1,72 = 4.22, p = .044, η2 = 0.055).

Figure 1.

Figure 1.

Open in a new tab

Group Comparison of EF-Mean and EF-IIV.

Group comparisons in terms of executive function performances for both (a) mean level performance (EF-Mean) and (b) intra-individual variability of executive function (EF-IIV). The mTBI group showed significantly lower EF-Mean compared to controls (p = .002), but after adjusting for current PTSD symptoms this difference was at a trend level (p = .08). In contrast, mTBI EF-IIV was significantly higher than controls (p = .02) even after adjusting for PTSD symptoms and EF-Mean (p = .04).

Follow-up analyses were conducted in order to evaluate possible contributions of PTSD symptoms to neuropsychological performance and variability in test scores. Pearson correlations showed that, across groups, PCL-M scores were significantly correlated with EF-Mean performance (r = −.310, p = .006), but not with EF-IIV (r = .039, p = .74). However, within subgroups (i.e., MCs and mTBI), PCL-M scores were not significantly correlated with either EF-Mean (MCs: r = −.215, p = .223; mTBI: r = −.185, p = .234) or EF-IIV (MCs: r = .183, p = .300; mTBI: r = −.131, p = .402).

Neuroimaging Findings

Results of the diffusion image connectometry analysis across both groups revealed that EF-IIV demonstrated significant negative associations with GFA in the fornix, right and left cingulum, genu, body and splenium of the corpus collosum, right and left corticospinal tract, the right and left inferior fronto-occipital fasciculus, and the right and left superior longitudinal fasciculus (FDR corrected p = .021; see Figure 2). There was no association between increased GFA and EF-IIV across any WM tracts (FDR=1). The mean GFA across all tracts associated with EF-IIV was then extracted, and partial correlations controlling for EF-Mean, PCL-M scores and age confirmed a strong negative association between EF-IIV and the mean tract GFA across groups (rp = −.474, p < .001) as well as within the mTBI group (rp = −.447, p = .004) and MC group (rp = −.573, p = .001). There were no statistically significant associations between PCL-M scores and extracted GFA for tracts associated with EF-IIV in the mTBI group (r = −.181, p = .250) or in the MC group (r = −.212, p = .237). In terms of group comparisons of the mean GFA across all tracts, groups did not significantly differ in GFA tracts associated with EF-IIV (F1,73 = 2.30, p = .133, η2 = 0.031).

Figure 2.

Figure 2.

Open in a new tab

Connectometry analysis results showing EF-IIV associations with GFA across mTBI and MC groups.

Results of connectometry analysis showing fiber pathways with significant negative associations between GFA and IIV-EF controlling for EF-Mean, PTSD symptoms, and age. Tracts include the fornix, right and left cingulum, genu, body and splenium of the corpus callosum, right and left cortical spinal tract (CST), and the right and left inferior fronto-occipital fasciculus (IFOF) and the right and left superior longitudinal fasciculus (SLF). Results are corrected for multiple comparisons via permutation testing (2,000 iterations) using false discovery rate (p-corrected < .05). Panels A & B show the tracts on the right and left hemispheres, respectively. Panel C shows tracts from a superior to inferior viewing angle. Panel D shows the scatter plot depicting the association between GFA across all tracts identified in the connectometry analysis against the EF-IIV values. The trend line represents the zero-order correlation between all tract GFA values and EF-IIV for both groups. Partial correlations controlling for EF-Mean, PCL-M scores, and age demonstrate the identified significant negative association between EF-IIV and the mean tract GFA across groups (rp = −.474, p < .001) as well as within the mTBI group (rp = −.447, p = .004) and MC group (rp = −.573, p = .001).

DISCUSSION

To our knowledge, the current study is the first to examine cognitive dispersion across tests of executive functions (EF-IIV) and associations with brain white matter microstructure in a well-characterized sample of Veterans with and without reported histories of mTBI. Key findings demonstrate that (1) relative to military controls, Veterans with a history of mTBI demonstrated elevated EF-IIV and (2) greater EF-IIV (i.e., greater inconsistency of scores) was associated with less uniform microstructure across a number of white matter pathways.

Our first primary finding of elevated EF-IIV being associated with mTBI aligns with prior reports showing greater IIV (cognitive dispersion) in populations with known cerebral dysfunction (Bangen et al., 2019; Hill et al., 2013; Hines et al., 2016; Merritt et al., 2018). Notably, when compared to controls, our mTBI group showed overall reduced performance on tests of executive function (i.e., their EF-Mean score); however, similar to other studies (Vasterling & Dikmen, 2012; Verfaellie, Lafleche, Spiro, & Bousquet, 2014), comorbid psychiatric factors played a significant role in driving those effects as mean differences were no longer significant once PTSD symptom severity was included in the model. In contrast to the EF-Mean results, the EF-IIV scores were significantly elevated in the mTBI group relative to controls regardless of the inclusion of PTSD symptoms. In fact, PTSD symptoms were not associated with EF-IIV when collapsed across or examined within groups. Furthermore, the EF-IIV results remained significant when adjusting for EF-Mean performance. Thus, the elevations in EF-IIV in the mTBI group do not appear to be a consequence of comorbid PTSD symptoms or a by-product of reduced EF-Mean scores, suggesting that EF-IIV may be a more robust indicator of persisting cognitive disruption following mTBI than traditional central tendency measures.

The notion that elevated EF-IIV may reflect cerebral dysfunction is further bolstered by our second significant finding showing that dispersion scores are highly correlated with the microstructure of multiple white matter pathways, independent of PTSD symptoms and EF-Mean performance. These pathways (depicted in Figure 2) reveal a broad network incorporating multiple cortical regions which span from frontal to more posterior areas. While cognitive variability has been thought to be facilitated by frontal lobe functions (see MacDonald, Li, & Backman, 2009 for review), more recent work suggests involvement of non-frontal regions including limbic and posterior attentional networks working in concert with frontal regions regulating cognitive processes. (Bangen et al., 2019; Hines et al., 2016; Jones et al., 2018). Taken together, our findings align with those of others suggesting that IIV is facilitated by neuroanatomical regions consistent with attentional and executive function networks (Hines et al., 2016; Kelly, Uddin, Biswal, Castellanos, & Milham, 2008).

Although it is widely thought that white matter is vulnerable to damage associated with mTBI (Bigler & Maxwell, 2012; Buki & Povlishock, 2006), we found that groups did not significantly differ in GFA values across tracts associated with EF-IIV. Other studies of Veterans with histories of mTBI have reported group differences in WM microstructure (Morey et al., 2013; Sorg et al., 2016) including within regions associated with EF-IIV such as the genu and the cingulum. The findings indicate that while elevations in IIV may manifest in conditions affecting the central nervous system such as mTBI, aspects of the neuroanatomical regions associated with increased IIV need not be compromised for such cognitive alterations to occur.

The lack of an association between PTSD symptoms and EF-IIV was somewhat unexpected as other studies have shown associations between PTSD symptoms and inconsistency-related IIV (i.e., response variability using reaction times, Clouston et al., 2017; Swick, Honzel, Larsen, & Ashley, 2013). For example, reaction time variability on a go/no-go task was found to be significantly greater in a sample of Veterans with PTSD compared to controls, and variability scores significantly positively correlated with the degree of PTSD symptoms in this sample (Swick et al., 2013). However, 34 of the 45 participants in the PTSD sample had a history of mTBI, and it was not clear from their analysis whether mTBI may have impacted observed increased variability. Additionally, the use of inconsistency-related IIV (i.e., performance within one test) as opposed to dispersion-related IIV (i.e., performance across multiple tests) may have contributed to the discrepant findings. Our results do align with prior work from our lab showing no association between dispersion-related IIV and PTSD symptoms using a broader range of neuropsychological tests encompassing multiple cognitive domains (Merritt et al., 2018). Our finding of significant associations between PTSD and the EF-Mean suggests that, rather than being a source of intermittent disruption to cognition during testing, PTSD symptoms apply a uniform strain across tests of EF. Of note, the PTSD ratings we used consider PTSD symptoms over the past 30 days, and thus do not reflect moment-to-moment variations in PTSD symptoms that may occur during a demanding testing session.

The clinical interpretation of variable test scores is challenging because many factors may contribute to variability in cognitive performance. For example, performance variability has been proposed as a measure of malingering (Strauss et al., 2002), and elevated dispersion has been observed in patients who display suboptimal effort on neuropsychological tests (Hill et al., 2013). However, it is not likely that effort played a significant role in our findings as our analysis excluded participants who performed below established cut-off scores on multiple tests of effort (i.e., TOMM and CVLT-II Forced Choice). Additionally, a prominent view within neuropsychology is that some degree of performance variability across a set of cognitive tests can be expected and is not in itself indicative of pathology (Binder, Iverson, & Brooks, 2009). Given the regularity of within-subject variability and questions regarding its significance as a marker of neural dysfunction, prior research attempted to establish a normative threshold for cognitive dispersion (Tanner-Eggen, Balzer, Perrig, & Gutbrod, 2015). Using a large database of normative test data, they suggested a cut score for “abnormally high IIV” (i.e., possibly indicating an underlying pathological condition) of IIV > 0.91, which corresponds to an IIV below the 16th percentile. Post-hoc application of this cut score to our sample found a statistically significant group difference, with 47% of the mTBI group showing EF-IIV values above threshold compared to only 21% of MCs (χ2= 7.073, p = .008). These findings provide strong evidence that the elevated EF-IIV shown within the mTBI group is consistent with a clinically significant level of dysregulation resulting in inconsistent performance in executive functions.

Our study has a number of strengths, including a well-characterized sample of military Veterans with and without a history of mTBI. Additionally, our use of a relatively novel diffusion MRI neuroimaging technique enables statistical analysis of whole-brain diffusion scalar properties and their associations with our variables of interest (i.e., EF-IIV). Importantly, the neuroimaging methods employed in this study were selected to best identify white matter tracts associated with EF-IIV and as such were not necessarily optimized to identify group differences in white matter. There are a number of alternative approaches including a priori regions of interest (e.g., Sorg et al., 2016) or “pothole” analysis (e.g., Miller et al., 2016) which may show greater sensitivity toward identifying group differences with respect to white matter microstructure in the context of mTBI; these will be considered in our future work. Additionally, the native acquisition resolution of the diffusion data may have contributed to partial voluming effects in voxels containing different tissue types or where relatively smaller structures may not be resolved. It is possible that effects reduced the sensitivity of the connectometry analysis within such regions due to reduced precision of the white matter diffusion estimates. However, the diffusion data are resampled to 2mm isotropic resolution within the analysis, which mitigates some concerns with respect to partial voluming. Finally, the specificity of the white matter pathways to executive functions is unclear. Future research efforts should be directed at examining whether cognitive dispersion may be uniformly elevated across other domains.

One potential limitations of the present study concerns the make-up of our sample. For example, a comparison group of Veterans with PTSD but without a history of TBI would enable a more robust examination of the effect of PTSD on IIV. Moreover, our groups were not well matched with respect to ethnicity and sex which may limit findings. When sex is included as a covariate in the analysis, alongside PCL-M scores and EF-Mean, EF-IIV shows a strong trend toward being significantly higher in the mTBI group compared to controls (F1,72 = 3.88, p = .053, η2 = 0.05); however, sex was not a significant predictor of EF-IIV (standardized beta = −.051, p = .694).

In conclusion, our results demonstrate that EF-IIV may be more sensitive to the subtle effects of mTBI than traditional measures of central tendency. Additionally, EF-IIV is associated with a broad distribution of white matter pathways which largely comprise frontal-parietal attentional networks. Additional research investigating long-term outcomes of mTBI for those with high EF-IIV will be important to determine if such elevations are predictive of further objective cognitive decline.

ACKNOWLEDGMENTS

The authors sincerely thank the OEF/OIF (Operation Enduring Freedom/Operation Iraqi Freedom) Veterans who volunteered to participate in this study. This work was supported by Veterans Affairs research grants including Career Development Awards to Drs. Delano-Wood (CDA-2-034-07F) and Schiehser (CDA-2-065-10S), a Veterans Affairs Merit Award to Dr. Delano-Wood (CX000842-01A2) as well as a Department of Defense Investigator-Initiated Research Grant awarded to Dr. Delano-Wood (W81XWH-10-2-0169). This material is further supported by resources of the Veterans Affairs Center of Excellence for Stress and Mental Health. Dr. Sorg received salary support from a VA Clinical Science Research & Development funded Career Development Award (CX001508) during the preparation of this manuscript, and Dr. Merritt received salary support during this work from a Career Development Award (IK2 CX001952) from the VA Clinical Science Research & Development Service.

Footnotes

The authors declare no conflicts of interest.

REFERENCES

  1. Bangen KJ, Weigand AJ, Thomas KR, Delano-Wood L, Clark LR, Eppig J, … Bondi MW (2019). Cognitive dispersion is a sensitive marker for early neurodegenerative changes and functional decline in nondemented older adults. Neuropsychology, 33(5), 599–608. 10.1037/neu0000532 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Belanger HG, Kretzmer T, Vanderploeg RD, & French LM (2010). Symptom complaints following combat-related traumatic brain injury: Relationship to traumatic brain injury severity and posttraumatic stress disorder. J Int Neuropsychol Soc, 16(1), 194–199. 10.1017/S1355617709990841 [DOI] [PubMed] [Google Scholar]
  3. Bigler ED, & Maxwell WL (2012). Neuropathology of mild traumatic brain injury: Relationship to neuroimaging findings. Brain Imaging Behav, 6(2), 108–136. 10.1007/s11682-011-9145-0 [DOI] [PubMed] [Google Scholar]
  4. Binder LM, Iverson GL, & Brooks BL (2009). To err is human: “abnormal” neuropsychological scores and variability are common in healthy adults. Arch Clin Neuropsychol, 24(1), 31–46. 10.1093/arclin/acn001 [DOI] [PubMed] [Google Scholar]
  5. Buki A, & Povlishock JT (2006). All roads lead to disconnection?—Traumatic axonal injury revisited. Acta Neurochir (Wien), 148(2), 181–193; discussion 193–4. 10.1007/s00701-005-0674-4 [DOI] [PubMed] [Google Scholar]
  6. Clouston S, Pietrzak RH, Kotov R, Richards M, Spiro A, Scott S, … Luft BJ (2017). Traumatic exposures, posttraumatic stress disorder, and cognitive functioning in World Trade Center responders. Alzheimers Dement (N Y), 3(4), 593–602. 10.1016/j.trci.2017.09.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cole WR, Gregory E, Arrieux JP, & Haran FJ (2018). Intraindividual Cognitive Variability: An Examination of ANAM4 TBI-MIL Simple Reaction Time Data from Service Members with and without Mild Traumatic Brain Injury. Journal of the International Neuropsychological Society, 24(2), 156–162. 10.1017/S1355617717001187 [DOI] [PubMed] [Google Scholar]
  8. De Santis S, Drakesmith M, Bells S, Assaf Y, & Jones DK (2014). Why diffusion tensor MRI does well only some of the time: Variance and covariance of white matter tissue microstructure attributes in the living human brain. Neuroimage, 89, 35–44. 10.1016/j.neuroimage.2013.12.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Delano-Wood L, Bangen KJ, Sorg SF, Clark AL, Schiehser DM, Luc N, … Bigler ED (2015). Brainstem white matter integrity is related to loss of consciousness and postconcussive symptomatology in veterans with chronic mild to moderate traumatic brain injury. Brain Imaging Behav, 9(3), 500–512. 10.1007/s11682-015-9432-2 [DOI] [PubMed] [Google Scholar]
  10. Delis D, Kaplan E, & Kramer JH (2001). The Delis-Kaplan Executive Function System. San Antonio, TX: Psychological Corporation. [Google Scholar]
  11. Delis D, Kramer JH, Kaplan E, & Ober B. (2000). The California Verbal Learning Test: Second Edition. San Antonio, TX: Psychological Corporation. [Google Scholar]
  12. Dolan S, Martindale S, Robinson J, Kimbrel NA, Meyer EC, Kruse MI, … Gulliver SB (2012). Neuropsychological sequelae of PTSD and TBI following war deployment among OEF/OIF veterans. Neuropsychol Rev, 22(1), 21–34. 10.1007/s11065-012-9190-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Donnelly K, Donnelly JP, Warner GC, Kittleson CJ, & King PR (2018). Longitudinal study of objective and subjective cognitive performance and psychological distress in OEF/OIF veterans with and without traumatic brain injury. The Clinical Neuropsychologist, 32(3), 436–455. [DOI] [PubMed] [Google Scholar]
  14. French LM, Lange RT, & Brickell T. (2014). Subjective cognitive complaints and neuropsychological test performance following military-related traumatic brain injury. J Rehabil Res Dev, 51(6), 933–950. 10.1682/JRRD.2013.10.0226 [DOI] [PubMed] [Google Scholar]
  15. Haynes BI, Bauermeister S, & Bunce D. (2017). Does within-person variability predict errors in healthy adults aged 18–90? Q J Exp Psychol (Hove), 70(8), 1722–1731. 10.1080/17470218.2016.1204328 [DOI] [PubMed] [Google Scholar]
  16. Hill BD, Rohling ML, Boettcher AC, & Meyers JE (2013). Cognitive intra-individual variability has a positive association with traumatic brain injury severity and suboptimal effort. Arch Clin Neuropsychol, 28(7), 640–648. 10.1093/arclin/act045 [DOI] [PubMed] [Google Scholar]
  17. Hines LJ, Miller EN, Hinkin CH, Alger JR, Barker P, Goodkin K, … Multicenter ACS (2016). Cortical brain atrophy and intra-individual variability in neuropsychological test performance in HIV disease. Brain Imaging Behav, 10(3), 640–651. 10.1007/s11682-015-9441-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Jones DK, Horsfield MA, & Simmons A. (1999). Optimal strategies for measuring diffusion in anisotropic systems by magnetic resonance imaging. Magn Reson Med, 42(3), 515–525. [PubMed] [Google Scholar]
  19. Jones JD, Kuhn T, Mahmood Z, Singer EJ, Hinkin CH, & Thames AD (2018). Longitudinal intra-individual variability in neuropsychological performance relates to white matter changes in HIV. Neuropsychology, 32(2), 206–212. 10.1037/neu0000390 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kelly AM, Uddin LQ, Biswal BB, Castellanos FX, & Milham MP (2008). Competition between functional brain networks mediates behavioral variability. Neuroimage, 39(1), 527–537. 10.1016/j.neuroimage.2007.08.008 [DOI] [PubMed] [Google Scholar]
  21. Kongs KS (2000). Wisconsin card sorting test-64 card version (WCST-64). Odessa, FL: Psychological Assessment Resources. [Google Scholar]
  22. Mac Donald CL, Barber J, Jordan M, Johnson AM, Dikmen S, Fann JR, & Temkin N. (2017). Early Clinical Predictors of 5-Year Outcome After Concussive Blast Traumatic Brain Injury. JAMA Neurol, 74(7), 821–829. 10.1001/jamaneurol.2017.0143 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. MacDonald SW, Li SC, & Backman L. (2009). Neural underpinnings of within-person variability in cognitive functioning. Psychol Aging, 24(4), 792–808. 10.1037/a0017798 [DOI] [PubMed] [Google Scholar]
  24. McCrea M, Iverson GL, McAllister TW, Hammeke TA, Powell MR, Barr WB, & Kelly JP (2009). An integrated review of recovery after mild traumatic brain injury (MTBI): Implications for clinical management. Clin Neuropsychol, 23(8), 1368–1390. 10.1080/13854040903074652 [DOI] [PubMed] [Google Scholar]
  25. Merritt VC, Clark AL, Crocker LD, Sorg SF, Werhane ML, Bondi MW, … Delano-Wood L. (2018). Repetitive mild traumatic brain injury in military veterans is associated with increased neuropsychological intra-individual variability. Neuropsychologia, 119, 340–348. 10.1016/j.neuropsychologia.2018.08.026 [DOI] [PubMed] [Google Scholar]
  26. Miller DR, Hayes JP, Lafleche G, Salat DH, & Verfaellie M. (2016). White matter abnormalities are associated with chronic postconcussion symptoms in blast-related mild traumatic brain injury: White Matter Abnormalities and PCS. Human Brain Mapping, 37(1), 220–229. 10.1002/hbm.23022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Moore BA, & Donders J. (2004). Predictors of invalid neuropsychological test performance after traumatic brain injury. Brain Inj, 18(10), 975–984. 10.1080/02699050410001672350 [DOI] [PubMed] [Google Scholar]
  28. Morey RA, Haswell CC, Selgrade ES, Massoglia D, Liu C, Weiner J, … McCarthy G. (2013). Effects of chronic mild traumatic brain injury on white matter integrity in Iraq and Afghanistan war veterans. Hum Brain Mapp, 34(11), 2986–2999. 10.1002/hbm.22117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Morgan EE, Woods SP, Delano-Wood L, Bondi MW, Grant I, & Neurobehavioral HIV Research Program Group. (2011). Intraindividual variability in HIV infection: Evidence for greater neurocognitive dispersion in older HIV seropositive adults. Neuropsychology, 25(5), 645–654. 10.1037/a0023792 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Nilsson M, Latt J, Stahlberg F, van Westen D, & Hagslatt H. (2012). The importance of axonal undulation in diffusion MR measurements: A Monte Carlo simulation study. NMR Biomed, 25(5), 795–805. 10.1002/nbm.1795 [DOI] [PubMed] [Google Scholar]
  31. Oouchi H, Yamada K, Sakai K, Kizu O, Kubota T, Ito H, & Nishimura T. (2007). Diffusion anisotropy measurement of brain white matter is affected by voxel size: Underestimation occurs in areas with crossing fibers. AJNR Am J Neuroradiol, 28(6), 1102–1106. 10.3174/ajnr.A0488 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Rabinowitz AR, & Arnett PA (2013). Intraindividual cognitive variability before and after sports-related concussion. Neuropsychology, 27(4), 481–490. 10.1037/a0033023 [DOI] [PubMed] [Google Scholar]
  33. Reese TG, Heid O, Weisskoff RM, & Wedeen VJ (2003). Reduction of eddy-current-induced distortion in diffusion MRI using a twice-refocused spin echo. Magn Reson Med, 49(1), 177–182. 10.1002/mrm.10308 [DOI] [PubMed] [Google Scholar]
  34. Rohling ML, Binder LM, Demakis GJ, Larrabee GJ, Ploetz DM, & Langhinrichsen-Rohling J. (2011). A meta-analysis of neuropsychological outcome after mild traumatic brain injury: Re-analyses and reconsiderations of Binder et al. (1997), Frencham et al. (2005), and Pertab et al. (2009). Clin Neuropsychol, 25(4), 608–623. 10.1080/13854046.2011.565076 [DOI] [PubMed] [Google Scholar]
  35. Sorg SF, Delano-Wood L, Luc N, Schiehser DM, Hanson KL, Nation DA, … Bondi MW (2014). White matter integrity in veterans with mild traumatic brain injury: Associations with executive function and loss of consciousness. J Head Trauma Rehabil, 29(1), 21–32. 10.1097/HTR.0b013e31828a1aa4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Sorg SF, Schiehser DM, Bondi MW, Luc N, Clark AL, Jacobson MW, … Delano-Wood L. (2016). White Matter Microstructural Compromise Is Associated With Cognition But Not Posttraumatic Stress Disorder Symptoms in Military Veterans With Traumatic Brain Injury. J Head Trauma Rehabil, 31(5), 297–308. 10.1097/HTR.0000000000000189 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Strauss E, Slick DJ, Levy-Bencheton J, Hunter M, MacDonald SW, & Hultsch DF (2002). Intraindividual variability as an indicator of malingering in head injury. Arch Clin Neuropsychol, 17(5), 423–444. [PubMed] [Google Scholar]
  38. Swick D, Honzel N, Larsen J, & Ashley V. (2013). Increased response variability as a marker of executive dysfunction in veterans with post-traumatic stress disorder. Neuropsychologia, 51(14), 3033–3040. 10.1016/j.neuropsychologia.2013.10.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Tanner-Eggen C, Balzer C, Perrig WJ, & Gutbrod K. (2015). The neuropsychological assessment of cognitive deficits considering measures of performance variability. Arch Clin Neuropsychol, 30(3), 217–227. 10.1093/arclin/acv008 [DOI] [PubMed] [Google Scholar]
  40. Tombaugh TN (1996). Test of memory malingering (TOMM). Toronto, Ontario, Canada: Multi-Health Systems, Inc. [Google Scholar]
  41. Traumatic Brain Injury Task Force. (2009). Report to the surgeon general: TBI task force. Http://www.healthquality.va.gov/guidelines/Rehab/mtbi/concussion_mtbi_full_1_0.pdf.
  42. Tuch DS (2004). Q-ball imaging. Magn Reson Med, 52(6), 1358–1372. 10.1002/mrm.20279 [DOI] [PubMed] [Google Scholar]
  43. Vanderploeg RD, Curtiss G, & Belanger HG (2005). Long-term neuropsychological outcomes following mild traumatic brain injury. J Int Neuropsychol Soc, 11(3), 228–236. 10.1017/S1355617705050289 [DOI] [PubMed] [Google Scholar]
  44. Vasterling JJ, & Dikmen S. (2012). Mild traumatic brain injury and posttraumatic stress disorder: Clinical and conceptual complexities. J Int Neuropsychol Soc, 18(3), 390–393. 10.1017/S1355617712000367 [DOI] [PubMed] [Google Scholar]
  45. Vanderploeg RD, Groer S, & Belanger HG (2012). Initial developmental process of a VA semistructured clinical interview for TBI identification. Journal of Rehabilitation Research and Development, 49(4), 545–556. 10.1682/jrrd.2011.04.0069 [DOI] [PubMed] [Google Scholar]
  46. Verfaellie M, Lafleche G, Spiro A, & Bousquet K. (2014). Neuropsychological outcomes in OEF/OIF veterans with self-report of blast exposure: Associations with mental health, but not MTBI. Neuropsychology, 28(3), 337–346. 10.1037/neu0000027 [DOI] [PubMed] [Google Scholar]
  47. Weathers F, Huska J, & Keane T. (1991). The PTSD checklist military version (PCL-M). Boston, MA: National Center for PTSD, 42. [Google Scholar]
  48. Yeh FC, Badre D, & Verstynen T. (2016). Connectometry: A statistical approach harnessing the analytical potential of the local connectome. Neuroimage, 125, 162–171. 10.1016/j.neuroimage.2015.10.053 [DOI] [PubMed] [Google Scholar]
  49. Yeh F-C, Panesar S, Barrios J, Fernandes D, Abhinav K, Meola A, & Fernandez-Miranda JC (2019). Automatic Removal of False Connections in Diffusion MRI Tractography Using Topology-Informed Pruning (TIP). Neurotherapeutics: The Journal of the American Society for Experimental NeuroTherapeutics, 16(1), 52–58. 10.1007/s13311-018-0663-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Yeh FC, Tang PF, & Tseng WY (2013). Diffusion MRI connectometry automatically reveals affected fiber pathways in individuals with chronic stroke. Neuroimage Clin, 2, 912–921. 10.1016/j.nicl.2013.06.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Yeh FC, & Tseng WY (2011). NTU-90: A high angular resolution brain atlas constructed by q-space diffeomorphic reconstruction. Neuroimage, 58(1), 91–99. 10.1016/j.neuroimage.2011.06.021 [DOI] [PubMed] [Google Scholar]
  52. Yeh FC, Verstynen TD, Wang Y, Fernandez-Miranda JC, & Tseng WY (2013). Deterministic diffusion fiber tracking improved by quantitative anisotropy. PLoS One, 8(11), e80713. 10.1371/journal.pone.0080713 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Yurgil KA, Barkauskas DA, Vasterling JJ, Nievergelt CM, Larson GE, Schork NJ, … Marine Resiliency Study T. (2014). Association between traumatic brain injury and risk of posttraumatic stress disorder in active-duty Marines. JAMA Psychiatry, 71(2), 149–157. 10.1001/jamapsychiatry.2013.3080 [DOI] [PubMed] [Google Scholar]

RESOURCES