A quantitative meta-analysis of neurocognitive functioning in posttraumatic stress disorder

Abstract

Posttraumatic stress disorder (PTSD) is associated with regional alterations in brain structure and function that are hypothesized to contribute to symptoms and cognitive deficits associated with the disorder. We present here the first systematic meta-analysis of neurocognitive outcomes associated with PTSD to examine a broad range of cognitive domains and describe the profile of cognitive deficits, as well as modifying clinical factors and study characteristics. This report is based on data from 60 studies totaling 4,108 participants, including 1,779with PTSD, 1,446 trauma-exposed comparison participants, and 895 healthy comparison participants without trauma exposure. Effect size estimates were calculated using a mixed-effects meta-analysis for nine cognitive domains: attention/working memory, executive functions, verbal learning, verbal memory, visual learning, visual memory, language, speed of information processing, and visuospatial abilities. Analyses revealed significant neurocognitive effects associated with PTSD, although these ranged widely in magnitude, with the largest effect sizes in verbal learning (d =−.62), speed of information processing (d =−.59), attention/working memory (d =−.50), and verbal memory (d =−.46). Effect size estimates were significantly larger in treatment-seeking than community samples and in studies that did not exclude participants with attention-deficit hyperactivity disorder, and effect sizes were affected by between-group IQ discrepancies and the gender composition of the PTSD groups. Our findings indicate that consideration of neuropsychological functioning in attention, verbal memory, and speed of information processing may have important implications for the effective clinical management of persons with PTSD. Results are further discussed in the context of cognitive models of PTSD and the limitations of this literature.

Posttraumatic stress disorder (PTSD) is a common, often debilitating psychiatric disorder that is triggered by an extreme stressor involving threat of death or serious injury. Characteristic symptoms of PTSD include re-experiencing of traumatic memories through intrusive thoughts or nightmares, avoidance of trauma reminders, distress and physiological reactivity in response to reminders of trauma, emotional numbing, dysphoria, and hyperarousal (American Psychiatric Association, 2013). PTSD affects approximately 8% of the general population (Kessler, Sonnega, Bromet, Hughes, & Nelson, 1995), with higher prevalence rates reported in certain subgroups, such as veterans exposed to combat (Hoge, Auchterlonie, & Milliken, 2006; Seal, Bertenthal, Miner, Sen, & Marmar, 2007).

Most current theories of PTSD agree that abnormalities in memory are primary contributors to a number of symptoms (American Psychiatric Association, 2013; Brewin, Gregory, Lipton, & Burgess, 2010; McNally, 2006). For example, individuals with PTSD experience frequent involuntary intrusions of vivid, trauma-related memories through flashbacks and nightmares and, somewhat paradoxically, have difficulty voluntarily recalling details of traumatic events (Brewin, 2007). Similarly, it has been proposed that multiple PTSD symptoms can be linked to dysfunction in attentional processing, including attention bias towards threat, persistent enhancement of attention to salient but extraneous environmental cues (i.e., hypervigilance), and problems with attentional control over trauma-related thoughts (Litz et al., 1996).

In addition to trauma-specific disruptions in memory and attention, individuals exposed to chronic stress (e.g., prisoners of war) and those with PTSD have long been noted to complain of persistent problems in concentration and everyday memory (Archibald & Tuddenham, 1965; Bleich, Siegel, Garb, & Lerer, 1986; Burstein, 1985; Roca & Freeman, 2001). Moreover, a substantial literature has amassed over the past 25 years showing performance deficits on neuropsychological measures of attention, working memory, episodic memory, speed of information processing, and executive functioning in individuals with PTSD (e.g., Aupperle, Melrose, Stein, & Paulus, 2012; Bremner et al., 1993; Dalton, Pederson, & Ryan, 1989; Uddo, Vasterling, Brailey, & Sutker, 1993; Vasterling & Brewin, 2005; Yehuda, Golier, Tischler, Stavitsky, & Harvey, 2005). For example, Vasterling and colleagues (2002) found significant deficits in sustained attention, working memory, and immediate verbal memory in Vietnam veterans with PTSD, even after adjusting for premorbid intellectual functioning and substance abuse. Similar findings have been reported in non-veteran samples with PTSD (e.g., Bremner, Vermetten, Afzal, & Vythilingam, 2004; Jenkins, Langlais, Delis, & Cohen, 1998; Stein, Kennedy, & Twamley, 2002). Importantly, these cognitive deficits have been shown to negatively affect treatment and functional outcomes in PTSD. For example, Geuze and colleagues (2009) showed that episodic memory performance uniquely predicted reports of both occupational and social functioning in veterans with PTSD. Furthermore, prior work has shown that greater efficiency of inhibitory control and performance on measures of verbal memory predict response to cognitive-behavioral therapy in individuals with PTSD (Falconer, Allen, Felmingham, Williams, & Bryant, 2013; Wild & Gur, 2008).

However, despite the considerable number of studies examining neurocognitive deficits associated with PTSD, consensus regarding the pattern and magnitude of these effects remains elusive, and some researchers question the link between PTSD and cognitive dysfunction (e.g., Crowell, Kieffer, Siders, & Vanderploeg, 2002; Danckwerts & Leathem, 2003; Demakis, Gervais, & Rohling, 2008). Clarifying the nature and extent of neurocognitive deficits in PTSD is important for understanding the correlates and mechanisms of PTSD, identifying factors that might impede treatment and worsen functional outcomes, and aiding in clinical neuropsychological profile interpretation. We present here the first systematic meta-analysis of neurocognitive outcomes associated with PTSD to examine a broad range of cognitive domains and describe modifying factors, features of the trauma that predict deficits, and the profile of cognitive deficits.

Neurocircuitry of PTSD

PTSD symptoms have been hypothesized to reflect structural and functional alterations in a number of interacting brain regions, including components of the limbic system (i.e., the amygdala, hippocampus, and cingulate cortex) and dorsolateral and ventromedial regions of the prefrontal cortex (PFC) (Bremner, Randall, Scott, Bronen, et al., 1995; Karl et al., 2006; Liberzon & Sripada, 2008; Morey et al., 2012; Rauch, Shin, & Phelps, 2006). The amygdala, hippocampus, cingulate cortex, and prefrontal cortex are critically involved in emotion processing and emotional memory formation, including the acquisition of fear and the establishment of emotional context and valence for memories (e.g., Etkin & Wager, 2007). They also play important roles in emotionally neutral neurocognitive performance. For example, the hippocampus is integral for encoding and storage of episodic memory (e.g., conscious memory for events), while the medial prefrontal cortex and anterior portions of the cingulate cortex are thought to be involved in both affective and cognitive control. It has also been proposed that the amygdala and dorsal anterior cingulate cortex contribute to processing of salient or ambiguous environmental stimuli, which may help direct or allocate attentional resources (Pessoa & Adolphs, 2010). Moreover, lateral and orbital portions of the prefrontal cortex play vital roles in attention, working memory, cognitive control, and decision making.

The integrity and function of these brain regions in PTSD have been primarily investigated with structural magnetic resonance imaging (MRI), functional MRI (fMRI), and positron emission tomography (PET). We briefly summarize the relevant findings below to provide a framework for understanding potential neural correlates of neurocognitive findings in PTSD, although a full review of this work is beyond the scope of this paper (see excellent reviews by Bremner, Elzinga, Schmahl, & Vermetten, 2008; Brown & Morey, 2012; Patel, Spreng, Shin, & Girard, 2012; Pitman et al., 2012).

Accumulating data from structural MRI studies have shown decreased volume in the hippocampus, anterior cingulate cortex, and amygdala in adults with PTSD (D. W. Hedges & Woon, 2010; Karl et al., 2006; Kitayama, Vaccarino, Kutner, Weiss, & Bremner, 2005; Kühn & Gallinat, 2013; Morey et al., 2012), as well as decreased cortical thickness in frontal and temporal cortex (Geuze et al., 2008; Lindemer, Salat, Leritz, McGlinchey, & Milberg, 2013; Woodward, Schaer, Kaloupek, Cediel, & Eliez, 2009). Reductions in N-acetylaspartate (NAA), a marker of neuronal integrity, have also been reported in both the hippocampus and anterior cingulate (Ham et al., 2007; Mahmutyazicioğlu et al., 2005; Schuff et al., 2001, 2008). It should be noted, however, that hippocampal volumetric changes have been proposed as both a pre-trauma vulnerability factor for PTSD (e.g., Gilbertson et al., 2002) and a consequence of the disorder (e.g., Bremner et al., 2008).

Although current functional neurocircuitry models of PTSD vary, most propose a hyperactive amygdala in response to threat or emotionally arousing stimuli combined with hypoactive regions of the ventromedial prefrontal cortex (vmPFC), which provide inadequate “top-down” regulation of amygdala activity (Koenigs & Grafman, 2009; Liberzon & Sripada, 2008; Patel et al., 2012; Rauch et al., 2006). Models that incorporate cognitive functioning suggest that, in response to cognitive demands, individuals with PTSD evidence hypoactivation of regions involved in attention, working memory, encoding, and executive processing, including dorsal prefrontal, inferior frontal, superior parietal, and orbitofrontal regions (Aupperle, Allard, et al., 2012; Bremner et al., 2008; Brown & Morey, 2012; Bryant et al., 2005; Falconer, Bryant, et al., 2008; Moores et al., 2008; Morey et al., 2009; Pannu Hayes, Labar, Petty, McCarthy, & Morey, 2009; Rauch et al., 2006).

In sum, results from functional and structural neuroimaging research in PTSD suggest dysfunction in neural networks comprised of prefrontal cortex, cingulate cortex, and limbic regions, which have the potential to impact emotion processing, cognitive functioning, and their interaction.

Neurocognitive Functioning in PTSD

Despite increased understanding of the alterations in neural circuitry associated with PTSD and the potential effects such alterations could have on behavior, the motivations for studying neurocognitive functioning in PTSD have, for the most part, not been driven by an integrated theory of disorder-specific cognitive dysfunction. Many studies have been driven by early preclinical research, which discovered that severe or prolonged stress exposure in rodents and primates exerted adverse effects on the structure and function of the hippocampus (Luine, Villegas, Martinez, & McEwen, 1994; McEwen & Sapolsky, 1995; Sapolsky, Uno, Rebert, & Finch, 1990). These results helped generate appealing hypotheses to investigate in studies of PTSD in humans (e.g., Bremner, Randall, Scott, Bronen, et al., 1995; Sapolsky, 2000), and initial investigations of neurocognitive functioning in PTSD primarily focused on episodic memory effects that were ostensibly mediated by the hippocampus. Although some studies reported robust effects of PTSD on memory functioning, including associations between reductions in hippocampal volume and episodic memory difficulties (Bremner et al., 1993; Bremner, Randall, Scott, Bronen, et al., 1995; Tischler et al., 2006; Vythilingam et al., 2005), a number of studies have failed to replicate these findings (Bremner et al., 1997; Lindauer, Olff, van Meijel, Carlier, & Gersons, 2006; Neylan et al., 2004; Stein, Koverola, Hanna, Torchia, & McClarty, 1997; Woodward, Kaloupek, et al., 2009), which raises questions about applying a hippocampal conceptualization to memory deficits in PTSD (Woodward, Kaloupek, et al., 2009).

Further investigations have refined the understanding of PTSD-associated memory deficits by applying models from cognitive psychology that emphasize the stages of processing at which episodic memory dysfunction can occur, including encoding, storage, and retrieval. Several studies have shown that while individuals with PTSD show deficits in initial learning, minimal forgetting occurs over time, and individuals typically recall additional information to be remembered when a recognition trial is administered, which minimizes demands on retrieval (Cohen et al., 2013; Jelinek et al., 2006; Jenkins et al., 1998; Johnsen, Kanagaratnam, & Asbjørnsen, 2008; Yehuda, Golier, Halligan, & Harvey, 2004). This pattern of results strongly suggests that episodic memory deficits in PTSD are associated with problems in strategic encoding and retrieval of information (Golier, Harvey, Legge, & Yehuda, 2006; Samuelson et al., 2006; Twamley et al., 2009), indicating that prefrontal systems may also contribute to memory dysfunction in PTSD (Brewin, Kleiner, Vasterling, & Field, 2007).

At the same time, other neurocognitive conceptualizations of PTSD have proposed that dysfunctional arousal and heightened noradrenergic activity may result in reduced cognitive processing resources and consequent problems in attention, episodic memory encoding, and executive functions in PTSD (e.g., Falconer, Felmingham, et al., 2008; Vasterling, Brailey, Constans, & Sutker, 1998). For example, heightened noradrenergic sensitivity, bias to threat, or hyperarousal may divert prefrontally-mediated attentional resources to extraneous stimuli, which may disrupt goal-based attention and negatively affect encoding and retention of verbal information, as well as other cognitive processes moderated by prefrontal cortical networks, such as sustaining focused attention over time (Etkin, Gyurak, & O’Hara, 2013; Eysenck, Derakshan, Santos, & Calvo, 2007). To this end, Vasterling et al. (1998, 2002) found PTSD-associated deficits in sustained attention, mental manipulation of information, and immediate memory, but not in selective attention or forgetting of information over time. They attributed these deficits to prefrontal cortex dysfunction potentially associated with arousal dysregulation.

Other investigators have noted that the neural circuitry affected by PTSD, which (as described above) is prominently involved in emotion processing and regulation, significantly overlaps with neural circuitry involved in certain aspects of neuropsychological functioning (e.g., Aupperle, Allard, et al., 2012; Koenen et al., 2001). Consistent with advances in the neuroscience of PTSD that have proposed a larger pathophysiological role for the prefrontal cortex, emerging work in PTSD has highlighted additional difficulties in executive functioning (Aupperle, Melrose, et al., 2012; Leskin & White, 2007) and speed of information processing (Aupperle, Allard, et al., 2012; Brandes et al., 2002; Cohen et al., 2013; Twamley et al., 2009), both of which rely on the integrity of prefrontal cortical networks and efficient connectivity between frontal regions and other large-scale brain networks (Bressler & Menon, 2010; Nakahachi et al., 2008, 2010). Although evidence concerning impairment in strategic planning, conceptual flexibility, and set-shifting aspects of executive functioning in PTSD remain unclear (Crowell et al., 2002; Jenkins, Langlais, Delis, & Cohen, 2000; Leskin & White, 2007; Stein et al., 2002; Twamley et al., 2009), converging evidence demonstrates that PTSD is associated with inhibitory dysfunction across a number of different measures, suggesting difficulty with inhibiting inappropriate or automatic responses (Casada & Roache, 2005; Jenkins et al., 2000; Leskin & White, 2007; Shucard, McCabe, & Szymanski, 2008; Vasterling et al., 1998). Such results have been used to support a model of generalized dysfunction in inhibitory control in PTSD, which could help explain difficulties in regulation of both neuropsychological and emotional processes (Aupperle, Melrose, et al., 2012; Johnsen & Asbjørnsen, 2009; Vasterling et al., 1998).

Results, however, have not been unequivocal, and other reports have questioned the presence or magnitude of cognitive impairments in PTSD (Crowell et al., 2002; Elsesser & Sartory, 2007; Gurvits et al., 1996; Neylan et al., 2004; Pederson et al., 2004; Twamley, Hami, & Stein, 2004). Thus, controversy endures regarding whether PTSD is associated with generalized cognitive dysfunction beyond impaired trauma-specific and episodic memory (Danckwerts & Leathem, 2003; Gilbertson, Gurvits, Lasko, Orr, & Pitman, 2001; Horner & Hamner, 2002; Parslow & Jorm, 2007; Wisdom et al., 2013).

The discrepancy in results may be due to methodological variance among studies, including differences in trauma type, patient characteristics, and exclusion criteria. For example, as mentioned above, studies have varied in their rationales for studying neurocognitive functioning in PTSD. As a result, studies have typically only assessed a circumscribed range of neurocognitive functions, often with varying tests, which can lead to ambiguity in determining the effects of PTSD on neurocognitive functioning when one examines results across this literature.

In addition, the criteria for assigning a PTSD diagnosis have varied across studies, spanning from chart diagnosis of PTSD to consensus diagnosis using multiple structured psychiatric interviews with documented sensitivity and specificity. Studies with less standardized criteria for diagnosis may evidence less diagnostic precision, although the effect of this imprecision on conclusions about neurocognitive functioning is unclear. Furthermore, studies with various index traumas, including combat, intimate partner violence, community violence, natural disasters, terrorism, state persecution, sexual trauma, and forced displacement, are included in this literature. Although it is unclear whether the symptom profile of PTSD may vary by trauma type (Chung & Breslau, 2008), neurocognitive functioning could be affected by the duration or severity of the trauma, as well as the specific characteristics of the population sampled. In particular, previous reviews have found that studies of war-related trauma show larger negative effects of PTSD on cognitive functioning (Polak, Witteveen, Reitsma, & Olff, 2012; Qureshi et al., 2011). Since a majority of neurocognitive studies of PTSD have been conducted in male veterans, it is important to show that these results are generalizable to populations with different index traumas and clinical profiles.

One possible source of variability in neurocognitive findings is confounding psychiatric, substance abuse, and neurologic (e.g., traumatic brain injury) comorbidities (Danckwerts & Leathem, 2003; Horner & Hamner, 2002; Isaac, Cushway, & Jones, 2006). Head injuries, especially those involving a loss of consciousness, may be particularly important because a majority of studies on neurocognitive functioning in PTSD have been conducted in combat veterans, who have a relatively high prevalence of head injuries. Most of these individuals will have experienced a mild traumatic brain injury (mTBI; e.g., loss of consciousness less than 30 minutes, post-traumatic amnesia less than 24 hours, Glasgow Coma Scale score of 13–15), which typically has been shown to have minimal or subtle cognitive effects 9–12 months post-injury (Boyle et al., 2014; Carroll et al., 2004; Rohling, Larrabee, & Millis, 2012; Soble, Spanierman, & Fitzgerald Smith, 2013; Vasterling et al., 2012; c.f. Bigler et al., 2013). However, individuals with moderate or severe head injuries or with a history of multiple head injuries (e.g., Belanger, Spiegel, & Vanderploeg, 2010) can evidence persistent deficits in attention, memory, executive functions, and speed of information processing (Dikmen, Machamer, & Temkin, 2009). Inclusion of such individuals could contaminate findings in studies examining cognition in PTSD.

It has also been proposed that psychiatric comorbidity may account for a significant proportion of the cognitive deficits typically reported in PTSD samples (e.g., Barrett, Green, Morris, Giles, & Croft, 1996; Gil, Calev, Greenberg, Kugelmass, & Lerer, 1990). In particular, symptoms of depression may explain certain cognitive deficits in individuals with PTSD (e.g., Brandes et al., 2002; Burriss, Ayers, Ginsberg, & Powell, 2008; Johnsen et al., 2008; Olff, Polak, Witteveen, & Denys, 2014). Major depressive disorderis associated with a profile of mild deficits in problem solving, inhibition, sustained attention, attentional switching, and episodic memory, with a particular deficit in visual memory in younger outpatients with PTSD (e.g., Fossati, Amar, Raoux, Ergis, & Allilaire, 1999; Lee, Hermens, Porter, & Redoblado-Hodge, 2012; Porter, Gallagher, Thompson, & Young, 2003; Snyder, 2013; Zakzanis, Leach, & Kaplan, 1998). Thus, it is possible that the neurocognitive deficits observed in studies of PTSD may simply reflect the established comorbidity of PTSD with depression.

High levels of alcohol and substance use in samples of individuals with PTSD could also impact cognitive findings. Almost half of individuals with PTSD will qualify for a diagnosis of an alcohol or substance use disorder in their lifetime (Kessler et al., 1995; Scherrer et al., 2008). A substantial literature indicates that chronic use of alcohol and substances such as cocaine, amphetamines, opiates, and benzodiazepines have detrimental effects on memory, attention, processing speed, visuospatial abilities, set shifting, and abstraction and conceptualization, even after months to years of abstinence (e.g., Barker, Greenwood, Jackson, & Crowe, 2005; Bartzokis et al., 2002; Grant & Rourke, 2009; Jovanovski, Erb, & Zakzanis, 2005; Pluck et al., 2012; Rourke & Grant, 1999; Scott et al., 2007). Most studies do account for these confounds by either excluding participants who meet current criteria for alcohol or substance use disorders or attempting to partial out their influence in analyses. However, these methods have not satisfied all critics (Horner & Hamner, 2002), and the question of whether alcohol and substance use comorbidities contribute to cognitive deficits in PTSD remains unclear.

Studies have also drawn attention to pre-trauma factors that might affect neurocognitive functioning, including attention-deficit/hyperactivity disorder (ADHD) and pre-trauma intelligence estimates. Because of the high comorbidity rates of PTSD and ADHD (Adler, Kunz, Chua, Rotrosen, & Resnick, 2004; Gurvits et al., 2006; Harrington et al., 2012), it is possible that unrecognized ADHD comorbidity could contribute to neurocognitive findings reported in PTSD; however, this has rarely been examined. Adult ADHD has been reported to have a profile of cognitive deficits in attention, episodic memory encoding, and executive function (e.g., Hervey, Epstein, & Curry, 2004), raising the question of whether ADHD could explain some of the neurocognitive findings associated with PTSD. In addition, although individuals with PTSD have average intelligence estimates overall, they nonetheless frequently have lower levels of estimated intelligence than comparison groups (e.g., Breslau, Lucia, & Alvarado, 2006; Koenen, Moffitt, Poulton, Martin, & Caspi, 2007; Macklin et al., 1998). Intelligence estimates have robust associations with neurocognitive performance. Therefore, it is possible that limited premorbid intellectual resources may be partially responsible for cognitive deficits in individuals with PTSD (Bustamante, Mellman, David, & Fins, 2001; Gilbertson et al., 2006).

Other sample characteristics, such as whether an index trauma occurred during childhood or adulthood, could also impact neurocognitive functioning. It is possible that individuals with PTSD who experienced an index trauma during a critical period of brain development would show a divergent pattern of brain dysfunction compared to those who were traumatized as an adult, when brain maturation has slowed significantly. Also, some studies have compared individuals with PTSD to non-traumatized populations while others have used control groups with high stress exposure and subclinical PTSD symptoms (Isaac et al., 2006; Knight & Taft, 2004; Yehuda, Stavitsky, Tischler, Golier, & Harvey, 2005). Exposure to trauma may itself be associated with changes in brain functioning and cognitive performance (e.g., Vasterling et al., 2006). Thus, studies that use a trauma-exposed comparison group may show smaller differences in neurocognitive functioning compared to those that use a healthy, trauma unexposed comparison group.

In sum, consensus regarding the neurocognitive effects of PTSD and the impact of other potential explanatory variables remains elusive due to inconsistencies in the literature. Our ability to draw clinically meaningful conclusions from the existing literature is limited by the absence of a quantitative determination of the nature and extent of cognitive deficits in individuals with PTSD based on results from multiple independent studies.

Meta-analysis Aims

Meta-analysis is a useful method to estimate effect sizes across a large literature of independent studies, investigate associations between constructs, and quantitatively examine the methodological variance among studies (Lipsey & Wilson, 2001). Although a number of useful qualitative reviews have addressed cognition in PTSD (e.g., Isaac et al., 2006; McNally, 2006; Qureshi et al., 2011; Vasterling & Brewin, 2005), meta-analysis offers a number of advantages in examining the neurocognitive effects associated with PTSD. First, meta-analysis helps reduce the effects of varying statistical power across studies, which is problematic in this literature (Brewin et al., 2007). Instead of interpreting effects from each study based on statistical significance, which is highly dependent on sample size, meta-analysis provides data about the magnitude of an effect that is sensitive across studies with varying statistical power (Lipsey & Wilson, 2001). Second, meta-analysis helps to deal with difficulties in interpretation created by the inconsistency in the neuropsychological tests used across a literature. By collapsing across measures, meta-analysis may reveal construct-level effects that are typically constrained by one’s ability to interpret and evaluate individual test findings. Lastly, meta-analysis offers the advantage of standardizing neurocognitive domain classification for individual tests, which reduces the uncertainty caused by the use of different descriptors for the same or similar tests across a literature.

To date, meta-analyses have examined memory and executive functioning deficits in individuals with PTSD. Brewin and colleagues (2007) examined memory performance in PTSD across 27 studies and reported significant differences between PTSD and non-PTSD participants, finding small and moderate effect size differences for visual and verbal memory, respectively. Johnsen and Asbjørnsen (2008) largely replicated these findings in a meta-analysis of 28 studies of verbal memory impairment in PTSD, finding that individuals with PTSD had greater verbal learning deficits than healthy controls; there were less pronounced differences between individuals with PTSD and those exposed to trauma but without PTSD. Polak and colleagues (2012) examined performance on measures of executive functioning in PTSD, finding small-to-moderate effect sizes and larger detrimental effects in samples with combat-related trauma. However, these authors excluded a large number of neuropsychological test results, providing a limited picture of executive functioning performance in PTSD.

Although previous meta-analyses and qualitative reviews have yielded valuable insights into cognitive functioning in PTSD, they have not examined individuals’ performances across a broad range of neuropsychological domains, restricting comparisons among cognitive ability domains and providing limited insight into the functional brain systems potentially affected in PTSD. For example, despite accumulating evidence for the relevance of attention and processing speed in PTSD (Gilbertson et al., 2001; Samuelson et al., 2006; Twamley et al., 2009; Vasterling et al., 2002; Woodward, Kaloupek, et al., 2009), potential deficits in these cognitive domains have not been examined meta-analytically. Moreover, prior meta-analyses have not examined specific variables that might contribute to the variability of findings in the literature, including the treatment status of subjects, psychiatric comorbidity, between-group differences in IQ, and demographic variables such as gender and age. Results from comprehensive meta-analyses could enhance our understanding of factors that contribute to neurocognitive outcomes in PTSD and help to identify potential explanatory variables of interest, such as clinical (e.g., treatment-seeking status), demographic (e.g., gender), and methodological (e.g., exclusion criteria) factors. Such results could provide an explanation for the variability in effect size estimates across studies. Moreover, specification of neurocognitive performance patterns with known brain-behavior correlates could help bolster or weaken support for current cognitive and neural circuitry models of PTSD.

In this study, we aimed to use meta-analytic techniques to examine the profile and magnitude of effect sizes of cognitive deficits associated with PTSD across several functional domains. We also examined aspects of study design and subject characteristics that influence cognitive dysfunction in PTSD.

Methods

Studies and Variables

We began by identifying an a priori set of study inclusion criteria to focus our analysis on informative studies, including reports that: (1) assessed human adults aged 18 years and older; (2) used specific criteria to classify study subjects as to the presence or absence of PTSD; (3) included a comparison group of healthy subjects with no history of PTSD (if available) or other neuropsychiatric disorder; (4) reported outcome measures that included at least one standardized neuropsychological test; (5) assessed neurocognitive functioning after more than one month following traumatization; (6) studied subjects with current (rather than past) PTSD; and (7) provided sufficient information about their neuropsychological results to calculate effect sizes. These criteria were intentionally liberal to be inclusive and provide a more representative review of the neurocognitive correlates of PTSD.

Preliminary literature searches using the keywords PTSD or traumatic stress paired with cognition, cognitive, neuropsychological, or domain-specific keywords (i.e., memory, attention, concentration, working memory, executive function, inhibition, planning, shifting, switching, verbal fluency, language, speed of information processing, processing speed, psychomotor, visual, visuospatial) were independently conducted through several online databases, including PubMed, PsychInfo, and ISI Web of Science. Any article published in English prior to March 2014 was considered eligible. All articles identified as potentially eligible were reviewed in detail to ensure that the criteria for inclusion (specified above) were met. We also reviewed the reference list for each study to identify omissions from our review. Studies that did not include a control group (e.g., Dalton et al., 1989) were excluded. Studies published by the same group of authors were carefully reviewed to minimize the inclusion of overlapping data from a single participant cohort. For example, three studies appeared to be drawn from the same Centers for Disease Control database and likely had significant overlap in measures and participants (Barrett et al., 1996; Crowell et al., 2002; Zalewski, Thompson, & Gottesman, 1994). Although the study by Barrett and colleagues (1996) had the largest sample, the authors used a lifetime rather than a current PTSD diagnosis as their study entry criterion. Zalewski et al. (1994) did not report data sufficient to generate effect size estimates for their group with a current PTSD diagnosis, and thus Crowell et al. (2002) was included. Five reports did not provide enough information to calculate effect sizes (Burriss et al., 2008; Jenkins et al., 1998; Leskin & White, 2007; Veltmeyer et al., 2005; Wessa, Jatzko, & Flor, 2006) and were not included in the meta-analysis.

When studies included more than one potential control group (e.g., trauma exposed and unexposed) and had independent data available for each control group on the neuropsychological tests, we used data from both groups. Also, we included both PTSD samples from Hart et al. (2008) and both PTSD samples from Samuelson et al. (2006), as both studies presented one PTSD sample with psychiatric comorbidities and one without. Studies that included symptom provocation or trauma recall in the same session as the administration of neuropsychological tests were included only if neuropsychological testing occurred before any potential symptom exposure.

A total of 60studies with 4,108 participants, including 1,779 participants with PTSD, 1,446 trauma-exposed comparison participants, and 895 healthy comparison participants without trauma exposure, were deemed eligible for inclusion. The following information was extracted from each study: (1) participant demographic variables (i.e., mean age, mean years of education, and gender proportion of sample); (2) PTSD and trauma exposure characteristics (i.e., type of index trauma in the PTSD group, type of control group, severity of PTSD as assessed by the Clinician Administered PTSD Scale [CAPS], duration of PTSD, PTSD diagnostic criteria [whether studies used a structured diagnostic interview, a self-report instrument, or a chart diagnosis, as well as the specific scoring criteria and DSM version, if available], childhood versus adult trauma exposure); (3) sample characteristics (i.e., comorbid substance use and alcohol use disorders, proportion of sample diagnosed with depression, treatment-seeking status of the PTSD group, difference in IQ estimates between groups [calculated as a Cohen’s d effect size], administration of neuropsychological symptom validity testing); (4) study inclusion/exclusion criteria (i.e., regarding attention deficit/hyperactivity disorder [ADHD], traumatic brain injury, psychiatric comorbidity, and exclusion or restriction of psychotropic medication use); (5) sample size; and (6) summary statistics for the calculation of effect sizes. Studies that did not specify ADHD exclusion criteria were presumed to have allowed them in the PTSD group. Similarly, studies that did not specify medication exclusion criteria were presumed to have allowed psychotropic medications, and studies were classified as excluding psychotropic medications if participants were designated as drug naïve or if participants underwent a medication abstinence period of two weeks or more before the cognitive assessment. Data for PTSD duration, symptom validity testing, and childhood versus adult trauma exposure were not analyzed because of insufficient data.

Effect Size Calculation

For each neuropsychological test that was administered in these 60 studies, an effect size and its variance were calculated. The effect size used in this meta-analysis was the standardized mean difference statistic (d). When possible, this statistic was calculated as d = (M_e − M_c)/S_p, where M_e and M_c are the mean raw scores on a neuropsychological test for the PTSD and comparison groups, respectively, and S_p is the pooled within-group standard deviation. For studies in which these data were not reported, standardized mean difference effect sizes were derived from t-values based on independent t-tests or F-ratios from a two-group one-way analysis of variance (Shadish, Robinson, & Lu, 1999). We applied Hedges and Olkin’s (1985) correction for small sample bias to all effect sizes. The variance for each d value was then calculated and used to determine a weighting factor for the unbiased effect size.

We coded 530 effect sizes from the 60 studies, with a range of 1 to 19 effect sizes and a mean of 9.71 (SD = 4.41) per study. When studies offered results from multiple neuropsychological tests, the battery was independently reviewed by the raters (JCS, KMW), who classified the tests into domains based on evidence of construct validity (see Table 1). In the event of disagreement, the raters determined the domains for each test by consensus with the assistance of a third rater (BCS). These domains were: (1) attention/working memory (e.g., Wechsler Adult Intelligence Scale, 3^rd Edition [WAIS-III] Digit Span, Continuous Performance Test); (2) executive functions (e.g., Wisconsin Card Sorting Test, Stroop Color-Word interference condition); (3) verbal learning (e.g., California Verbal Learning Test [CVLT] trials 1–5, Wechsler Memory Scale-III[WMS-III] Logical Memory I); (4) visual learning (e.g., WMS-III Visual Reproduction I); (5) verbal memory (e.g., CVLT Delayed Recall, WMS-III Logical Memory II); (6) visual memory (e.g., Rey Complex Figure Delayed Recall, WMS-III Visual Reproduction II); (7) psychomotor (e.g., Grooved Pegboard); (8) language (e.g., Verbal Fluency, Boston Naming Test); (9) speed of information processing (e.g., WAIS-III Digit Symbol, Trail making Test, Part A); and (10) visuospatial functioning (e.g., Rey Complex Figure Copy). Note that “learning” as identified here is synonymous with “immediate recall,” while “memory” is synonymous with “delayed recall.” Only 3 studies reported tests that were classified within the psychomotor domain; therefore, this domain was excluded from analysis. If multiple subtests assessing the same cognitive construct were reported (e.g., CVLT Delayed Free Recall and Cued Recall), the subtest with the best evidence of construct validity (based on consensus) was chosen for inclusion (e.g., CVLT Delayed Free Recall). Table 1 lists the tests that were included in each cognitive domain, their frequency, references that provide evidence of their validity for assessing that particular cognitive domain, and reliability. Measures for which low scores indicate better performance were adjusted to assure that a negative d indicated that the PTSD group performed worse than the comparison group.

Table 1.

Neuropsychological tests analyzed in the meta-analysis, by domain, with validity and reliability information (where available).

Neuropsychological Domain
Attention/Working Memory
Test	k	%	Validity Evidence	Reliability	Reliability Source
WAIS/WAIS-R/WAIS-III or WMS-R/III Digit Span	21	16.0	Boone et al. (1998); Burton et al. (2002; 2003)	Cronbach’s α = .90	Wechsler (1997)
WAIS/WAIS-R/WAIS-III or WMS-R/III Digit Span Forward	9	6.9	Wechsler (1997)	Cronbach’s α = .87	Colom et al. (2008)
WAIS/WAIS-R/WAIS-III or WMS-R/III Digit Span Backward	9	6.9	Wechsler (1997)	Cronbach’s α = .90	Wechsler (1997)
CVLT Trial 1	9	6.9	Donders (2008a, 2008b)	Test-retest = .58	Woods et al. (2006)
PASAT	8	6.1	O’Donnell et al. (1994); Larrabee & Curtiss (1995)	Cronbach’s α = .90	Crawford, Obonsawin, & Allan (1998)
WAIS-R/WAIS-III Arithmetic	5	3.8	Burton et al. (2002, 2003)	Test-retest =.89	Wechsler (1997)
Continuous Performance Test-Commissions	5	3.8	Egeland & Kovalik-Gran (2010)	Split-half=.83	Conners (2000)
Continuous Performance Test-Omissions	5	3.8	Barkley et al. (2001); Egeland & Kovalik-Gran (2010)	Split-half=.94	Conners (2000)
WAIS-III/WMS-III Letter-Number Sequencing	5	3.8	Burton et al. (2002, 2003)	Split-half=.82	Wechsler (1997)
Digit Vigilance Test-Time	4	3.1	Grant et al. (1987); Kelland & Lewis (1996)	Test-retest =.70	Kelland & Lewis (1996)
Digit Vigilance Test-Errors	3	2.3	Kelland & Lewis (1996)	Test-retest =.66	Kelland & Lewis (1996)
Picture Word Memory Test-Trial 1 (Verbal)	3	2.3	Jelinek et al. (2006)	Not available	Not available
Picture Word Memory Test-Trial 1 (Non-Verbal)	3	2.3	Jelinek et al. (2006)	Not available	Not available
RAVLT Trial 1	3	2.3	Geffen et al. (1990)	Test-retest =.90	Snow et al. (1988)
WMS-III Spatial Span	3	2.3	Burton et al. (2002, 2003); Wechsler (1997)	Split-half=.77	Wechsler (1997)
WMS-R Visual Memory Span	3	2.3	Nicks et al. (1992)	Split-half=.81	Wechsler (1987)
Continuous Performance Test-d’	2	1.5	Egeland & Kovalik-Gran (2010)	Split-half = .81	Conners (2000)
Continuous Performance Test-Hits	2	1.5	Barkley et al. (2001); Egeland & Kovalik-Gran (2010)	Split-half=.95	Conners (2000)
Continuous Performance Test-Random Errors	2	1.5	Not available	Not available	Not available
Corsi Block Tapping-Forward	2	1.5	Colom et al. (2008)	Cronbach’s α = .83	Colom et al. (2008)
Corsi Block Tapping-Backward	2	1.5	Colom et al. (2008)	Cronbach’s α = .83	Colom et al. (2008)
d2	2	1.5	Bates & Lemay (2004)	Cronbach’s α = .97	Bates & Lemay (2004)
Letter Cancellation-Omissions	2	1.5	Uttl & Pilkenton-Taylor (2001)	Test-retest = .87	Parrott (1991)
WMS/WMS-R Attention/Concentration Index	2	1.5	Bornstein & Chelune (1989); Johnstone et al. (1995)	Test-retest=.90	Wechsler (1987)
WMS-III Spatial Span-Forward	2	1.5	Burton et al. (2002, 2003); Wechsler (1997)	Split-half=.77	Wechsler (1997)
WMS-III Spatial Span-Backward	2	1.5	Burton et al. (2002, 2003); Wechsler (1997)	Split-half=.77	Wechsler (1997)
WMS-III Working Memory Index	2	1.5	Wechsler (1997)	Split-half=.94	Wechsler (1997)
Adaptive Digit Ordering Test	1	0.8	Werheid et al. (2002)	Split-half=.86	Werheid et al. (2002)
Benton Visual Form Discrimination-Matching	1	0.8	Moses (1986)	Test-retest = .71	Campo & Morales (2003)
CANTAB Spatial Working Memory	1	0.8	Robbins et al. (1994)	Test-retest = .68	Lowe and Rabbit (1998)
Cognitive Evaluation Protocol (CEP) Attention-Simple	1	0.8	Not available	Not available	Not available
Cognitive Evaluation Protocol (CEP) Attention-Double	1	0.8	Not available	Not available	Not available
Cognitive Evaluation Protocol (CEP) Attention-Reverse	1	0.8	Not available	Not available	Not available
DKEFS Trails Visual Scanning	1	0.8	Delis et al. (2001)	Test-retest = .56	Delis et al. (2001)
IntegNeuro Digit Span	1	0.8	Paul et al. (2005); Silverstein et al. (2010)	Test-retest = .63	Williams et al. (2005)
IntegNeuro Span of Visual Memory	1	0.8	Paul et al. (2005); Silverstein et al. (2010)	Test-retest = .53	Silverstein et al. (2010)
Sustained Attention to Response Task-Commission Errors	1	0.8	Robertson et al. (1997)	Test-retest = .76	Robertson et al. (1997)
Sustained Attention to Response Task-Omission Errors	1	0.8	Robertson et al. (1997)	Test-retest = .76	Robertson et al. (1997)
Total	131	100
Executive Functions
Trail making Test, Part B	18	21.2	Wilcutt et al. (2001, 2005)	Test-retest =.77	Calamia et al. (2013)
Stroop-Interference	13	15.3	Boon et al. (1998); MacLeod (1991)	Test-retest = .84	Dikmen et al. (1999)
Wisconsin Card Sorting Test-Perseverative Responses	7	8.2	Miyake et al. (2000); Willcutt et al. (2005)	Cronbach’s α = .72	Kongs et al. (2000)
Wisconsin Card Sorting Test-Total Correct	7	8.2	Perrine (1993); Greve et al. (1998; 2005)	Test-retest = .79	Tate et al. (1998)
Wisconsin Card Sorting Test-Categories Completed	6	7.1	Perrine (1993); Greve et al. (1998, 2005)	Test-retest = .88	Tate et al. (1998)
Wisconsin Card Sorting Test-Perseverative Errors	5	5.9	Shute & Huertas (1990)	Cronbach’s α = .76	Kongs et al. (2000)
DKEFS Color-Word Interference-Inhibition	3	3.5	Delis et al. (2001); Mattson et al. (1999)	Test-retest = .75	Delis et al. (2001)
DKEFS Color-Word Interference-Inhibition Switching	3	3.5	Delis et al. (2001); Mattson et al. (1999)	Test-retest = .65	Delis et al. (2001)
Wisconsin Card Sorting Test-Total Errors	3	3.5	Perrine (1993); Greve et al. (1998, 2005)	Cronbach’s α = .85	Kongs et al. (2000)
Category Test-Total Errors	2	2.4	O’Donnell et al. (1994)	Test-retest = .85	Dikmen et al. (1999)
DKEFS Trail Making Switching	2	2.4	Delis et al. (2001)	Test-retest = .38	Delis et al. (2001)
DKEFS Verbal Fluency-Switching	2	2.4	Baldo et al. (2001); Delis et al. (2001)	Test-retest = .88	Delis et al. (2001)
CANTAB Intra/Extra Dimensional Set Shift Total Errors	1	1.2	Robbins et al. (1994)	Test-retest = .70	Lowe and Rabbit (1998)
CANTAB Stockings of Cambridge Choices to Correct	1	1.2	Robbins et al. (1994)	Not available	Not available
CANTAB Stop-Signal Task Median Correct on Go Trials	1	1.2	Robbins et al. (1994)	Not available	Not available
Color Trails Part 2	1	1.2	Maj et al. (1993),Uchiyama et al. (1994)	Test-retest = .79	D’Elia et al. (1996)
DKEFS Design Fluency-Switching vs. Combined	1	1.2	Baldo et al. (2001); Delis et al. (2001)	Test-retest = .37	Crawford et al. (2008)
Go-NoGo Composite Score	1	1.2	Garavan et al. (2002); Trommer et al. (1988)	Not available	Not available
Hayling Sentence Completion Test-Suppression	1	1.2	Clark et al. (2000); Andres & Van der Linden (2000)	Test-retest = .62	Shallice & Burgess (1997)
Hayling Sentence Completion Test-Initiation	1	1.2	Clark et al. (2000); Andres & Van der Linden (2000)	Test-retest = .78	Shallice & Burgess (1997)
IntegNeuro Executive Maze Total Completion Time	1	1.2	Paul et al. (2005); Silverstein et al. (2010)	Test-retest = .86	Silverstein et al. (2010)
IntegNeuro Switching of Attention Numbers & Letters	1	1.2	Paul et al. (2005); Silverstein et al. (2010)	Test-retest = .78	Silverstein et al. (2010)
IntegNeuro Verbal Interference Total Score	1	1.2	Paul et al. (2005); Silverstein et al. (2010)	Test-retest = .71	Williams et al. (2005)
Porteus Maze Test	1	1.2	Gow & Ward (1982)	Cronbach’s α = .81	Krikorian & Bartok (1998)
Short Category Test-Errors	1	1.2	Gelowitz & Paniak (1992)	Split-half = .81	Wetzel & Boll (1987)
Tower of London	1	1.2	Miyake et al. (2000)	Cronbach’s α = .79	Schnirman et al. (1998)
Total	85	100
Verbal Learning (Immediate Memory)
CVLT-2 Trials 1–5/CVLT Total Learning	17	28.3	Donders (2008a, 2008b)	Test-retest = .75	Calamia et al. (2013)
WMS/WMS-R/WMS-III Logical Memory I (Immediate)	12	20.0	Wechsler (1997)	Split-half=.88	Wechsler (1997)
RAVLT Total Recall (Trials 1–5)	9	15.0	Salthouse (1996)	Cronbach’s α = .90	Van den Burg & Kingma (1999)
WMS/WMS-R Verbal Memory Index	6	10.0	Bornstein & Chelune (1989)	Test-retest = .77	Wechsler (1987)
Buschke Verbal Selective Reminding Test Total Recall	2	3.3	Allen & Ruff (1999); Larrabee & Curtiss (1995)	Test-retest = .62	Dikmen et al. (1999)
Paired Associates Recall (Low Associates)	2	3.3	Lupien et al. (1994)	Not available	Not available
Paired Associates Recall (High Associates)	2	3.3	Lupien et al. (1994)	Not available	Not available
WMS-R Verbal Paired Associates I	2	3.3	Nicks et al. (1992)	Test-retest = .60	Wechsler (1987)
Guild Memory Test Paragraph Recall Immediate	2	3.3	Crook et al. (1980)	Split-half=.87	Gilbert (1970)
Hopkins Verbal Learning Test-Total Learning	2	3.3	Shapiro et al. (1999)	Test-retest = .74	Benedict et al. (1998)
Verbal Paired Associates-Total	2	3.3	Wechsler (1987)	Test-retest = .58	Dikmen et al. (1999)
WMS-III Auditory Immediate Index	2	3.3	Wechsler (1997)	Split-half = .93	Wechsler (1997)
Total	60	100
Visual Learning (Immediate Memory)
WMS-R/WMS-III Visual Reproduction I	8	24.1	Bornstein & Chelune (1989); Wechsler (1997)	Test-retest = .62	Dikmen et al. (1999)
WMS/WMS-R Visual Memory Index	7	24.1	Nicks et al. (1992)	Split-half=.70	Wechsler (1987)
WMS-III Visual Immediate Recall	4	13.8	Wechsler (1997)	Split-half=.82	Wechsler (1997)
Continuous Visual Memory Test Total Correct	4	13.8	Strong & Donders (2008)	Test-retest = .80	Trahan & Larrabee (1988)
Buschke Visual Selective Reminding Test Recall	2	6.9	Allen & Ruff (1999); Larrabee & Curtiss (1995)	Test-retest = .74	Salinsky et al. (2001)
Benton Visual Form Discrimination-Memory	1	3.4	Moses (1986)	Cronbach’s α = .74	Lopez et al. (2005)
Rey Visual Design Learning Test Total Learning	1	3.4	Moye (1997)	Test-retest = .45	Strauss, Sherman, & Spreen (2006)
WMS-III Faces 1	1	3.4	Burton et al. (2002, 2003); Wechsler (1997)	Split-half = .74	Wechsler (1997)
WMS-III Family Pictures 2	1	3.4	Burton et al. (2002, 2003); Wechsler (1997)	Split-half = .84	Wechsler (1997)
WMS-R Visual Paired Associates I	1	3.4	Bornstein & Chelune (1989)	Test-retest = .58	Wechsler (1987)
Total	30	100
Verbal (Delayed) Memory
CVLT/CVLT-2 Long Delay Free Recall	17	26.2	Donders (2008a, 2008b)	Test-retest = .75	Calamia et al. (2013)
WMS/WMS-R/WMS-III Logical Memory II (Delayed)	14	21.5	Millis et al. (1999); Wechsler (1997)	Split-half = .79	Wechsler (1997)
RAVLT Long Delay Recall	7	6.2	Salthouse (1996)	Test-retest = .88	Calamia et al. (2013)
WMS-R Verbal Paired Associates II	6	9.2	Nicks et al. (1992)	Test-retest = .41	Wechsler (1987)
Buschke Verbal Selective Reminding Test	4	6.2	Allen & Ruff (1999);	Test-retest = .64	Dikmen et al. (1999)
Long-Term Storage			Larrabee & Curtiss (1995)
Picture Word Memory Test-Trial 4 (Verbal)	3	4.6	Jelinek et al. (2006)	Not available	Not available
Guild Memory Test Paragraph Recall Delayed	2	3.1	Crook et al. (1980)	Split-half=.87	Gilbert (1970)
WMS-III Auditory Delayed Index	2	3.1	Tulsky & Price (2003)	Split-half = .87	Wechsler (1997)
WMS-R Verbal Memory Delayed Recall	2	3.1	Nicks et al. (1992)	Test-retest = .77	Wechsler (1987)
Cognitive Evaluation Protocol (CEP) Numbers Recall	1	1.5	Not available	Not available	Not available
Cognitive Evaluation Protocol (CEP) Words Recall	1	1.5	Not available	Not available	Not available
Cognitive Evaluation Protocol (CEP) Extended Memory	1	1.5	Not available	Not available	Not available
Cognitive Evaluation Protocol (CEP) Forms Recall	1	1.5	Not available	Not available	Not available
Hopkins Verbal Learning Test-Total Retention	1	1.5	Shapiro et al. (1999)	Test-retest = .66	Benedict et al. (1998)
WMS-III Logical Memory % Retention	1	1.5	Griffith et al. (2006); Wechsler (1997)	Split-half = .79	Wechsler (1997)
WMS Paired Associates (Total)	1	1.5	Wechsler (1987)	Test-retest = .58	Dikmen et al. (1999)
Total	65	100
Visual (Delayed) Memory
Rey-O Complex Figure Delayed Recall	10	19.2	Meyers & Meyers (1995)	Test-retest = .72	Calamia et al. (2013)
WMS-R Figural Memory/WMS-III Visual Reproduction II	8	15.4	Bornstein & Chelune (1989); Wechsler (1997)	Split-half = .77	Wechsler (1997)
Continuous Visual Memory Test Total Correct (Delay)	6	11.5	Strong & Donders (2008)	Test-retest = .66	Trahan et al. (1996)
Buschke Visual Selective Reminding Test Long-Term Storage	4	7.7	Allen & Ruff (1999); Larrabee & Curtiss (1995)	Test-retest = .74	Salinksy et al. (2001)
WMS-III Visual Delayed Index	4	7.7	Wechsler (1997)	Split-half = .83	Wechsler (1997)
Picture Word Memory Test-Trial 4 (Nonverbal)	3	5.8	Jelinek et al. (2006)	Not available	Not available
WMS-R Visual Memory Delayed Recall	3	5.8	Bornstein & Chelune (1989)	Test-retest = .70	Wechsler (1987)
Benton Visual Retention Test	2	3.8	Moses (1986)	Test-retest = .63	Calamia et al. (2013)
Thurstones’s Picture Memory Test	2	3.8
Warrington Recognition Memory Test (Faces)	2	3.8	Hunkin et al. (2000)	Cronbach’s α = .77	Malina et al. (1998)
Brief Visuospatial Memory Test-Revised (BVMT-R) Recall	1	1.9	Benedict et al. (1996)	Test-retest = .71	Nuechterlein et al. (2008)
Bender Gestalt Test-Design Recall	1	1.9	Arbit & Zager (1978)	Split-half = .91	Brannigan (2003)
Cognitive Evaluation Protocol (CEP) Forms Recall	1	1.9	Not available	Not available	Not available
Rey-O Complex Figure-Savings Ratio	1	1.9	Shorr, Delis, & Massman (1992)	Test-retest = .72	Calamia et al. (2013)
Rey Visual Design Learning Test Retention	1	1.9	Moye (1997)	Test-retest = .45	Strauss, Sherman, & Spreen (2006)
WMS-R Visual Paired Associates II	1	1.9	Bornstein & Chelune (1989)	Test-retest = .58	Wechsler (1987)
WMS-III Faces 2	1	1.9	Wechsler (1997)	Split-half = .74	Wechsler (1997)
WMS-III Family Pictures 2	1	1.9	Wechsler (1997)	Split-half = .84	Wechsler (1997)
Total	52	100
Verbal and Visual Memory (Combined)1
WMS-R Delayed Memory Index	3	50.0	Bornstein & Chelune (1989)	Test-retest = .77	Wechsler (1987)
Rivermead Behavioral Memory Test-Total	3	50.0	Fennig et al. (2002)	Test-retest = .85	Wilson et al. (2008)
Total	6	100
Language
COWAT (FAS)	13	35.1	Henry & Crawford (2004)	Test-retest = .79	Calamia et al. (2013)
Animal Fluency	12	32.4	Henry & Crawford (2004)	Test-retest = .74	Nuechterlein et al. (2008)
Boston Naming Test	3	8.1	Axelrod et al. (1994); Schefft et al. (2003)	Test-retest = .92	Dikmen et al. (1999)
Letter Fluency	3	8.1	Henry & Crawford (2004)	Test-retest = .72	Dikmen et al. (1999)
DKEFS Category Fluency	2	5.4	Baldo et al. (2001)	Test-retest = .79	Delis et al. (2001)
DKEFS Letter Fluency	2	5.4	Baldo et al.(2001)	Test-retest = .80	Delis et al. (2001)
IntegNeuro Verbal Fluency	1	2.7	Paul et al. (2005); Silverstein et al. (2010)	Test-retest = .74	Silverstein et al. (2010)
Semantic Fluency (Animals, Fruits, Vegetables)	1	2.7	Henry & Crawford (2004)	Test-retest = .59	Vlaar & Wade (2003)
Total	37	100
Speed of Information Processing
Trail making Test, Part A	18	37.5	Crowe (1998)	Test-retest = .77	Nuechterlein et al. (2008)
WAIS/WAIS-R/WAIS-III/WAIS-IV Digit Symbol/Coding	9	18.8	Joy et al. (2004); Kreiner & Ryan (2001)	Test-retest = .85	Calamia et al. (2013)
Stroop-Color	8	16.7	Felmingham, Baguley, & Green (2004)	Test-retest = .89	Salinksy et al.
Symbol Digit Modalities Test	5	10.4	Benedict & Zivadinov (2007)	Test-retest = .85	Benedict & Zivadinov (2007)
DKEFS Color-Word Interference-Color	2	4.2	Delis et al. (2001); Mattson et al.	Test-retest = .76	Delis et al. (2001)
Color Trails Part 1	1	2.1	Maj et al. (1993)	Test-retest = .64	D’Elia et al. (1996)
DKEFS Trails Number Sequencing	1	2.1	Delis et al. (2001)	Test-retest = .59	Delis et al. (2001)
DKEFS Design Fluency-Combined	1	2.1	Baldo et al. (2001); Delis et al. (2001)	Test-retest = .58	Delis et al. (2001)
IntegNeuro Switching of Attention-Numbers	1	2.1	Paul et al. (2005); Silverstein et al. (2010)	Test-retest = .67	Silverstein et al. (2010)
WAIS-IV Processing Speed Index	1	2.1	Holdnack et al. (2011)	Cronbach’s α = .90	Wechsler (2008)
WAIS-IV Symbol Search	1	2.1	Holdnack et al. (2011)	Test-retest = .74	Calamia et al. (2013)
Total	48	100
Visuospatial Functioning
Rey-O Complex Figure-Copy	9	75.0	Meyers & Meyers (1995)	Test-retest = .50	Calamia et al. (2013)
Judgment of Line Orientation	2	16.7	Trahan (1998)	Cronbach’s α = .90	Qualls et al. (2000)
Benton Visual Retention Test (Reproduction)	1	8.3	Crook & Larrabee (1988)	Test-retest = .85	Benton (1974)
Total	12	100

¹Not included in neurocognitive domain analyses or comparisons.

Note. % = Percent of journal articles within each domain that included the neuropsychological test in the primary source, k = number of studies. CANTAB = Cambridge Neuropsychological Test Automated Battery; CVLT = California Verbal Learning Test; COWAT= Controlled Oral Word Association Test; DKEFS= Delis Kaplan Executive Functioning System; PASAT = Paced Auditory Serial Addition Test; RAVLT = Rey Auditory Verbal Learning Test; Rey-O = Rey-Osterrieth; Split-half = split-half reliability; Test-retest = test-retest reliability; WAIS = Wechsler Adult Intelligence Test; WMS = Wechsler Memory Scale

Statistical Analyses

A mixed-effects multivariate model was used in our meta-analysis computations for a number of theoretical and practical reasons (for review, see Arends, Vokó, & Stijnen, 2003; Kalaian & Raudenbush, 1996). In many meta-analyses, a single study may contribute more than one effect size estimate because studies report multiple outcome measures, such as multiple follow-up times, multiple control groups, multiple treatments, or multiple assessments of related constructs. In recognition of the likely non-independence of effect sizes within studies, conducting multiple separate univariate meta-analyses has been a common analytic approach. Unfortunately, this approach precludes the comparison and syntheses of effect sizes within studies, and leads to repetitive and partly redundant analyses if within-study effect sizes are correlated. Riley (2009) has demonstrated that treating multiple effect sizes within studies as if they were statistically independent does not provide a solution either. In fact, such an approach may lead to biased estimates and invalid conclusions, unless the within-study variance is small relative to the between-study variance and the within-study covariances differ little across studies.

The statistically and substantively more sound approach is a multivariate model that allows for multiple correlated within-study effect sizes, takes the hierarchical (clustered) data structure into account, and allows for different cluster sizes (i.e., different number of effect sizes per study). Moreover, a multivariate mixed-effects model for meta-analysis allows us to increase generalizability and make inferences about the population of studies on the neurocognitive effects of PTSD, including ones that differ from the included studies in such factors as participants, PTSD characteristics, and outcome measures, instead of solely allowing inferences about this particular set of studies. A general framework for such analyses is provided by the Generalized Linear Latent and Mixed Models (gllamm) implemented in Stata 12 (Grilli & Rampichini, 2006; Rabe-Hesketh, Skrondal, & Pickles, 2004; StataCorp, 2011).

Specifically, we defined a two-level mixed effects model, where level 1 is represented by multiple effect sizes within studies, and level 2 is represented by the different studies. This model examines the variability of effects sizes between studies (random factor) and the association between various explanatory variables (fixed factors) and effect sizes. To apply this model to meta-analytic data, we first calculated standardized mean effect sizes (d) and determined the sampling variance of each effect size, as detailed above. The model considers the level-1effect size variances as fixed/known (as calculated). The fixed and random effects parameters and their variances and covariance are estimated via adaptive quadrature, a robust and flexible numeric integration approach that allows for heteroscedastic level-1 variances (Rabe-Hesketh et al., 2004; Rabe-Hesketh, Skrondal, & Pickles, 2005).

We first tested a simple model without explanatory variables to estimate an overall mean effect size and the between-study variance (Scott et al., 2007):

$$y_{ij}=\alpha +u_j+e_{ij}{u}_j~N(0,{\sigma }_u^2)e_{ij}~N(0,s_{ij}^2)$$

where y_ij refers to effect sizes (i) within studies (j), α is a constant (i.e., the overall mean), u_j are the study-level random effects, and e_ij is the effect-size level residual. ${\sigma }_u^2$ is the variance parameter to be estimated for the between-study variance, and $s_{ij}^2$ are the known conditional variances of the effect sizes. This analysis revealed that the overall mean effect size was d = −.49 (SE = .038) and the between-study variance estimate was .085 (SE = .017, p < .001), indicating that the variance between studies was significantly more than that explained by sampling error alone. The significance of the between-study variance prompted an exploration as to whether neurocognitive test domain, participant clinical and sociodemographic characteristics, between-group IQ discrepancy, psychiatric comorbidity, or study inclusion/exclusion criteria could account for some of the between-study variance.

To examine single explanatory variables, we fit the following model:

$$\begin{array}{l}{y}_{ij}=\alpha +\beta x_{ij}+u_j+e_{ij}\\ u_j~N(0,{\sigma }_u^2)e_{ij}~N(0,s_{ij}^2)\end{array}$$

where β is the regression slope associated with the explanatory variable.

All models were fit using the program gllamm of Stata version 12 (Grilli & Rampichini, 2006; Rabe-Hesketh et al., 2004; StataCorp, 2011). The level-1 variances of the effect sizes were fixed to the estimates of the conditional effect-size variances.

Table 2 presents the included participants’ demographic data and PTSD characteristics, and Table 3 presents a summary of the studies used in the meta-analysis.

Table 2.

Overview of Studies Included in the Meta-Analysis.

Study	PTSD n	Neurocognitive Domains Assessed	Trauma Type	Diagnostic Method	Head Injury Status	Control Group
Aupperle et al. (2012)	37	Executive Functions, SIP	Interpersonal	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Healthy
Beckham et al. (1998)	45	Executive Functions, SIP	Military	Mississippi Scale for Combat-Related PTSD	Any and Mild TBI excluded	Trauma Exposed
Bremner et al. (1993)	21	Verbal Learning and Memory, Visual Learning and Memory	Military	SCID-III, Mississippi Scale for Combat-Related PTSD > 107, and Consensus Diagnosis; DSM-IV Criteria	Any and Mild TBI excluded	Healthy
Bremner et al. (1995)	26	Verbal Learning and Memory, Visual Learning and Memory	Interpersonal	SADS-L Criteria for Current PTSD	Any and Mild TBI excluded	Healthy
Bremner et al. (2004)	18	Verbal Learning and Memory, Visual Learning and Memory	Interpersonal	SCID-IV; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
						Healthy
Cohen et al. (2013)	196	Executive Functions, Verbal Learning, Language, SIP	Military	CAPS, F1/I2 Method; DSM-IV Criteria	Unspecified	Trauma Exposed
Cottencin et al. (2006)	30	Attention/WM, Executive Functions, Verbal Learning, Visual Learning, Language, SIP	Mixed or Unknown	CAPS; DSM-IV Criteria	Unspecified	Healthy
Crowell et al. (2002)	80	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Memory, Language, Visuospatial	Military	DIS-III-A; DSM-III Criteria in Past Year and Elevations on MMPI Profile Consistent with PTSD Symptomatology	Any and Mild TBI excluded	Trauma Exposed
Dileo et al. (2008)	31	Verbal Memory, Visual Memory, Language, Visuospatial	Military	PCL-M Total > 50	“Significant head trauma” excluded	Healthy
Dretsch et al. (2012)	23	Attention/WM	Military	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Elsesser & Sartory (2007)	20	Learning and Memory (total scores comprising both verbal and visual)	Mixed or Unknown	DIPS; DSM-IV Criteria	Unspecified	Healthy
Emdad et al. (2005)	30	Visual Memory	Mixed or Unknown	CAPS	Any and Mild TBI excluded	Healthy
Eren-Kocak et al. (2009)	16	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Memory, Language, SIP, Visuospatial	Mixed or Unknown	CAPS (Turkish Version); DSM-IV Criteria	Any and Mild TBI excluded	Healthy
Falconer et al. (2008)	44	Attention/WM, Executive Functions, Language, SIP	Mixed or Unknown	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Healthy
Flaks et al. (2014)	81	Attention/WM, Executive Functions, SIP	Interpersonal	CAPS; DSM-IV Criteria & Total Score > 45; Consensus Conference	Any and Mild TBI excluded	Healthy
						Trauma Exposed
Geuze et al. (2009)	25	Verbal Learning and Memory, Visual Learning and Memory	Military	CAPS; DSM-IV Criteria & Total Score > 50	“Significant head trauma” excluded	Trauma Exposed
Gil et al. (1990)	12	Attention/WM, Verbal Memory, Visual Memory, Language, SIP, Visuospatial	Mixed or Unknown	DSM-III Criteria Confirmed by Consensus of Two Psychiatrists	“Significant head trauma” excluded	Healthy
Gilbertson et al. (2001)	19	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Learning and Memory, SIP	Military	SCID-III; DSM-III-R Criteria	Any and Mild TBI excluded	Trauma Exposed
Golier et al. (1997)	24	Attention/WM, Language	Military	CAPS, Mississippi Scale for Combat-Related PTSD, & Consensus Conference; DSM-III-R Criteria	Any and Mild TBI excluded	Healthy
Golier et al. (2002)	31	Verbal Learning	State Persecution/Terror	CAPS & SCID-IV; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
						Healthy
Golier et al. (2005)	14	Verbal Learning and Memory	State Persecution/Terror	CAPS, SCID-IV, & Consensus Conference; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
						Healthy
Gurvits et al. (1993)	27	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Learning and Memory, SIP	Military	SCID-III; DSM-III-R Criteria	“Significant head trauma” excluded	Trauma Exposed
Gurvits et al. (1996)	7	Attention/WM, Verbal Learning and Memory, Visual Learning and Memory	Military	CAPS; DSM-III-R Criteria	Any and Mild TBI excluded	Trauma Exposed
Hart et al. (2008)	14	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Memory, Language, SIP, Visuospatial	Military	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Horner et al. (2013)	19	Attention/WM, Executive Functions	Military	Diagnosed by neuropsychologist; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Jelinek et al. (2006)	40	Attention/WM, Verbal Memory, Visual Memory	Mixed or Unknown	SCID (German Version); DSM-IV Criteria	Unspecified	Healthy
Jelinek et al. (2008)	15	Attention/WM	Mixed or Unknown	SCID (German Version); DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
Jelinek et al. (2009)	26	Attention/WM	Mixed or Unknown	SCID (German Version); DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
						Healthy
Jelinek et al. (2013)	20	Attention/WM, Executive Functions, Verbal Memory, Visual Memory, SIP	State Persecution/Terror	SCID (German Version); DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
						Healthy
Jenkins et al. (2000)	16	Attention/WM, Executive Functions, SIP	Interpersonal	SCID and PTSD Interview; DSM-III-R Criteria	Any and Mild TBI excluded	Trauma Exposed
						Healthy
Johnsen et al. (2008)	21	Attention/WM, Verbal Memory, Visual Memory	State Persecution/Terror	CAPS; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
Kanagaratnam & Asbjornsen (2007)	22	Executive Functions, SIP	State Persecution/Terror	CAPS; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
Kivling-Boden & Sundbom (2003)	21	Visual Memory	State Persecution/Terror		Unspecified	Trauma Exposed
Koenen et al. (2001)	16	Executive Functions, Verbal Learning and Memory, Visual Learning and Memory, Language, SIP	Mixed or Unknown	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Healthy
Koso & Hansen (2006)	20	Attention/WM, Executive Functions, SIP, Learning and Memory (total scores comprising both verbal and visual)	Military	Diagnosed by Psychiatrists; DSM-IV Criteria	Any and Mild TBI excluded	Healthy
Lindauer et al. (2006)	12	Executive Functions, Verbal Learning and Memory, Visual Learning and Memory	Mixed or Unknown	Structured Interview for PTSD; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Matsuo et al. (2003)	8	Attention/WM, Verbal Learning, Visual Learning	State Persecution/Terror	CAPS; DSM-IV Criteria	Unspecified	Trauma Exposed
Moores et al. (2008)	13	Attention/WM, Executive Functions, Language, SIP	Mixed or Unknown	CAPS; DSM-IV Criteria	“Significant head trauma” excluded	Healthy
Neylan et al. (2004)	24	Attention/WM, Verbal Learning and Memory, Visual Learning and Memory	Military	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Olff et al. (2014)	28	Attention/WM, Executive Functions	Mixed or Unknown	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Pederson et al. (2004)	17	Attention/WM, Verbal Learning and Memory, Visual Learning and Memory	Interpersonal	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
						Healthy
Sachinvala et al. (2000)	36	Attention/WM, Verbal Memory, Visual Memory	Military	Assessed by Two Senior Clinicians; DSM-IV Criteria	“Significant head trauma” excluded	Healthy
Samuelson et al. (2006)	67	Attention/WM, Verbal Learning and Memory, Visual Learning and Memory, SIP	Military	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Sarac-Hadzihalilović et al. (2008)	45	Memory (total scores comprising both verbal and visual)	Military	Mississippi Scale for Combat-Related PTSD > 110	“Significant head trauma” excluded	Healthy
Shandera-Ochsner et al. (2013)	19	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Memory, Language, SIP	Military	CAPS, F1/I2 Method of Scoring; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Stein et al. (2002)	17	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Memory, Language, SIP, Visuospatial	Interpersonal	CAPS; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed Healthy
Sullivan et al. (2003)	11	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Learning and Memory, Language, SIP	Military	CAPS; DSM-III-R Criteria	Unspecified	Trauma Exposed
Twamley et al. (2004)	37	Attention/WM, Executive Functions, Language, SIP	Mixed or Unknown	PDS; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed Healthy
Twamley et al. (2009)	55	Attention/WM, Executive Functions, Visual Memory, Language, SIP, Visuospatial	Interpersonal	CAPS, F1/I2 Method of Scoring; DSM-IV Criteria	Any and Mild TBI excluded	Healthy
Uddo et al.(1993)	16	Attention/WM, Verbal Learning and Memory, Visual Memory, Language, Visuospatial	Military	MMPI-derived PTSD Scale > 30, score > 107 on Mississippi Scale for Combat-Related PTSD, and SCID diagnosis of current PTSD	Any and Mild TBI excluded	Healthy
Vasterling et al. (2000)	26	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Learning and Memory	Military	SCID; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed Healthy
Vasterling et al. (2002)	26	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Learning and Memory	Military	SCID; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Vasterling et al. (1998)	19	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Learning and Memory	Military	SCID; DSM-IV Criteria	Any and Mild TBI excluded	Trauma Exposed
Vythilingam et al. (2005)	14	Verbal Learning and Memory, Visual Learning and Memory	Interpersonal	SCID; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
Wisdom et al. (2013)	30	Attention/WM, Executive Functions, Verbal Learning and Memory, Visual Memory, Language, SIP	Military	PCL-C Total > 50	Any and Mild TBI excluded	Healthy
Woodward et al. (2009)	48	Verbal Learning and Memory, Visual Learning and Memory, Visuospatial	Military	CAPS; DSM-IV Criteria	Unspecified	Trauma Exposed
Yehuda et al. (1995)	20	Attention/WM, Verbal Learning and Memory, Language	Military	Consensus Conference based on CAPS, Mississippi Scale for Combat-Related PTSD, and Clinical History; DSM-IV Criteria	Any and Mild TBI excluded	Healthy
Yehuda et al. (2004)	36	Attention/WM, Verbal Learning and Memory	State Persecution/Terror	CAPS; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
						Healthy
Yehuda et al. (2005)	30	Attention/WM, Verbal Learning and Memory	Military	CAPS; DSM-IV Criteria	“Significant head trauma” excluded	Trauma Exposed
						Healthy
Yehuda et al. (2007)	17	Attention/WM, Verbal Memory	Military	CAPS; DSM-IV Criteria	Unspecified	Trauma Exposed

Note: CAPS = Clinician Administered PTSD Scale (Weathers, Keane, & Davidson, 2001);DIS = Diagnostic Interview Schedule; DIPS = Diagnostisches Interview für psychische Störungen (Margraf, Schneider, & Ehlers, 1994);DSM = Diagnostic and Statistical Manual of Mental Disorders; MMPI = Minnesota Multiphasic Personality Inventory; PCL-C = PTSD Checklist-Civilian Version ; PDS = Posttraumatic Diagnostic Scale (Foa, 1995); SADS-L = Schedule for Affective Disorders and Schizophrenia-Lifetime; SCID = Structured Clinical Interview for DSM (First, Spitzer, Gibbon, & Williams, 2002; Spitzer, Williams, Gibbon, & First, 1992);SIP = speed of information processing; TBI = traumatic brain injury; WM = working memory.

Table 3.

Participants’ demographic data

	PTSD Group			TC Group			NC Group
	k	N	Mean (SD)	k	N	Mean (SD)	k	N	Mean (SD)
Age	54	1,705	44.02 (13.69)	38	1338	46.25 (15.20)	32	843	42.12 (14.17)
Education	46	1,247	13.20 (1.42)	36	967	14.82 (1.73)	25	693	14.18 (1.60)
% Male	56	1,631	67.95%	40	1338	68.54%	34	821	55.10%

Note. PTSD = posttraumatic stress disorder; TC = trauma exposed comparison; NC = non-trauma exposed comparison; k = number of studies; N = number of participants.

Preliminary Analyses

Funnel plot tests and exploratory analyses were conducted to examine potential small study bias in the literature. Figure 1 displays a funnel plot of effect size estimates across the 60 studies along with their standard error. Visual inspection of this funnel plot revealed asymmetry, suggestive of small study effects, and Egger and colleagues’ (1997) method to test small study effects revealed significant bias (t = 7.78, p < .001). When the “trim and fill” method of Duval and Tweedie (2000) was used to examine the effect of “filling” the funnel plot with the missing effect sizes, a significant adjusted mean effect size remained (p < .001). However, it is estimated that this overall effect size would be reduced by approximately 29%.

Figure 1.

Funnel plot with effect sizes (d) separated by psychiatric comorbidity exclusion criteria (psychiatric disorders allowed).

Note. CI = confidence interval.

In line with recent recommendations (Sterne et al., 2011), we undertook further examination of a number of potential causes of these small study effects to aid in their interpretation. We chose potential explanations by examining characteristics of the studies included in the meta-analysis that had the largest standard error values. First, a new variable was coded to indicate whether the study was conducted in a non-English speaking country, as diagnostic and neuropsychological measures that are translated from English without proper psychometric investigation potentially suffer from reduced reliability and validity. Egger’s test showed that the problem of small study effects was not diminished when examining only studies from English speaking countries (t = 6.41, p < .001). We also examined whether the timeframe of the study (1990–1999–2000–2009) could help explain small study effects, because as the research literature expands in a field of study, the precision of the effect size estimates generally improves with larger and more rigorous studies. However, both time periods were associated with significant bias according to Egger’s test (1990s: t = 3.13, p = .002; 2000s: t = 6.31, p < .001).

We also examined whether studies allowing a greater number of comorbid psychiatric diagnoses were more likely to exhibit funnel plot asymmetry. Testing those studies that allowed either no comorbid diagnoses or allowed only depression (compared to studies allowing additional psychiatric disorders or those that did not specify psychiatric exclusion criteria) revealed a generally symmetrical funnel plot (Egger’s test t = 1.81, p = .072), as shown in Figure 1. Thus, studies with more rigorous psychiatric exclusion criteria were less susceptible to small study effects.

Results

Neurocognitive Domains

Figure 2 displays the mean weighted effect sizes and 95% confidence intervals for each neurocognitive domain across the 60 studies, which ranged from d = −.29 to −.62. The 95% confidence interval surrounding the mean effect size for each domain did not contain zero, and thus effect sizes in every domain examined were significantly different from zero. By convention, d-values of .2, .5, and .8 correspond to small, medium, and large effect sizes, respectively (Cohen, 1988), although it should be noted that these categorizations are broad and do not necessarily signify levels of practical significance. As illustrated in Figure 1, the largest effect sizes were seen in the domains of verbal learning (d = −.62), speed of information processing (d = −.59), and attention/working memory (d = −.50), which were all in the medium range. Effect sizes of a slightly smaller magnitude were observed in the domains of verbal memory (d = −.46), executive functions (d = −.45), and language (d = −.43), with small effects in visuospatial functioning (d = −.38), visual learning (d = −.32), and visual memory (d = − .29).

Figure 2.

Mean effect sizes and 95% confidence intervals for each neurocognitive test domain.

Note. *k=60, Mean=grand mean effect size, LB = lower bound, UB = upper bound, SIP = speed of information processing, WM = working memory, CI = confidence interval.

Overall, significant differences in mean effect size estimates were found across neurocognitive test domains (χ²=48.92, p < .001). Specific contrasts revealed that attention/working memory had significantly larger effect sizes than visuospatial functioning (χ² = 4.88, p = 0.03), visual learning (χ² = 6.70, p = 0.01), and visual memory (χ² = 15.06, p < 0.001). Verbal learning displayed significantly larger effect sizes than verbal memory (χ² = 8.59, p = 0.003), executive functions (χ² = 6.45, p = 0.01), language (χ² = 9.10, p = 0.003), visuospatial processing (χ² = 6.61, p = 0.01), visual learning (χ² = 17.85, p < 0.001), and visual memory (χ² = 32.26, p < 0.001). Verbal memory had significantly larger effect sizes than visuospatial functioning (χ² = 4.37, p = 0.04) and visual memory (χ² = 8.61, p =0.003). Speed of information processing had significantly greater effect sizes than executive functions (χ² = 7.77, p =0.005), language (χ² = 6.30, p = 0.01), visuospatial processing (χ² = 4.62, p =0.03), visual learning (χ² = 12.37, p < 0.001), and visual memory (χ² = 23.09, p < 0.001). Executive functions had significantly larger effect sizes than visuospatial functioning (χ² = 8.11, p = 0.04).

Analyses examining the associations between study characteristics and effect size estimates were performed individually with a number of explanatory variables, including participant demographics, sample characteristics, and inclusion/exclusion criteria.

Effect of Control Group and Type of Trauma

In line with previous meta-analyses (Brewin et al., 2007; Polak et al., 2012), the PTSD samples were coded into four types of index traumas: (1) military trauma; (2) interpersonal trauma; (3) state persecution/terror; and (4) mixed/unknown trauma type. Table 4 shows the results from mixed effects meta-analyses of these different groups. No significant differences were found between the four trauma types in magnitude of effect size estimates (χ² = 1.38, p = 0.71).

Table 4.

Results of mixed effects meta-analyses by trauma type

Trauma	k	d	SE	t	p	95% CI
Military	29	−0.50	.06	8.80	<.001	−0.58, −0.38
Interpersonal	9	−0.41	.09	5.08	<.001	−0.57, −0.26
State persecution/terror	9	−0.55	.10	5.67	<.001	−0.73, −0.36
Mixed/unknown	13	−0.44	.09	5.18	<.001	−0.60, −0.27

Note. k = number of studies; d = Cohen’s d; SE = standard error; CI = confidence interval

Similarly, although use of trauma unexposed control groups resulted in a numerically larger effect size estimate than use of trauma exposed control groups, there were no significant differences in the magnitude of effect size estimates between the two (χ² = 1.83, p = 0.18; Trauma Unexposed, d = −.53; Trauma Exposed, d = −.43).

Effects of PTSD and Clinical Variables

The severity of PTSD symptoms (as assessed by the CAPS Total, available for k =21 studies) did not have an appreciable influence on the magnitude of the effect size (β = −.003, p = .36). However, after reviewing the literature to examine which tests were most often associated with PTSD severity, we performed a post hoc analysis to examine whether the severity of PTSD symptoms were specifically associated with performances in attention/working memory, verbal learning, or speed of information processing. This analysis showed that severity of PTSD symptoms as assessed by the CAPS was associated with the magnitude of effect size in verbal learning (β = −.015, p = .02), such that a 10 point increase in CAPS Total would be associated with the magnitude of the effect size estimate in verbal learning increasing by 0.15. CAPS Total was not associated with performance in attention/working memory of speed of information processing.

To examine the influence of additional clinical characteristics, we created a variable to indicate whether the PTSD sample was treatment-seeking (k = 25), from the community (k = 9), or a mixture of treatment-seeking and community samples (k = 26). Analyses of this variable indicated that treatment-seeking status exhibited a significant influence on the magnitude of the effect size estimates (χ² = 15.35, p < 0.001). Specific contrasts revealed that treatment-seeking PTSD samples evidenced effect sizes of a significantly greater magnitude (d = −.65) than both community (d= −.30, p < .001) and mixed groups (d= −.40, p = .001), while the community and mixed sample groups did not differ significantly (p = .30).

The strictness of study exclusion criteria for psychotropic medications was also examined as a explanatory variable, although it failed to have a significant influence on the magnitude of effect size (χ² = 0.76, p = .38).

Comorbidity Effects

Analyses of the percent of individuals with major depression (β = −.001, p = .59), alcohol 9use disorders (β = .002, p = .36), and substance use disorders (β = .002, p = .41) in the PTSD group in each study revealed that none of these variables exhibited a significant influence on the magnitude of effect sizes. We also created a variable to indicate whether studies had included individuals from the PTSD group with mental health disorders other than PTSD (No Other Disorders or Depression Only, k = 15; Anxiety & Depression, k = 17; Anxiety, Depression, and Other Psychiatric Illnesses, k = 11; Unknown, k = 17). This variable also failed to have an impact on the magnitude of effect sizes (χ² = 5.64, p = 0.23). However, studies that excluded individuals with Attention Deficit/Hyperactivity Disorder (ADHD) (Excluded ADHD, k = 9; Did Not Exclude ADHD, k = 51) yielded a significantly smaller effect size estimate than those that did not exclude individuals with ADHD or were silent on ADHD exclusion (Excluded, d = −.27, Did Not Exclude, d = −.51; p = .009).

In line with Brewin and colleagues (2007), effect sizes were coded to indicate the strictness of exclusion criteria for traumatic brain injury (TBI) that were used in the studies as follows: (1) studies with no information about head injuries in their sample (Unspecified, k = 10); (2) studies that excluded “significant head trauma” from their sample (Significant Head Trauma Excluded, k = 17); and (3) studies that excluded all head injuries, including mild head injury, from their sample; (Mild Head Trauma Excluded, k = 33). Analyses showed no significant differences in effect size estimates between these three groups (χ² = 0.92, p = 0.34; Unspecified, d = −.39; Significant Head Trauma Excluded, d = −.45; Mild Head Trauma Excluded, d = −.50).

Demographic and IQ Variables

Including gender in a model with the neurocognitive test domains showed that for every 10 percent increase in men in the PTSD group, the magnitude of the effect size estimate (i.e., the difference between the groups) increased by .03 (β = −.003; p = .04), indicating greater performance discrepancy. Analyses revealed that the age of the PTSD group did not exhibit a significant effect on the magnitude of the effect size estimates (β = −.003, p = .274). A majority of the studies examined [k = 35; 58.3%] did not match PTSD and normal comparison groups on measures of premorbid IQ, and a variable representing the raw difference in IQ estimate between these groups was created for each study to examine the influence of difference in estimated IQ on the magnitude of effect size. Analysis of the variable reflecting IQ discrepancy revealed that it had a significant influence on study effect sizes (β = .24; p < .001), with greater neurocognitive performance differences associated with greater discrepancy in IQ between groups.

Models Testing Multiple Explanatory Variables

In models that simultaneously tested main effects from multiple explanatory variables, the treatment seeking status of the PTSD group (χ² = 6.72, p = 0.02), whether a study excluded participants with ADHD (β = −.19, p = 0.04), and IQ discrepancy (β = .16, p = .03) were all predictive of variance in the neuropsychological effect size estimates. In contrast, the proportion of men in the PTSD group (β = −.0005, p = .726) was not a significant explanatory variable in this model. This model reduced the between-study variance in effect size to .050 (SE = .013, p < .001). However, it should be noted that only 9 studies specifically indicated that the presence of ADHD was exclusionary, and this finding should therefore be interpreted cautiously.

Discussion

A large literature associates PTSD with structural and functional brain alterations and associated functional impairment, which are most often attributed to dysfunction in fronto-limbic circuitry. It has been hypothesized that alterations in this circuitry may also contribute to PTSD-associated neuropsychological deficits. The results of this meta-analysis generally support this contention. Despite significant variation in methods and samples, and even while modeling the correlations between effect sizes in each study, our analyses examining the cognitive outcomes associated with PTSD from emotionally neutral neurocognitive tests revealed an overall medium effect size (d = −.49). Moreover, these deficits were fairly consistent across the types of inciting trauma and were not statistically greater when studies used a trauma-unexposed group versus a trauma-exposed comparison group. On the other hand, our results also suggest that neurocognitive dysfunction is not an invariant feature of PTSD and varies by a number of important explanatory variables, as described below, including cognitive domain. Thus, significant deficits of a medium magnitude were observed in the cognitive processes of verbal learning (i.e., immediate memory) and delayed memory, complex information processing speed, attention/working memory, and executive functioning, while smaller effects were evident in language, visuospatial functioning, and visual learning and memory.

While our results share some similarities with neuropsychological models of anxiety and affective disorders, there are also some notable differences that highlight the disparate cognitive profiles of these disorders. For example, prior studies and meta-analyses in major depressive disorder have found a profile of episodic learning and memory findings that is somewhat discrepant from what we found in the current meta-analysis (Fossati et al., 1999; Lee et al., 2012; Porter et al., 2003; Wang et al., 2006; Zakzanis et al., 1998). Specifically, while we found effect sizes in verbal learning and memory to be significantly greater than effect sizes in visual learning and memory, these studies found significant visual learning and memory deficits in major depressive disorder, while non-significant or lesser effects were found in verbal learning and memory. Moreover, minimal effects were found in working memory in the meta-analyses of major depressive disorder, while we found some of the largest effects on attention/working memory. Lastly, the largest magnitude deficits in major depressive disorder are often found in executive functioning, while this domain was relatively more modest in our analysis. Therefore, the neurocognitive profile found in this meta-analysis does not appear to reflect general distress or psychopathology. Moreover, although some authors have posited that the cognitive deficits observed in PTSD are primarily due to depressive symptomatology in the context of PTSD (e.g., Burriss et al., 2008; Johnsen et al., 2008; Olff et al., 2014), this pattern of deficits (in combination with negative results in our analyses of the effect of depression on effect size variance) argues against this hypothesis.

Previous literature in PTSD provides a neurobiological framework that supports and parallels our findings. As an example, Kuhn and Gallinat (2013) recently performed a meta-analysis of MRI whole brain voxel-based morphometry results in PTSD and discovered significant clusters of reduced gray matter density in anterior cingulate cortex, ventromedial prefrontal cortex, left hippocampus, and left temporal pole/middle temporal gyrus. While these structures are involved in fear processing, emotion regulation, and memory encoding and retrieval, they also comprise interconnected brain networks that support broad cognitive constructs such as attentional switching, working memory, and speed of information processing (Bressler & Menon, 2010). Thus, such structural changes could impact both emotion processing and cognitive functioning in PTSD, although tests of the associations between brain structure and performance on neurocognitive tests have been notably lacking.

Results from functional neuroimaging studies have complemented these findings to examine the functional implications of PTSD-associated brain dysfunction. Results from two recent meta-analyses of functional neuroimaging studies demonstrated that individuals with PTSD evidence hyperactivity within networks that activate in response to salient stimuli, including regions such as the amygdala and dorsal anterior cingulate (Hayes, Hayes, & Mikedis, 2012; Patel et al., 2012). Furthermore, this excess activity is present even while the subject is ‘at rest’ (i.e., not presented with stimuli) when neuroimaging data are collected (Sripada, King, Garfinkel, et al., 2012; Sripada, King, Welsh, et al., 2012). In combination, these results suggest that individuals with PTSD may have exaggerated attention to extraneous but subjectively salient stimuli, which may reflect a pervasive underlying state. Since this network appears to be responsible for efficient switching between other large-scale brain networks (e.g., Menon, 2011), such as those involved in emotionally neutral cognitive functioning (e.g., executive control networks), it is not surprising that excess activity in this network has been associated with diminished performance on cognitive tasks in PTSD (Morey et al., 2009; Zhang et al., 2013). In addition, when individuals with PTSD perform cognitive tasks while undergoing functional neuroimaging, studies have reliably found hypoactivity in networks involved in working memory, cognitive control, planning, and emotion regulation (e.g., lateral prefrontal cortex) in individuals with PTSD (Hayes et al., 2012; Patel et al., 2012), Emerging evidence also supports disrupted connectivity between these regions and those involved in salience detection and internally focused thought in PTSD (Daniels et al., 2010; Sripada, King, Welsh, et al., 2012). Taken together, these results and our data provide support for models of cognition in PTSD that emphasize dysregulated arousal and salience detection combined with disrupted functional connectivity between the prefrontal cortex and limbic system (Brown & Morey, 2012; Rauch et al., 2006; Sripada, King, Welsh, et al., 2012).

Origin of Neurocognitive Findings in PTSD

Our data cannot help determine whether the neurocognitive deficits observed in PTSD samples are a consequence of the disorder, constitute preexisting vulnerabilities, or reflect the interaction of both. A competing framework to the interpretation that cognitive deficits result from PTSD comes from studies of Vietnam veteran twin pairs (Pitman et al., 2006). In a series of studies, investigators from the Harvard/VA PTSD Twin Study examined two groups of identical twin participants: Vietnam combat veterans with PTSD and their identical twins without combat exposure or PTSD; and Vietnam combat veterans without PTSD and their identical twins without combat exposure or PTSD. By comparing these four groups, investigators hoped to differentiate factors that were resultant from versus predictive of combat exposure and PTSD. Results showed that some cognitive deficits in memory and executive functions (Gilbertson et al., 2006) and some but not all morphometric brain findings (Gilbertson et al., 2002; Kasai et al., 2008) in PTSD may have existed prior to a trauma and represent a vulnerability factor contributing to the development of PTSD. Although these studies provide evidence that certain neurocognitive factors that enhance vulnerability for PTSD may be familial, this evidence does not exclude the possibility that cognitive abilities could be worsened by neurobiological changes associated with PTSD. To this end, Vasterling & Brailey (2005) proposed that pre-trauma neurocognitive dysfunction may increase the risk of developing PTSD (perhaps by impacting one’s ability to effectively implement coping strategies post-trauma), but cognitive functioning may also be impaired by the development of PTSD.

In other words, subtle cognitive weaknesses that exist prior to a trauma may progress to mild cognitive deficits as a result of alterations in neural circuitry that occur with the onset of PTSD. In order to determine the precise origin of the neurocognitive dysfunction associated with PTSD, pre- to post-trauma longitudinal data are essential (Gilbertson et al., 2006), and a few studies have been informative in this regard. A number of studies have shown that performance on military aptitude tests, which were collected before any military trauma could occur and are considered measures of intelligence, are predictive of the development of PTSD, even after adjusting for combat exposure (e.g., Gale et al., 2008; Macklin et al., 1998), although this effect may diminish with higher levels of combat exposure (Thompson & Gottesman, 2008). Similarly, Parslow and Jorm (2007) found that greater pre-trauma cognitive performance in working memory, verbal episodic memory, and processing speed were correlated with PTSD symptoms after exposure to a natural disaster. However, significant PTSD symptoms subsequent to the disaster were also associated with detrimental effects on measures of verbal immediate and delayed recall. Longitudinal studies in which soldiers have undergone neuropsychological performance assessments both before and after deployment have also revealed interesting, though complex, results. Marx and colleagues (2009) showed that visual immediate recall performance measured before deployment was associated with severity of PTSD measured after deployment, although this effect was strongest in individuals with higher levels of pre-deployment PTSD symptoms. In addition, individuals who developed PTSD after deployment also demonstrated further declines in visual episodic memory. Similar studies have shown that both war zone deployment and PTSD symptoms are significantly related to declines in speed of information processing, sustained attention, and episodic memory, even after accounting for TBIs that occurred during deployment (Vasterling et al., 2006, 2012). Taken together, converging data support the assertion that certain aspects of neurocognitive dysfunction are both risk factors for and consequences of PTSD, although further specification of these relationships are clearly needed.

Specific Findings within Neurocognitive Domains

Dysfunction in emotionally neutral episodic memory has been studied extensively in PTSD, and it has been suggested that difficulties in encoding and retrieval are primarily responsible for the observed memory deficits in PTSD (Golier et al., 2006; Vasterling et al., 1998). In support of this hypothesis, a slightly larger effect was observed on measures of verbal learning than delayed recall, suggesting that difficulties in verbal encoding (and perhaps retrieval) rather than consolidation (i.e., retention) difficulties underlie the overall episodic verbal memory deficit in PTSD. Although previous studies have shown associations between verbal memory performance and hippocampal volumes (Bremner, Randall, Scott, Bronen, et al., 1995), as well as hippocampal activation with PET during verbal episodic memory tasks (Bremner et al., 2003; Kitayama et al., 2005), studies have not reliably found associations between hippocampal volume reductions and verbal memory impairment in PTSD (Bremner et al., 1997; Lindauer et al., 2006; Neylan et al., 2004; Stein et al., 1997; Woodward, Kaloupek, et al., 2009). For example, a well-powered recent study (Woodward, Kaloupek, et al., 2009) found relatively modest correlations between the volumes of memory-relevant brain regions, including the hippocampus and parahippocampal regions, and episodic memory performance in PTSD. Combined with our significant difference in immediate versus delayed verbal memory, these results lend support to a model of PTSD-associated episodic memory deficits in which fronto-limbic (e.g., strategic verbal encoding) dysfunction may play are latively greater role than mediotemporal systems.

Interestingly, our results showing a significantly greater effect on verbal learning and memory than on nonverbal (i.e., visual) learning and memory are concordant with a previous meta-analysis of memory in PTSD (Brewin et al., 2007). A number of hypotheses have been advanced to address this discrepancy. Some have speculated that lateralized neural dysfunction in PTSD might help explain the relative sparing of visual memory in PTSD (e.g., Vasterling & Brailey, 2005), including relative reductions in left hippocampal gray matter density (Kühn & Gallinat, 2013). Others have pointed to findings highlighting the overall separation of verbal and visual processing in PTSD (e.g., dual representation theory; Brewin, 2001) and proposed that the prominence of certain symptoms, such as flashbacks and vivid emotional memories, suggest that visual processing and image-based memory systems are relatively intact in PTSD. Given the divergence from findings in the depression literature described above, this will be an interesting area for future study. It should be noted, though, that the parameters of the tests used to assess learning and memory may have subtly influenced the observed differences between verbal and visual memory. However, the fact that we examined standardized neuropsychological tests of visual learning and memory that predominantly evidence comparable reliability, validity, and sensitivity to verbally based tests helps to diminish this concern (Brewin et al., 2007).

Although a handful of authors have emphasized the relevance of speed of information processing in PTSD (Samuelson et al., 2006; Twamley et al., 2009; Woodward, Kaloupek, et al., 2009), there has been little direct exploration of this cognitive domain in the PTSD literature. Processing speed may have been relatively ignored previously because authors have often classified these tests (e.g., Trail making Test, Part A; WAIS Digit Symbol) as assessing attention. While attention deficits can contribute to slower processing speed and the two constructs exhibit functional anatomical overlap, factor analytic studies support the separation of speed of information processing from attention in both healthy persons (e.g., Tulsky & Price, 2003) and those with neurological or neuropsychiatric illness (e.g., Park et al., 2012; Schretlen et al., 2013). Interestingly, we found processing speed to have the second largest effect size discrepancy of any domain between individuals with PTSD and healthy comparison groups. The reasons for these deficits are unclear at the present time, although a range of factors associated with PTSD could contribute to slowed processing of information, including sleep alterations or deprivation (Fernandez-Mendoza et al., 2010), hyperarousal (Shucard et al., 2008), or reduced processing resources to devote to the intended task because of attention to internal or external stimuli (Morey et al., 2009). Of particular clinical relevance, our results suggest that some individuals with PTSD may have mild processing in efficiencies, which may have important implications for optimizing the effectiveness of psychotherapeutic interventions. Future studies in PTSD patients should examine the impact of inefficient processing on performance in other neurocognitive domains and associated functional outcomes, such as treatment implementation and understanding.

Primary symptoms of PTSD include difficulties with attention and concentration, and many symptoms of PTSD have been conceptualized within an attentional framework (e.g., attentional bias, hypervigilance; Esterman et al., 2013). Our meta-analysis showed that individuals with PTSD displayed moderate deficits on laboratory tasks of attention and working memory. It has been hypothesized that this effect may depend on the type of task employed, such that basic attention abilities are unaffected, while more pronounced deficits emerge with increasingly complex processing demands. These deficits may be due to PTSD-associated arousal dysregulation, disinhibition, or attentional capture, all of which can disrupt goal-directed attention. To this end, individuals with PTSD have been shown to display intrusive errors and errors of commission on tasks of complex attention, which have been related to symptoms of hyperarousal (Daniels et al., 2010; Vasterling et al., 1998). Thus, attention deficits may also be most apparent when working memory, inhibitory function, and sustained attention are taxed (e.g., with an N-back task), although further parsing of attentional functioning in PTSD awaits future study.

Recent work has also highlighted the relevance of executive functions in PTSD (Aupperle, Melrose, et al., 2012). Many studies of PTSD have focused on difficulties with inhibition, attentional switching, and flexibility, which appear to show the most consistent results in the literature (Casada & Roache, 2005; Koenen et al., 2001; Leskin & White, 2007; Vasterling et al., 1998) and may be related to the difficulty individuals with PTSD experience in disengaging from certain salient stimuli (Pineles, Shipherd, Mostoufi, Abramovitz, & Yovel, 2009). These specific deficits are consistent with findings from functional neuroimaging research in PTSD, which points to altered prefrontal network activity with tasks requiring inhibition and attentional switching (e.g., Bryant et al., 2005; Falconer, Bryant, et al., 2008). However, the effect sizes found within this domain are somewhat smaller than might be expected given the previous research examining executive functions in PTSD. One possible explanation for this observation is that collapsing measures of concept formation and problem solving, such as the Wisconsin Card Sorting Test, into one domain with measures of inhibition and attentional switching may obscure more prominent effects, as measures of concept formation, planning, and problem solving appear to be mostly unaffected in PTSD (Aupperle, Melrose, et al., 2012; Twamley et al., 2009; Vasterling et al., 1998).

The moderate language deficits demonstrated in PTSD participants may be partially explained by the information processing speed and executive deficits described above. Our language domain predominantly contained measures of verbal fluency, which require individuals to generate words under time constraints. Adequate performance on these tasks relies on the efficiency of executive and speeded processes, including rapid, rule-guided search, retrieval, switching, and production abilities, as well as the integrity of lexicosemantic memory stores. Considering the hypothesized fronto-limbic dysfunction associated with PTSD, it may be that the language deficit observed in our meta-analysis reflects problems with executive control of search and retrieval strategies or slowed information processing, rather than degraded semantic memory stores.

Few studies have previously examined visuospatial processing in PTSD. Unfortunately, in this meta-analysis this domain consisted almost entirely of effect sizes from studies that used the copy trial from the Rey Complex Figure. Perhaps related to this finding, work from Gurvits and colleagues (2000, 2002, 2006) has shown that individuals with PTSD exhibit deficits in the visuospatial copying of simple three-dimensional figures. The authors interpreted these deficits as neurodevelopmental in nature and indicated that they likely serve as a vulnerability factor for the development of PTSD. Whether the visuospatial deficits observed in our analyses are related to executive dysfunction (e.g., planning), perceptual organizational impairment, neurodevelopmental vulnerability, or a combination of these factors remains to be determined by future studies.

Clinical and Comorbidity Factors

A number of specific clinical factors deserve consideration in the interpretation of cognitive findings in the PTSD literature, including treatment seeking status, psychiatric comorbidity, and history of head injury. We examined these factors as explanatory variables in our analyses to investigate their contribution to effect size estimates in the cognitive PTSD literature.

A notable and robust finding in this study was that samples of study participants that were seeking or undergoing treatment for PTSD evidenced significantly larger effects size estimates than samples of individuals with PTSD recruited from the community and samples that combined both community and treatment seeking individuals. Although the proportions of the two latter groups that were receiving treatment were largely unknown, they were likely much lower than those specifically presenting for treatment. Compared to individuals with PTSD who are not undergoing treatment, individuals seeking or undergoing treatment may have more severe PTSD symptoms, greater medical and psychiatric comorbidity, and a greater likelihood of having a longer illness duration, all of which may result in a greater likelihood of cognitive deficits (Horner & Hamner, 2002). Of particular clinical relevance, it may be that individuals with PTSD presenting for treatment are most likely to exhibit cognitive deficits, which could have implications for treatment implementation, adherence, and outcomes. Neuropsychological functioning has clear relevance for certain empirically-validated treatments for PTSD that rely on efficient learning and processing of new information, such as cognitive processing therapy. In fact, PTSD patients with poorer performance in certain cognitive abilities, such as episodic memory and inhibitory control, have been shown to have worse treatment outcomes in cognitive-behavioral therapy for PTSD (Falconer et al., 2013; Wild & Gur, 2008), although additional research is clearly needed in this regard.

Although our analysis of PTSD symptom severity did not reveal a significant influence on overall neurocognitive effect size estimates, this analysis was hindered by incomplete CAPS score data (k = 21; 35.6%) and may be limited by a study-level versus individual-level analysis. Since most studies that have examined correlations between the severity of PTSD symptoms and neurocognitive performance have found significant associations (Bremner et al., 1993, 2004; Cohen et al., 2013; Gilbertson et al., 2001; Lindauer et al., 2006; Olff et al., 2014; Twamley et al., 2009; Vasterling et al., 2002, 1998), we performed post hoc analyses to examine whether PTSD symptom severity might contribute to the magnitude of effect size estimates within specific cognitive domains. In general, although results have been variable, measures of immediate verbal memory, speed of information processing, sustained attention, and working memory appeared to have the most consistent correlations with PTSD symptoms. Thus, we examined whether PTSD symptom severity might contribute to the variance in effect sizes within verbal learning, speed of information processing, or attention/working memory domains. Results revealed that severity of PTSD symptoms was associated with the magnitude of effect size estimates within the verbal learning domain, but not within the other domains. Thus, overall PTSD symptom severity may be more associated with verbal learning deficits than other neurocognitive domains. The reasons for this specificity are unclear, although it is possible that specific clusters of PTSD symptoms (e.g., hyperarousal) might have higher associations with performance in other neurocognitive domains (e.g., attention) than total severity (Olff et al., 2014; Vasterling et al., 1998).

Comorbidity and Medications

Previous research has documented the high comorbidity of PTSD with other psychiatric disorders and both alcohol and drug use disorders (e.g., Kessler et al., 1995; Scherrer et al., 2008), and studies and reviews of cognition in PTSD have often discussed the potential confound these disorders may represent for study findings (Barrett et al., 1996; Danckwerts & Leathem, 2003; Horner & Hamner, 2002; Samuelson et al., 2006). Although our results regarding treatment-seeking samples may speak indirectly to this possibility, our results directly addressing these questions were mixed. We found no significant effects of comorbid depressive disorders, although reporting of these data was not uniform across studies. In addition, we found that the strictness of exclusion criteria for psychiatric comorbidities (e.g., depression, anxiety) did not significantly influence effect size estimates. Similarly, our variables reflecting the percentage of participants in the PTSD groups with alcohol or drug use disorders did not have an appreciable influence on effect size estimates. This lack of effect was surprising because meta-analytic studies have shown that chronic, sustained use of alcohol and other substances can result in cognitive deficits, some with larger magnitude effects than those reported here (e.g., Chapman, Byas-Smith, & Reed, 2002; Jovanovski et al., 2005; Scott et al., 2007; Stavro, Pelletier, & Potvin, 2013). However, this equivocal effect is nonetheless consistent with one previous study showing a lack of interaction between PTSD and alcohol abuse on neuropsychological test results (Samuelson et al., 2006). In addition, this analysis should be treated with caution because only 30 studies (50%) provided information about alcohol or substance use disorders. Our lack of findings may also partially reflect the methodological limitations of analyzing these variables at the study versus individual level. Future studies should carefully assess and report the presence of alcohol and drug use disorders in their samples to provide greater confidence in the interpretation of their results.

In contrast, the exclusion of individuals with ADHD did exert a significant influence on effect size estimates. ADHD is a neurodevelopmental disorder that can persist into adulthood and is conceptualized as resulting from dysfunction of dopaminergic and noradrenergic systems (Biederman & Faraone, 2005), implicating cognitive and behavioral dysfunction characteristic of an underlying frontal-striatal pathophysiology (Nigg, 2005). ADHD in adults has been associated with neurocognitive deficits in sustained attention, new learning of information, and executive functions (Hervey et al., 2004). Thus, as might be expected, studies included in the meta-analysis that specifically excluded participants with ADHD diagnoses evidenced an overall effect size that was significantly less than those from studies that either included subjects with ADHD or were silent regarding this exclusion. Although these studies may have simply had less stringent exclusion criteria, for which the lack of ADHD exclusion served as a proxy, these samples could also have included individuals with unrecognized ADHD (Barkley & Brown, 2008), particularly considering the moderately high comorbidity rates between PTSD and ADHD (Atshel et al., 2013; Harrington et al., 2012). However, since we do not know how many individuals with ADHD might be included in such studies, this finding should be considered preliminary and awaits further study. Interestingly, a recent study directly compared neuropsychological functioning in individuals with comorbid ADHD and PTSD to those with ADHD alone, finding that PTSD conferred additional cognitive deficits in working memory, speed of information processing, and visuospatial processing (Antshel, Biederman, Spencer, & Faraone, 2014). Since ADHD has been proposed as a vulnerability factor for the development of PTSD (Adler et al., 2004; Biederman et al., 2013; Gurvits et al., 2006), future neurocognitive and neuroimaging studies in PTSD should carefully consider the primary aims of the study when deciding whether to exclude individuals with ADHD. For example, if concerns about generalizability of findings are paramount, then including individuals with ADHD could be appropriate. However, if the primary aim of the study is to isolate neurocognitive or neurobiological findings associated with the development of PTSD, including individuals with ADHD might represent a significant confound.

It has also been suggested that studies examining neurocognitive effects associated with PTSD might have confounded results because the authors did not appropriately account for the effects of head injuries (Vasterling & Brailey, 2005). Because individuals with PTSD are more likely to have TBIs than healthy control samples (McAllister & Stein, 2010), the greater cognitive deficits observed in PTSD may be attributable to TBI instead of PTSD. We attempted to investigate this possibility by coding a variable reflecting the strictness of a study’s exclusion criteria for TBI. The magnitude of effect sizes for the levels of TBI exclusion criteria did not show a discernable increase across levels of exclusionary stringency, and the variable did not have a significant effect on effect sizes. Although this finding was somewhat surprising, it is consistent with a prior meta-analysis of episodic memory in PTSD (Brewin et al., 2007) and with a growing literature highlighting limited long-term cognitive deficits in a vast majority of patients with mild TBI (Belanger et al., 2010; Moser et al., 2007; Rohling et al., 2012; Vasterling et al., 2012), who are those most likely to be included in these studies. However, findings might also reflect the coarseness of the coded variable (i.e., this variable did not capture the actual proportion of subjects with specific TBI severity). Taken together with previous findings, our results indicate that evidence for TBI contaminating cognitive findings in the current PTSD literature is weak, although TBI is clearly important to document and examine in the context of PTSD research (Bryant, 2011).

Information regarding medication use was not provided for many (k = 24; 40.0%) studies, even though certain medications that are commonly prescribed for PTSD (e.g., benzodiazepines) have clear effects on neurotransmission and detrimental effects on cognition (Barker, Greenwood, Jackson, & Crowe, 2004; Barker et al., 2005). Our analyses did not find a significant effect of medication exclusion criteria on neurocognitive performance in PTSD, although this variable was coded dichotomously to simply reflect whether studies excluded any psychoactive medication use for at least two weeks prior to the assessment, which does not reflect chronicity of use or the variability in classes of medications (e.g., exclusion of benzodiazepines versus antidepressants). Notably, several studies that were the most conservative regarding psychotropic medication exclusion nonetheless showed neuropsychological performance deficits (Flaks et al., 2014; Geuze et al., 2009; Gilbertson et al., 2001; Golier et al., 1997; Lindauer et al., 2006; Yehuda et al., 1995), although residual performance deficits that remain even after a medication washout period cannot be excluded. In contrast, evidence from two longitudinal studies has shown that treatment with the selective serotonin reuptake inhibitor (SSRI) paroxetine not only reduces PTSD symptom severity but also increases hippocampal volumes and improves verbal memory (Fani et al., 2009; Vermetten, Vythilingam, Southwick, Charney, & Bremner, 2003). Future studies would benefit from more explicit exploration of the possible beneficial and detrimental effects of psychotropic medication use on cognition and brain function in PTSD.

Demographic Factors

A substantial body of research has indicated that greater intellectual resources may protect against the development of PTSD (Breslau et al., 2006; Macklin et al., 1998; McNally & Shin, 1995; Vasterling, Brailey, Constans, Borges, & Sutker, 1997), and some authors have suggested that limited premorbid intellectual resources may be partially responsible for cognitive deficits in individuals with PTSD (Bustamante et al., 2001; Gilbertson et al., 2006). To address these concerns, we constructed difference scores for discrepancies in estimated IQ to determine whether patients poorly matched to a healthy comparison group on these variables varied systematically in neuropsychological performance. Our analyses showed that discrepancies in IQ estimates between the PTSD and healthy comparison groups significantly influenced effect sizes. Although a majority of studies statistically controlled for IQ in their analyses when discrepancies were present, we nonetheless found that IQ discrepancy can represent a significant confound when the literature is examined as a whole. Thus, to truly isolate brain or behavior correlates of PTSD, alternative analytical or modeling approaches may be warranted.

Analysis of gender revealed that it had a relatively minor, but nonetheless significant, influence on the magnitude of the PTSD-associated effect sizes. Specifically, studies that had a larger proportion of men in the PTSD sample also had greater overall levels of neuropsychological deficits. It should be noted, however, this effect was generally small (β = −.003) and may lack clinical significance. The reasons for this effect are unclear, as few studies have examined potential gender differences in neuropsychological or neurobiological findings in the PTSD literature. It is possible that this effect is confounded with studies of veterans, although our findings regarding trauma type do not reflect such differences in effect sizes. In contrast, the mean age of the PTSD group did not exert a significant influence of the magnitude of effect size estimates. This result was surprising, as normal aging is associated with structural and functional changes in prefrontal systems (e.g., Mielke et al., 1998), which are often accompanied by cognitive decline (e.g., Craik & Bialystok, 2006). Moreover, prior research has reported that normal aging leads to subtle additive cognitive effects in PTSD (Yehuda, Golier, Tischler, et al., 2005; Yehuda, Golier, Harvey, et al., 2005), although other recent research contradicts these findings (Jelinek, Wittekind, Moritz, Kellner, & Muhtz, 2013). It has been suggested that older individuals with PTSD who participate in research may represent an especially resilient group, as they typically are required to be physically healthy and have minimal risk of cognitive decline, which may help explain the variability of findings in this population (Jelinek et al., 2013).

Small Sample Effects

Although meta-analyses can produce useful estimates of neuropsychological deficits associated with particular disorders by quantitatively synthesizing results across the published literature, they are not exempt from bias (Matt & Cook, 2009). It is widely acknowledged that studies with small sample sizes that are published in the research literature are likely to show larger effects than larger studies, which can lead to “small study effects” in meta-analyses (Egger et al., 1997). A number of factors can lead to small study effects (e.g., Sterne et al., 2011; Sterne, Gavaghan, & Egger, 2000), including heterogeneity of the studies included. For example, there may be differences in the settings, methodologies used, or clinical characteristics of the samples between studies, which may be associated with variance in effect size. Another potential cause of small study effects is publication bias, which refers to the greater tendency for statistically significant results to be published (Dwan et al., 2008; Song, Eastwood, Gilbody, Duley, & Sutton, 2000). Despite the smaller samples that are typical of this literature, no prior meta-analysis has examined small sample bias in neuropsychological studies of PTSD.

Our analyses revealed potential small study effects in the available literature examining PTSD and cognition, although explanations for this small study bias were inconclusive. Studies that either excluded all psychiatric comorbidity or only allowed depressive disorders had less evidence of funnel plot asymmetry than those that allowed their samples to have more comorbid psychiatric disorders. Thus, studies with more strict exclusion criteria regarding comorbid psychiatric disorders were less likely to introduce small study bias. Studies with less stringent exclusion criteria may have yielded larger effects with smaller samples because of diagnostic contamination, or they may have included more symptomatic patients with greater psychiatric and medical comorbidity, which can result in increased neurocognitive deficits.

It is also possible that publication bias may have contributed to the observed small study effects and asymmetrical funnel plots. The publication process, along with the difficulty of recruiting a sample of research subjects representative of the intended population, introduces biases that may lead to an overestimation of effect sizes. Factors that could lead to publication bias in this literature include selective outcome reporting, selective analysis reporting, the reduced likelihood of publishing equivocal neuropsychological results with smaller samples, the greater pressure to publish large-scale studies, and reduced incentives for authors to pursue publication of equivocal findings because of the potential “unimportance” of cognitive outcomes in PTSD. Thus, although our results provide a valuable synthesis of the data on PTSD and neurocognitive functioning that is publicly available in the literature, whether they reflect the larger volume of studies on this topic and the true population effect sizes is less clear. Therefore, although the results of this meta-analysis are informative, they should be interpreted with caution.

To provide a potentially informative correction for small study effects, we also applied Duval and Tweedie’s (2000) “trim and fill” method, which adjusts the analyses to insert the “missing” effect sizes in an asymmetrical funnel plot. This analysis generated an adjusted mean effect size estimate that was still significant but diminished by approximately 29% from the original potentially biased estimate (d = −.49). Although this analysis potentially decreases the clinical significance of these findings, the methods are data augmentation techniques that provide estimates and are by no means conclusive. It should also be emphasized that even mild neuropsychological impairments are often associated with clinically significant functioning difficulties (Dikmen et al., 2009), as more complex processing demands occur in the “real world” than in the laboratory/clinic due to, among other factors, environmental contingencies and demands (e.g., distraction; Marcotte, Scott, Kamat, & Heaton, 2009). Reinforcing this notion, Geuze and colleagues (2009) showed that memory deficits, though mild, accurately predicted current social and occupational functioning in a sample of veterans with PTSD.

Limitations and Future Directions

Although we found effect size discrepancies between individuals with PTSD and those without a diagnosis of PTSD across a broad range of neurocognitive domains, a limitation of this literature is the scarcity of data concerning whether individuals with PTSD exhibit cognitive impairment when test results are compared to normative standards (Mackin, Lesselyong, & Yaffe, 2012; Twamley et al., 2009). Although individuals with PTSD may exhibit statistically significant differences in neuropsychological measures when compared to a control group, the scores of those with PTSD may nonetheless fall within the normal range of performance when compared to an appropriate normative data set (e.g., Gilbertson et al., 2001), which may reduce the clinical significance of study findings. However, it should also be noted that normative comparisons do not signify potential individual decline, and scores that reflect low average performance normatively may nonetheless be distressing for an individual with higher pre-trauma cognitive functioning. Thus, future studies could add valuable data on the clinical significance of neurocognitive findings by not only reporting the statistical significance of group comparisons but also examining the contribution of cognitive deficits to functional decline and comparing individual scores to available normative data.

Our results also should be considered in light of the limitations of neuropsychological meta-analyses in general. The range of neuropsychological tests that are administered is highly variable both across and within studies. Although most tests purport to measure a specific domain of neurocognitive functioning, they also frequently involve multiple cognitive skills. For example, attention is a fundamental cognitive process that, if impaired, can significantly impact performance in other domains of functioning (Lezak, Howieson, & Loring, 2004), which could lead to diagnostic imprecision and error in determining underlying mechanisms. Another problem is that assigning mean neurocognitive effect sizes into cognitive domains is likely to provoke some degree of controversy, as no consensus exists regarding the domain to which certain tests should be assigned. The classification of tests within particular neurocognitive domains is also limited by the data provided by investigators. Specific to this meta-analysis, even though attention is not a unitary construct, we combined attention and working memory into one domain because articles often only provided summary indices for measures that separately assessed these two constructs (e.g., WAIS digit span). Prospective studies and future meta-analyses should contribute to further characterization of these neurocognitive domains in PTSD.

A number of factors that could affect interpretation of these results deserve analysis. Only three studies used symptom validity tests to examine the influence of effort on neuropsychological test performance (Sullivan et al., 2003), despite indications of their importance in psychiatric populations (e.g., Schroeder & Marshall, 2011; Wisdom et al., 2013), especially those with potential secondary gain (Demakis et al., 2008; Heilbronner et al., 2009; c.f. Barrash et al., 2007). However, it should also be noted that concerns regarding cognitive symptom validity might vary by the context (i.e., research versus clinical) of the evaluation (McCormick, Yoash-Gantz, McDonald, Campbell, & Tupler, 2013). Future studies should consider the influence of symptom validity/effort on neurocognitive test performance in PTSD, especially in veterans with comorbid TBI and individuals with potential secondary gain (Howe, 2009; Lange, Pancholi, Bhagwat, Anderson-Barnes, & French, 2012). In addition, despite the high comorbidity of PTSD and substance use disorders, only four studies reported use of urine toxicology or breathalyzer examinations to screen for acute intoxication or recent substance use, both of which can affect neuropsychological test performance. Future studies should routinely incorporate both of these measurements in their study design, as they require minimal investment on the part of the investigator. Lastly, most subjects in these studies were younger or middle-aged adults, so caution is warranted in generalizing these results to children or older adults.

As mentioned above, although chronicity of PTSD would seem to be a critical explanatory variable in these analyses and has shown some relationship to cognitive (e.g., Emdad, Söndergaard, & Theorell, 2005b) and neurobiological (e.g., Felmingham et al., 2009) outcomes in prior studies, only four studies reported the duration of illness for their participants (Cottencin et al., 2006; Emdad, Söndergaard, & Theorell, 2005a; Lindauer et al., 2006; Moores et al., 2008), precluding the inclusion of this variable. A few studies of individuals with recent trauma and PTSD symptomatology have shown that attention deficits are more prominent than memory dysfunction in acute PTSD (Brandes et al., 2002; Elsesser & Sartory, 2007), although other cognitive domains have not been assessed in this context. Moreover, though age at traumatization was rarely reported in the available studies, the timing of trauma could significantly influence cognitive functioning in PTSD and could be an appropriate topic for future research. For example, if trauma occurs in the context of a developing brain, it is possible that pathophysiological mechanisms associated with PTSD could result in divergent neurobehavioral outcomes compared to traumatic exposure in adulthood, when the brain has significantly slowed its maturation.

Finally, because publication bias may have influenced the effect size estimates, future studies examining cognitive functioning in PTSD and related conditions would benefit from clearer reporting standards. Given the heterogeneity in reporting of even basic sociodemographic (e.g., education) and psychiatric (e.g., depression) data, studies would greatly benefit from detailed reporting of inclusion/exclusion criteria, clinical and cohort characteristics, and reporting of data for all planned analyses. Moreover, future meta-analyses in this research area would benefit from examining unpublished data to avoid the “file drawer” problem in meta-analysis (Matt & Cook, 2009).

Summary and Conclusions

Results of our meta-analysis indicate that PTSD is associated with neurocognitive deficits of a medium magnitude in verbal learning and memory, attention/working memory, and processing speed, and smaller deficits in executive functions, language, visual learning and memory, and visuospatial abilities. This pattern of deficits is broadly consistent with dysfunction in the fronto-limbic networks implicated in the pathophysiology of PTSD. However, neurocognitive deficits are not an invariant feature of PTSD, and a number of additional sociodemographic and clinical variables also contributed to the variance in effect size estimates, including gender, treatment-seeking status, ADHD exclusion criteria, and discrepancies in IQ between the samples in studies. Our results also highlight methodological limitations in the literature, including the presence of small study bias, the relative absence of cognitive symptom validity/performance validity assessments, and the frequent mismatch of subject groups on premorbid intelligence estimates. Although the cognitive deficits observed were significant even after adjusting for small study effects, they were appreciably reduced and might be best appreciated as subtle within all but the largest magnitude cognitive domains (i.e., attention/working memory, verbal learning and memory, and information processing speed). Thus, the size of the deficits reported here should not be interpreted in absolute terms, although the overall profile of deficits is likely less affected by these small study effects, as there is little reason to suspect that any particular cognitive domain is more susceptible to small study bias than any other.

Clinically, our findings emphasize that individuals seeking treatment for PTSD are those most likely to exhibit cognitive deficits, indicating that consideration of neuropsychological functioning has important implications for the clinical management of persons with PTSD. For example, regardless of the origin of cognitive deficits, “fine-tuning” PTSD treatments to match the cognitive functioning of specific patients may help increase the effectiveness of treatment. Moreover, these results highlight a pattern of cognitive deficits that could provide novel information for the design and implementation of treatments for patients with PTSD, particularly our finding of moderate PTSD-associated deficits in speed of information processing. Clearly, additional intervention research is needed to understand the potential effects of cognitive deficits on the implementation of specific PTSD treatments. Moreover, structured cognitive remediation training has shown some efficacy in improving cognition and functional outcomes in individuals with TBI (e.g., Cicerone et al., 2011), depression (Bowie et al., 2013), and severe mental illness (e.g., McGurk, Twamley, Sitzer, McHugo, & Mueser, 2007; Twamley, Vella, Burton, Heaton, & Jeste, 2012) and may therefore may be appropriate to evaluate for remediation of attention, memory, and processing speed deficits in individuals with PTSD. Our analyses of explanatory variables also point to the importance of examining specific study characteristics and how they may match with the patient being treated when considering the potential impact of cognition on the manifestation and treatment of PTSD. Future studies should consider the interplay of these factors when designing mechanistic studies of PTSD to enhance understanding of the neurobiological effects of traumatic stress.