Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Feb 1.
Published in final edited form as: Cogn Affect Behav Neurosci. 2021 Aug 26;22(1):199–213. doi: 10.3758/s13415-021-00943-4

Value Estimation and Latent-State Update-Related Neural Activity During Fear Conditioning Predict Posttraumatic Stress Disorder Symptom Severity

Allison M Letkiewicz 1, Amy L Cochran 2, Anthony A Privratsky 3, G Andrew James 3, Josh M Cisler 4
PMCID: PMC8792199  NIHMSID: NIHMS1747963  PMID: 34448127

Abstract

Learning theories of posttraumatic stress disorder (PTSD) purport that fear learning processes, such as those that support fear acquisition and extinction, are impaired. Computational models that aim to capture specific processes involved in fear learning have primarily assessed model-free, or trial-and-error, reinforcement learning (RL). Although previous studies indicate that aspects of model-free RL are disrupted among individuals with PTSD, research has yet to identify whether model-based RL, which is inferential and contextually driven, is impaired. Given empirical evidence of aberrant contextual modulation of fear in PTSD, the present study sought to identify whether model-based RL processes are altered during fear conditioning among women with interpersonal violence (IPV)-related PTSD (n=85) using computational modeling. Model-free, hybrid, and model-based RL models were applied to skin conductance responses (SCR) collected during fear acquisition and extinction, and the model-based RL model was found to provide the best fit to the SCR data. Parameters from the model-based RL model were carried forward to neuroimaging analyses (voxel-wise and independent component analysis). Results revealed that reduced activity within visual processing regions during model-based updating uniquely predicted higher PTSD symptoms. Additionally, after controlling for model-based updating, greater value estimation encoding within the left frontoparietal network during fear acquisition and reduced value estimation encoding within the dorsomedial prefrontal cortex during fear extinction predicted greater PTSD symptoms. Results provide evidence of disrupted RL processes in women with assault-related PTSD, which may contribute to impaired fear and safety learning, and, furthermore, may relate to treatment response (e.g., poorer response to exposure therapy).

1. Introduction

Posttraumatic stress disorder (PTSD) is an impairing anxiety-related disorder that is marked by deficits in several aspects of learning (Jovanovic et al., 2010; Jovanovic, Kazama, Bachevalier, & Davis, 2012; Pacella, Hruska, & Delahanty, 2013). For example, individuals with PTSD exhibit overgeneralization of conditioned fear from threat-related cues to approximations of threat-related cues that are safe (Kaczkurkin et al., 2017; Lopresto, Schipper, & Homberg, 2016), impaired inhibition of previously learned fear associations, and impaired recall of extinction learning (for a review, see Lissek & van Meurs, 2015). Given the ubiquity of learning deficits in PTSD, identifying specific learning-related processes that are disrupted is an important research endeavor.

Computational methods have enhanced researchers’ ability to circumscribe cognitive processes with greater sensitivity and precision (Stephan & Mathys, 2014; Price, Brown, & Siegle, 2019). The progression of reinforcement learning (RL) models has been particularly successful, with computational models increasingly capturing behavior and learning phenomena not well explained by prior models (Le Pelly, 2004; Mihatsch & Neuneier, 2002; Cochran & Cisler, 2019; Redish, Jensen, Johnson, & Kurth-Nelson, 2007). For example, whereas the standard Rescorla-Wagner (RW) model, which was developed to quantitatively formalize Pavlovian RL, models simple trial-and-error (“model-free”) learning, it does not capture more dynamic processes that occur during conditioning and extinction. Indeed, hybrid Pearce-Hall/RW learning models, which track trial-by-trial associability (i.e., the salience of a cue), provide a better fit for probabilistic (Brown et al., 2018) and fear-related learning (Homan et al., 2019; Li, Schiller, Schoenbaum, Phelps, & Daw, 2011) than the standard RW model among healthy controls and individuals with PTSD.

In addition to data-driven model development, alignment of computational models with psychological theory is paramount (Huys, Maia, & Frank, 2016), as this will allow for the testing and refinement of current hypotheses regarding the role of learning in disorders, such as PTSD. Whereas prior research has primarily assessed relatively simple trial-and-error learning in PTSD (Brown et al., 2018; Cisler et al., 2015; Cisler et al., 2019; Ross et al., 2018), few have included models that capture higher-level cognitive processes. Importantly, theories of anxiety and PTSD implicate abstract, higher-order cognition in fear learning and extinction (Dunsmoor & Murphy, 2015). For example, the degree to which learned fear is generalized may depend on individuals’ reasoning about internal conceptual representations (Dunsmoor & Murphy, 2015). In addition to the ability to reason about, integrate, and abstract categorical representations, which is facilitated by the anterior prefrontal cortex (Davis, Goldwater, & Giron, 2016), the abstraction of rules and contextual information is mediated by prefrontal regions (Cools, Clark, & Robbins, 2004; Fogelson, Shah, Scabini, & Knight, 2009). Evidence of impaired contextual modulation of fear learning among individuals with PTSD further suggests a potentially important role of complex cognitive functions in the acquisition and revision of fear (Steiger, Nees, Wicking, Lang, & Flor, 2015).

Unlike model-free (e.g., RW) and hybrid RL, which involve trial-and-error revisions of cue-outcome associations, model-based RL captures structured and dynamically shifting conditions or “rules” of learning (Daw et al., 2005; Redish et al., 2007). Model-based RL is theorized to support the development of internal models (i.e., cognitive maps)1 that contain hypotheses about different task conditions to allow a learner to make predictions about future actions within a changing environment (Daw et al., 2005; Gläscher, Daw, Dayan, & O’Doherty, 2010). During a situation in which a stimulus is paired with an aversive outcome (e.g., a circle is paired with a shock), an individual utilizing model-free RL would develop a single outcome expectation for that stimulus (e.g., “circles are dangerous”). By contrast, during a situation in which the pairing of an aversive outcome and a stimulus can change depending on situational factors (i.e., the abstract ‘state’ of the environment), an individual utilizing model-based RL would develop separate outcome expectations for the various conditions and would differentially weight their expectations depending on which situation (i.e., state) was currently relevant (e.g., “circles are dangerous in situation X, but not Y”). In PTSD, difficulty with the latter may contribute to challenges with new safety learning, which is common in PTSD (Fani et al., 2012).

Whereas model-free and hybrid RL are primarily implemented within regions of the ventral striatum, amygdala, and the salience network (Beierholm, Anen, Quartz, & Bossaerts, 2011; Brown et al., 2018; Cisler et al., 2019; Daw, Gershman, Seymour, Dayan, & Dolan, 2011; Gläscher et al., 2010; Ross et al., 2018), model-based RL is primarily implemented within regions of the prefrontal cortex and frontoparietal network (FPN; Beierholm et al., 2011; Gläscher, et al., 2010). Supporting the possibility that individuals with PTSD may have deficits in model-based RL, prior research has documented deficits in cognitive functions that are implemented within brain regions that overlap with those that support model-based RL, such as dorsolateral prefrontal cortex, inferior frontal gyrus, and anterior prefrontal cortex (Leskin & White, 2007; Polak, Witteveen, Reitsma, & Olff, 2012; Stein, Kennedy, & Twamley, 2002; Woon, Farrer, Braman, Mabey, & Hedges, 2017; Alvarez & Emory, 2006; Doll, Bath, Daw, & Frank, 2016). Thus, model-based RL may be disrupted in PTSD to a greater extent than model-free processes.

The primary goal of the present study was to build on previous work assessing model-based RL during acquisition and extinction of fear among women with assault-related PTSD. The study focused on assault-related PTSD because previous research has consistently shown that assault is a more potent risk factor for the development of PTSD than other forms of trauma (Breslau et al., 1998; Cisler et al., 2012; Frans et al., 2005; Kessler et al., 2017; Resnick et al., 2003). Additionally, because different forms of trauma predict different PTSD symptom profiles (Kelley et al., 2009), we selected participants with assault-related PTSD to increase the homogeneity of our participants, allowing us to avoid potential confounds of trauma type. Women were specifically selected for inclusion because women are (1) twice as likely as men to develop PTSD (Kessler et al., 2005; Kilpatrick et al., 2013) and (2) at higher risk of exposure to many forms of interpersonal violence than men, including rape, sexual assault, and physical assault by an intimate partner (Iverson et al., 2013). It was hypothesized that the model-based RL model would provide a better fit for participants’ behavior than the model-free and hybrid models, which do not allow a learner to develop sets of cue-outcome associations that are differentially applied and updated based on inferences about task rules (e.g., rules that differ for the acquisition and extinction context). It was further hypothesized that FPN encoding of trial-by-trial updates of current beliefs about task conditions, which are specific to the model-based model and contribute to differential weighting of cue value expectations based on a learner’s hypotheses, would predict PTSD symptom severity. Specifically, it was anticipated that reduced FPN encoding would predict greater PTSD symptom severity, reflecting poorer contextually derived learning. Because PTSD is related to difficulties with fear extinction/safety learning (Jovanovic et al., 2009; Jovanovic et al., 2012), potential differences in the encoding of model-belief updates during acquisition versus extinction were explored. Due to high overlap between belief update and prediction error parameters during extinction, the acquisition versus extinction analyses focused on value expectations while controlling for belief updates.

2. Methods

2.1. Participants

One hundred and three women were enrolled as part of a larger randomized clinical trial across two study sites: (1) University of Arkansas Medical Sciences and (2) University of Wisconsin-Madison (note: n=175 participants were assessed for eligibility and n=103 were enrolled; for full recruitment information, see Cisler et al., 2020). Primary inclusion criteria included female sex, age between 21 and 50 years old, and a current diagnosis of PTSD related to sexual or physical assault. Primary exclusion criteria included psychotic symptoms, pregnancy, learning disability, and medication or MRI contraindications. Twelve women were excluded due to task visit no show, claustrophobia, or a positive drug screen, yielding 91 subjects. Of these 91 subjects, a total of 85 had either viable skin conductance responses or neuroimaging data (see Computational Modeling and Neuroimaging sections below).

2.2. Clinical Interview and Measures

The past month version of the Clinician Administered PTSD Scale for DSM-5 (CAPS-5) was used to assess for the presence of current PTSD related to interpersonal violence (i.e., assault-related PTSD; Weathers et al., 2018). The CAPS-5 is a 30-item structured clinical interview that assesses for PTSD symptoms across four clusters: reexperiencing, avoidance, negative cognitions and mood, and hyperarousal. Symptoms are rated on a scale from 0 (absent) to 4 (extreme/incapacitating). To meet criteria for current PTSD, individuals must endorse a score of 2 or above for at least one reexperiencing symptom, one avoidance symptom, two negative cognitions/mood symptoms, and two hyperarousal symptoms. In addition to a providing a categorical diagnosis, total symptom scores provided a dimensional measure of current PTSD symptoms. Although all participants had a diagnosis of PTSD, there was substantial variation in the CAPS-5 symptom severity scores (see distribution of scores in Supplemental Figure S1). The One-Word Receptive Picture Vocabulary Test, Fourth Edition (ROWPVT-4) was used as a proxy measure of intelligence quotient (IQ; Martin & Brownell, 2011). IQ was estimated to account for potential effects of individual differences in IQ on model-based RL, given that IQ deficits have previously been found to relate to poorer model-based RL (Culbreth et al., 2016). During the ROWPVT-4, participants match vocabulary words that are administered verbally to illustrations that are presented in a book (Brownell, 2000). Scores were normed according to chronological age. Additional clinical and trauma assessments were completed by participants but were not of primary interest. Follow-up tests were implemented to account for the potential the impact of these variables on results (for a description of the assessments and results of the follow-up tests, see the Supplement).

2.3. Fear Conditioning and Fear Extinction Task

Participants completed four task blocks that alternated between fear acquisition and fear extinction (see Figure 1A). The first acquisition block was preceded by a baseline (habituation) period of 12 trials (6 for presentations of each cue) without any administrations of the unconditioned stimulus (UCS).2 The UCS was an electrotactile stimulation that was delivered to participants’ lower leg. Stimulation level was set to a maximum of 50 mA and participants’ stimulation level was individually calibrated prior to the task at a level that was uncomfortable, but not painful (approximately a 7 out of 10 on a Likert scale, 0=not uncomfortable, 10=extremely uncomfortable/painful). Triangles and circles served as the conditioned stimuli and different colored backgrounds identified the current context (i.e., the acquisition or extinction block), which were counterbalanced across participants (i.e., for half of participants, the CS+ was a triangle and for the other half the CS- was a triangle).

Figure 1.

Figure 1.

Open in a new tab

A. Schematic representation of the acquisition and extinction blocks of the Fear Conditioning and Fear Extinction Task. B. Representation showing the temporal mapping of value estimations (V), prediction errors (PE), and latent state belief updates (dB) during the no shock and shock trials. For the neuroimaging analyses, the onset phase was parametrically modulated by trial-by-trial value expectations (Vt,c) and the outcome phase was parametrically modulated by trial-by-trial PEs (positive and negative; PEt,c) and latent-state belief updates (dBt,c).

During each fear acquisition block, 18 conditioned safety cues (CS-) and 18 conditioned danger/threat cues (CS+) were presented for 3 seconds, with an intertrial interval of 2–6 seconds. During acquisition, the presentation of the CS+ was followed by an electrotactile stimulation (UCS) on 50% of trials, which occurred 2.5 seconds after the CS+ onset for duration of 500 msec. During each fear extinction block, there were 18 trials each of the CS- and CS+ cues, which were presented for 3 seconds, with an intertrial interval of 2–6 seconds. During extinction, no electrotactile stimulations occurred following the presentation of the CS+. The CS- and CS+ stimuli were pseudorandomly presented during each block, and participants completed a total of 156 task trials.

2.4. Skin Conductance Response Acquisition and Preprocessing

Following an approach used in previous studies (e.g., Homan et al., 2019; Li et al., 2011), participants’ SCR data was used to test the fit of several RL models. SCR has previously been shown to map onto value expectations and associability during RL (e.g., Li et al., 2011). More specifically, anticipatory SCR scales with the degree to which individuals expect that an outcome will occur for a given cue (e.g., delivery of an electrotactile stimulation), with larger SCR responses reflecting greater expectation of an outcome. Model fit was tested by minimizing the error between model estimated trial-wise value expectations and participants’ trial-wise SCR.

SCR data were acquired from participants’ left hand with the BIOPAC MP150 Data Acquisition System using the EDA100C module with the MECMRI-TRANS (MRI compatible) cable system. BIOPAC AcqKnowledge 4.3 software recorded SCR data at a rate of 2000 Hz at the Arkansas site and 1000 Hz at the Wisconsin site. Data were preprocessed using an approach that is consistent with our prior studies (Cisler et al., 2020; Privratsky, Bush, Bach, Hahn, & Cisler, 2020) and contemporary recommendations on modeling skin conductance data (Bach, 2014; Bach & Friston, 2013; Bach, Flandin, Friston, & Dolan, 2010; Bach, Friston, & Dolan, 2013). This pipeline used a 10 ms median filter, unidirectional butterworth filter with .0159 hz and 5 hz low and high pass frequencies, and by downsampling to 10 hz. Next, trial-by-trial SCR responses were estimated using a forward convolution model of SCR and were normalized to individuals’ maximum SCR response. Seventeen participants were excluded from computational modeling analyses due to flat responding, an excessive number of artifacts, or missing SCR data, yielding 74 participants whose data were included in the computational modeling. The amount of SCR data loss (19%) is comparable to prior fear extinction studies using SCR (e.g., Garfinkel et al., 2014; Haaker et al., 2013; Raij et al., 2018).

2.5. Computational Models

To identify whether a model-free, hybrid, or model-based RL model (from here on referred to simply as model-free, hybrid, and model-based models) provided a better fit to participants’ SCR data, several models were tested. The primary set of models that were tested against the model-based model included a standard RW model and a hybrid model, both of which have previously been used to estimate learning parameters from SCR data during fear conditioning tasks and the latter of which has been found to provide a better fit than the standard RW model (Li et al., 2011). Additional versions of the standard RW model were tested for completeness (see the Supplement).

2.5.1. Model-Free Model

Model-free RL was assessed with several versions of the Rescorla-Wagner (RW) model. The standard RW assumes that a learning agent keeps track of associative strengths representing the learner’s expectations for an outcome following the presentation of a cue. For a learning agent, we let a continuous variable Vt,c denote the associative strength (i.e., value expectation) on trial t for observed cue c. We also let a binary variable outcomet denote the outcome on trial t, with outcomet equal to 1 if the participant received a shock and 0 otherwise. Associative strengths are updated via prediction errors (PE), given by the difference between what happened (outcome) and what was expected, scaled by the learning rate. If cue c is presented on trial t, then the PE is denoted by δt = outcomet – Vt,c, and the associative strength of cue c is updated as follows: Vt+1,c= Vt,c + αt δt, where αt is a constant on trial t known as a learning rate. For the standard RW model, the same learning rate is used for all trials and associative strengths are only updated for presented cues, i.e., αt = α for all t and Vt+1,c= Vt,c for cue c not presented on trial t.

2.5.2. Hybrid Model

The hybrid model builds on the RW model: Vt+1,c=Vt,c + καt × δt, where Vt,c is the value expectation of cue (c) on the current trial (t), κ is a learning rate, αt is the associability for the current trial, and δt is the PE for the current trial (Le Pelley, 2004; Li et al., 2011). As with the RW model, δt = outcomet – Vt,c. Unlike the RW model, a constant learning rate (κ) scales an associability parameter that changes from trial to trial and is defined as αt+1 = η|δt| + (1-η)αt. The free parameter, η, scales the prior magnitude of PE and its additive inverse scales the prior associability (i.e., weighted PE). A higher η reflects greater weighting of the prior PE relative to the prior associability.

2.5.3. Model-Based Model

The latent state (LS) model was used to capture model-based RL (Cochran & Cisler, 2019; Letkiewicz, Cochran, & Cisler, 2020). This model was previously found to explain learning phenomena characterized by contextually based learning better than other widely used models (e.g., renewal, spontaneous recovery). Latent states are unobserved task rules/conditions that, in aggregate, define a learning environment. Each latent state contains sets of associations between cues and outcomes and a learner must infer which associations are currently most applicable. The LS model builds on the RW model by using integer l (i.e., the latent state) to index these sets of associations, where Vt,c,l is the current value strength of option c for latent state l: Vt+1,c,l= Vt,c,l + αt,c,l × δt,l. For the LS model, the learning rate (αt,c,l) is specific to the cue (c) on the current trial (t) for latent state l. The learning rate is proportional to a quantity that captures the degree to which a learner believes that the current task conditions are captured by a given latent state, referred to as latent-state beliefs, pl,t. The PE is specific to the latent state, whereby δt,l = outcomet – Vt,c,l for cue c on trial t for latent state l. Trial-by-trial expectations Vt,c of cue c are computed by taking a weighted average of Vt,c,l with weights pl,t. Following an outcome on a given trial, beliefs about current task conditions are updated (delta beliefs, dB). Larger updates in latent-state beliefs reflect larger changes in a learner’s internal model of the current task rules (see Supplement for additional details).

2.6. Analyses

2.6.1. Computational Modeling

Skin conductance responses that were acquired during the Fear Conditioning and Extinction Task were used to identify optimal participant model parameter estimates. For each participant, model parameters were estimated by fitting computational models to SCR data from participants without any missing data (n=74) via maximum likelihood estimation. Following convention, a square root transformation was applied the SCR (prior to the transformation, SCR values were rescaled between 0 and 1). Normalized, square root transformed SCR values were regressed onto trial-by-trial linear value estimation terms (Vt,c). Skin conductance responses were also regressed onto associability (αt) for the hybrid model and onto updates in latent state beliefs (dBt) for the LS model. Regression error was assumed to follow a normal distribution with mean zero and unknown variance. Maximum likelihood estimation was performed by minimizing squared regression error summed only over trials in which a shock was omitted (Li et al., 2011) using fmincon in Matlab (The MathWorks, Inc., Natick, MA). For each participant and RL model, estimation yielded regression coefficients and RL model parameters. Resulting log-likelihood values were compared to identify the best-fitting model. Additional regression parameters were included in exploratory analyses to identify whether the inclusion of non-linear terms would capture large trial-by-trial changes in SCR not readily captured by linear terms, thereby yielding better model fit (results provided in the Supplement).

2.6.2. Neuroimaging

All neuroimaging analyses focused on the LS model parameters (see Supplement for neuroimaging acquisition and preprocessing details). Trial-by-trial dBs, which characterize model-based RL updates, were carried forward to voxelwise and independent component analysis (ICA) to identify brain regions and networks that support implementation/encoding of latent-state updates among women with PTSD. Additionally, analyses focused on identifying whether PTSD symptom severity modulated dB-related encoding. Because individual-level parameters have previously been shown to be too noisy to yield reliable neuroimaging results (i.e., they exhibit high levels of error variance), learning parameters (e.g., V, PE, dB) were averaged across participants and the resulting trial-by-trial mean parameters were used in the neuroimaging analyses in accordance with previous research (Daw et al., 2005; Daw, Gershman, Seymour, Dayan, & Dolan, 2011; Li et al., 2011; Schönberg et al., 2007; Schönberg, Daw, Joel, & O’Doherty, 2010). Of the 91participants who were eligible for the present study, seventy-seven participants had viable neuroimaging data. However, one participant was excluded due to missing clinical variables (final sample: n=76; see the Supplement for information regarding the overlap between the computational modeling and neuroimaging samples). See Table 1 for demographic and clinical characteristics of participants included in the computational modeling and/or neuroimaging analyses (n=85).

Table 1.

Participant Demographic and Clinical Characteristics

Participants Arkansas Site Wisconsin Site
n 85 39 46
Age, Mean (SD) 33.7 (8.7) 36.1 (8.4)* 31.6 (8.5)
Race/Ethnicity (%)
 White 74 69 78
 Black/African American 17 23 11
 Asian 0 0 0
 Hispanic, Latina 4 0 7
 Pacific Islander 0 0 0
 Native American 0 0 0
 Other 5 8 4
IQ, Mean (SD) 98.8 (20.6) 88.1 (21.6)* 107.5 (15.0)
CAPS Symptom Severity, Mean (SD) 42.4 (11.2) 42.2 (11.5) 40.9 (10.9)
CAPS Current # Symptoms 14.3 (2.8) 13.8 (2.7) 14 (2.9)
PCL-C 43.6 (13.8) 45.6 (13.7) 41.9 (14.7)
Open in a new tab
*

IQ=Intelligence Quotient, One-Word Receptive Vocabulary Test; CAPS=Clinician- Administered PTSD Scale; PCL-C=PTSD Checklist - Civilian Version

2.6.2.1. Voxelwise Analyses

Participants’ voxelwise time courses were regressed onto the design matrix using AFNI (3dREML; Cox, 1996). The design matrix included the stimulus onset and outcome phases of the task. The onset phase was parametrically modulated by trial-by-trial value expectation (Vt,c) from the LS model, and the outcome phase was parametrically modulated by trial-by-trial PE (positive and negative; PEt,c) and latent-state belief updates (dBt,c) from the LS model (see Figure 1B). To account for the potential impact of the electrotactile stimulation on neural activity, the design matrix also included a “shock” regressor. Because PE and the occurrence of the shock are highly correlated (Erdeniz, Rohe, Done, & Seidler, 2013), PE-related neural activity was not interpreted. Beta coefficients for the onset phase modulated by V and the outcome phase modulated by dB were included in the second-level analyses. Linear mixed effects models (LMEs) were implemented using MATLAB (fitlme; The MathWorks, Inc., Natick, MA) to test for main effects of neural activity during dB and V, controlling for age, IQ, and study site:

  • 1

    dB β ~ age + IQ + study site + (1|subject)

  • 2

    V β ~ age + IQ + study site + (1|subject)

Additionally, a model tested for potential unique effects of dB and V-related neural activity on CAPS-5 symptom severity. PE was included as a predictor, but PE results were not interpreted (for the reasons stated above):

  • 3

    CAPS ~ V β + dB β + PE β + age + IQ + study site + (1|subject)

A set of follow-up analyses assessed for differential effects of acquisition versus extinction on the results of models 2 and 3. The design matrix included separate regressors for trial onset modulated by value expectation during acquisition (Vta,c) and extinction (Vte,c), and separate regressors for outcome modulated by dB during acquisition (dBta,c) and extinction (dBte,c). Because correlations between PE during extinction (PEte,c) and Vte,c and between PEte,c and dBte,c were highly correlated (|r|>.70), PE was not included in these analyses. Additionally, given that outcome-related neural activity associated with PE could not be separated from that of dB, LME tests focused on whether neural activity during V significantly differed during acquisition versus extinction (contrast coded: acquisition=1, extinction=−1), controlling for age, IQ, and study site:

  • 1

    V β ~ contrast + age + IQ + study site + (1|subject)

Separate models tested for potential unique effects of V and dB-related neural activity on CAPS symptom severity during acquisition and extinction.

  • 2

    CAPS ~ V βacquisition + dB βacquisition + age + IQ + study site + (1|subject)

  • 3

    CAPS ~ V βextinction+ dB βextinction + age + IQ + study site + (1|subject)

Voxelwise comparisons were implemented within a sample specific grey matter mask and cluster-level thresholding controlled for voxelwise comparisons using an uncorrected p < .001 and cluster size k ≥ 18, which was identified using AFNIs 3dClustSim.

2.6.2.2. Independent Component Analysis

ICA was used to identify temporally coactivated spatially distributed neural large-scale networks and was implemented using the Group ICA of fMRI Toolbox (GIFT; Calhoun et al., 2001) in Matlab R2016a. A model order of 35 was selected to balance the tradeoff between component estimation reliability and interpretability. Thirteen of the 35 components were identified as functional networks theoretically related to learning or PTSD, including a left and right FPN that were of primary interest (22 networks that represented either motion artifact, CSF, or networks of non-interest such as motor cortex were excluded). Additionally, follow-up analyses were implemented with the remaining eleven networks (see Supplemental Figure S2). ICA timecourses were regressed onto the same design matrices described above using AFNI (3dREML; Cox, 1996) and resulting beta coefficients were included in the second-level analyses. The same series of LMEs described above were implemented for 1) each FPN and 2) follow-up networks using Matlab (fitlme; The MathWorks, Inc., Natick, MA). Bonferroni correction was applied for the two FPN networks (p < .025) and for the post-hoc analyses across the eleven additional networks (p < .005). Additionally, following an approach used by Erdeniz et al. (2013), several GLMs were fitted to participants’ ICA time courses to identify whether the inclusion or removal of the PE and/or shock regressors altered the main effects of dB and/or V (see Supplement).

3. Results

3.1. Model Fit

The standard RW and hybrid models, which are nested models, were formally tested using a log-likelihood ratio test. Similar to previous studies (Boll et al., 2013; Homan et al., 2019; Li et al., 2011), the hybrid model outperformed the standard RW model, χ2=515.43, df=148, p<.001. It also outperformed the additional RW models that were tested (see Supplement). Because the LS model does not contain the terms included in the hybrid model, a log-likelihood ratio test was not performed to compare these models. A comparison between Akaike Information Criterion values, which were summed across participants, revealed that the LS model provided a better fit than the standard RW and hybrid models (see Figure 2), as well as the additional RW models that were tested (see Supplemental Figure S2).

Figure 2.

Figure 2.

Open in a new tab

Summed Akaike Information Criterion (AIC) values across participants showing that the Latent State model outperformed the Rescorla Wagner and Hybrid models (note: lower AIC values reflect better model fit).

3.2. Main Effects

3.2.1. Voxelwise Analyses

Table 2 lists the brain regions in which neural activation predicted latent-state belief updates. Activity in the left inferior frontal gyrus, left and right insula, right paracentral lobule, left and right calcarine gyrus, left postcentral/precentral gyrus, cerebellum, and right cuneus were positively related to latent-state belief updates (see Figure 3A). Activity in left and right inferior occipital gyrus, superior orbital gyrus, left superior frontal gyrus, precuneus, left middle frontal gyrus, and left angular gyrus were negatively related to latent-state belief updates. Table 3 lists the brain regions in which neural activation predicted value expectation. Greater value expectation-related activity emerged in the paracentral lobule, right middle frontal gyrus, and left thalamus (see Figure 3B). Lower value expectation-related activity emerged in the left precuneus.

Table 2.

Regions Associated with Trial-by-Trial Changes in Latent-State Beliefs (dB) during Outcome

Center of Mass Coordinates
Region Cluster Size in mm3 Peak t-value X Y Z
R. IFG/Insula 1664 11.60 42 −8 16
R. Paracentral Lobule 1600 9.53 1 −23 59
L. Insula Lobe 1132 12.81 −34 −6 10
R. Calcarine Gyrus 153 6.81 18 −60 7
L. Postcentral/Precentral Gyrus 149 7.13 −43 −7 48
R. Inferior Occipital Gyrus 135 −5.51 41 −75 −3
L. Cerebellum 125 6.74 −5 −38 −20
Superior Orbital Gyrus 103 −6.89 −4 59 −21
L. Calcarine Gyrus 70 6.09 −15 −68 7
L. Superior Frontal Gyrus 63 −5.43 −14 39 47
Precuneus 57 −6.15 −2 −56 27
Cerebellum 34 5.47 1 −58 −36
L. Cerebellum 31 5.18 −31 −53 −28
R. Cuneus 29 4.61 13 −77 33
L. Inferior Occipital Gyrus 24 −6.48 −46 −70 −9
L. Middle Frontal Gyrus 23 −4.58 −33 15 47
L. Angular Gyrus 23 −4.72 −46 −66 29
L. Postcentral Gyrus 18 −4.60 −38 −28 53
Open in a new tab
Figure 3.

Figure 3.

Open in a new tab

Brain regions associated with A. latent-state belief updates (dB) and B. value estimations (V). Parameter-related neural activity that uniquely predicted PTSD symptom severity for C. latent-state updates and D. value estimations. L = left. Warm colors = positive z-values.

Table 3.

Regions Associated with Trial-by-Trial Associative Strength (V) during Stimulus Onset

Center of Mass Coordinates
Region Cluster Size in mm3 Peak t-value X Y Z
Paracentral Lobule 252 6.64 4 −23 66
R. Middle Frontal Gyrus 29 4.99 43 −3 50
L. Thalamus 26 5.42 −6 −19 7
L. Precuneus 21 −5.16 −7 −65 44
Open in a new tab

Table 4 lists the brain regions in which the parameter-related neural activity uniquely predicted PTSD symptom severity. Figure 3C shows that lower latent state update-related activation within the right calcarine gyrus/right posterior cingulate cortex predicted higher CAPS scores. As shown in Figure 3D, greater value expectation-related activation within the left angular gyrus/inferior parietal lobule predicted higher CAPS scores.

Table 4.

Parameter-Related Activity Uniquely Predictive of Clinician-Administered PTSD Scale Severity

Center of Mass Coordinates
Region Parameter Cluster Size in mm3 Peak t-value X Y Z
Left Angular Gyrus/IPL V 27 4.68 −46 −66 36
Right Calcarine Gyrus/Right PCC dB 22 −3.75 8 −62 15
Open in a new tab

3.2.2. ICA

Table 5 lists the ICA networks that predicted trial-by-trial latent-state belief update encoding. Reduced encoding was evident in the left frontoparietal network (FPN), as well as the limbic, dorsomedial prefrontal cortex/posterior cingulate cortex, and hippocampal networks (see Figure 4A). Increased encoding was evident in the pre-supplementary motor area and striatal networks. Table 5 lists the ICA networks that predicted trial-by-trial value estimation encoding. Increased encoding emerged in the pre-supplementary motor area and striatal networks (see Figure 4B). Results for all networks are provided in the Supplement. A similar pattern of results emerged across study sites (see Figure S5 in the Supplement).

Table 5.

Networks Associated with Trial-by-Trial Encoding of Latent-State Updates (dB) and Value Estimation (V)

Parameter ICA Network t-value p-value
dB L. Frontoparietal −3.64 <.001
Pre-Supplementary Motor Area 6.70 <.001
Limbic −7.44 <.001
Hippocampus −3.14 .003
Dorsomedial PFC/Posterior Cingulate Cortex −6.26 .001
Insula/Middle Frontal Gyrus 2.45 .017
Striatum 10.66 <.001
V Pre-Supplementary Motor Area 4.28 <.001
Limbic −2.24 .029
Striatum 3.86 p<.001
Open in a new tab

Note: Networks in gray font did not survive Bonferroni correction.

Figure 4.

Figure 4.

Open in a new tab

Depiction of the main effects of encoding during trial-by-trial changes in A. latent-state beliefs (dB) and B. value estimation (V) for GLM model 1 (full model: dB, PE, V, and shock regressor included).

Table 6 lists the networks in which parameter-related encoding uniquely predicted PTSD symptom severity on the CAPS. Increased encoding of value expectation within the left FPN predicted greater PTSD symptom severity, t(68)=3.58, p<.001. No other results held above correction.

Table 6.

Parameter-Related Activity Uniquely Predictive of Clinician-Administered PTSD Scale Severity

ICA Network Parameter t-value p-value
L. Frontoparietal V 3.58 .0007
Default Mode V 2.01 .049
Dorsomedial PFC/Posterior Cingulate Cortex V 2.02 .048
Inferior Frontotemporal V −2.37 .021
Open in a new tab

Note: Networks in gray font did not survive Bonferroni correction.

3.3. Task Phase Effects (Acquisition versus Extinction)

3.3.1. Voxelwise Analyses

Table 7 lists the brain regions in which neural activation differentially predicted value expectation during the acquisition versus extinction task phases. Greater activity during extinction than acquisition emerged in the middle cingulate cortex, thalamus, left fusiform gyrus, right lingual gyrus, left calcarine gyrus, left middle occipital gyrus, cuneus, and right lingual gyrus.

Table 7.

Regions Associated with Trial-by-Trial Value Expectation (V) During Stimulus Onset, Acquisition versus Extinction

Center of Mass Coordinates
Region Cluster Size in mm3 Peak t-value X Y Z
Middle Cingulate Cortex 107 −5.68 −1 −40 35
Thalamus 62 −5.05 −1 −21 9
L. Fusiform Gyrus 60 −4.56 −40 −51 −14
R. Lingual Gyrus 49 −5.13 9 −44 3
L. Calcarine Gyrus 47 −4.45 −18 −64 9
L. Middle Occipital Gyrus 36 −5.27 −31 −84 5
Cuneus 28 −4.53 6 −67 24
R. Lingual Gyrus 27 −3.89 17 −58 3
Open in a new tab

3.3.2. ICA

Table 8 lists the ICA networks that exhibited differential value expectation encoding during the acquisition versus extinction task phases. A significant effect of task phase emerged for value expectation encoding within both FPNs. Specifically, greater value expectation encoding was evident in the left and right FPN during acquisition relative to extinction, t(146)=2.53, p=.013 and t(146)=2.62, p<.010, respectively (see Figure 5). Significantly greater value expectation encoding was also evident in medial/lateral prefrontal cortex network during acquisition versus extinction, t(146)=3.38, p<.001, whereas lower value estimation encoding was evident in hippocampal network during acquisition versus extinction, t(146)=−2.92, p<.004. Results followed a similar pattern across study site (see Figure S6 in the Supplement).

Table 8.

Networks Associated with Trial-by-Trial Encoding of Value Estimation (V) during Acquisition versus Extinction

Parameter ICA Network t-value p-value
V L. Frontoparietal 2.53 .013
R. Frontoparietal 2.62 .010
Medial/Lateral PFC 3.38 <.001
Medial PFC/Striatum −2.44 .016
Limbic 2.78 .006
Hippocampus −2.92 .004
Dorsomedial PFC/Posterior Cingulate Cortex 2.51 .013
Open in a new tab

Note: Regions in gray font did not hold above Bonferroni correction.

Figure 5.

Figure 5.

Open in a new tab

Depiction of the effects of acquisition versus extinction on encoding during trial-by-trial changes in value estimation (V).

Table 9 lists the networks in which value expectation-related encoding uniquely predicted PTSD symptom severity on the CAPS during acquisition and extinction. As shown in Figure 6A, during acquisition, but not extinction, greater value expectation encoding in the left FPN uniquely predicted higher PTSD symptom severity, t(69)=2.99, p=.004. As shown in Figure 6B, during extinction, but not acquisition, lower value expectation encoding in the dorsomedial prefrontal cortex/posterior cingulate cortex network uniquely predicted higher PTSD symptom severity, t(69)=−4.05, p<.001. A similar pattern of results was evident across study site (see Figure S7 in the Supplement).

Table 9.

V-Related Activity Uniquely Predictive of Clinician-Administered PTSD Scale Severity

ICA Network t-value p-value
Acquisition
 L. Frontoparietal 3.45 .001
Extinction
 Dorsomedial PFC/PCC −4.05 <.001
Open in a new tab
Figure 6.

Figure 6.

Open in a new tab

A. Scatterplots depicting relationships between left frontoparietal network encoding and PTSD symptom severity during (1) acquisition and (2) extinction, and a graphical depiction of the left frontoparietal ICA network. B. Scatterplots depicting relationships between dorsomedial prefrontal cortex network encoding and PTSD symptom severity during (1) acquisition and (2) extinction, and a graphical depiction of the dorsomedial prefrontal cortex network ICA network. The dark blue scatterplots represent the relationships that reached the level of significance. The light blue (non-significant) scatterplots are provided for reference.

4. Discussion

Results in this sample of adult women with IPV-related PTSD revealed that the latent-state model provided a better fit for participants’ physiological responses during the fear conditioning task than the standard RW and hybrid models. Notably, the latter model is considered a gold-standard for modeling fear conditioning responses, as it has been found to fit fear conditioning responsivity better than standard RW models in several previous studies (Homan et al., 2019; Li et al., 2011), which was replicated in the current study. However, it was outperformed by the latent-state model in the present study. This suggests that the latent-state model captures learning dynamics during fear conditioning and extinction that are not readily measured by the hybrid model, such as learning that varies as a function of task conditions.

Contrary to predictions, latent-state belief updates were associated with reduced (as opposed to increased) activity in regions previously associated with model-based RL and cognitive control, including the left FPN and left superior frontal gyrus. Similar to previous research on model-based RL, latent-state belief updates were related to increased striatal network activity (Daw, Gershman, Seymour, Dayan, & Dolan, 2011; McDannald, Lucantonio, Burke, Niv, & Schoenbaum, 2011; but also see Gläscher et al., 2010). Latent-state belief updates were also related to increased activity of the left and right insula, which are implicated in punishment and loss-related learning (Palmentari et al., 2012). Although the bilateral anterior insula are components of the salience network, which is often associated with model-free PE encoding (Cisler et al., 2019; Preuschoff, Quartz, & Bossaerts, 2008), this is not the first study to identify insula activation during model-based RL (Lee et al., 2014). Heightened insula activity during updates to latent-state beliefs may reflect sensitivity to unexpected and changing environmental demands that signal increased need for cognitive control processes during learning (Jiang, Beck, Heller, & Egner, 2015).

Although the latent-state model provided a better overall fit to the SCR data than the other computational models that do not capture model-based RL processes, PTSD symptom severity was not predicted by FPN-related encoding of latent-state belief updates. Instead, higher PTSD symptom severity was uniquely related to reduced activity during latent-state updates in the right calcarine gyrus/posterior cingulate cortex, which are implicated in visual processing, visual imagery, and the focus of attention (Klein, Paradis, Poline, Kosslyn, & Le Bihan, 2000; Leech & Sharp, 2014). By contrast, increased value estimation-related encoding in the left FPN and activity of the angular gyrus/intraparietal lobule uniquely predicted greater PTSD symptom severity. It was further revealed that increased encoding within the left FPN during acquisition predicted greater PTSD symptom severity during acquisition, whereas reduced encoding within the dorsomedial prefrontal cortex (dmPFC/PCC) predicted greater PTSD symptom severity during extinction. Atypical FPN and dmPFC/PCC activity have been identified in previous studies of PTSD, including fear conditioning and extinction studies (for a review, see Suarez-Jimenez et al., 2020), though the direction of these effects is somewhat mixed in PTSD. It is important to note that, in contrast with prior studies that did not separate RL and non-RL sources of variance, we were able to identify effects for distinct RL processes using our model-based model. Heightened encoding of value estimation during the acquisition blocks (i.e., when threat expectancies are high) among cognitive control regions may contribute to enhanced representation of fear, while reduced encoding of value estimation during the extinction blocks (i.e., when threat expectancies are low) of the dmPFC/PCC may contribute to difficulties revising stimulus-outcome associations within a safe environment (Wang et al., 2014).

A notable limitation of the present study is the lack of a control comparison group. Although altered neural activity during latent-state belief updates did not emerge within cognitive control regions or networks in relation to PTSD symptoms, we did not test whether individuals with PTSD exhibit poorer encoding of latent-state belief updates within FPNs relative to individuals without PTSD. It is possible that PTSD symptoms scaled with value expectations, rather than latent-state belief updates, within the left FPN because overall individuals with PTSD generally had difficulty engaging contextual learning processes during fear conditioning. It will be important to establish the typical pattern of neural activity and encoding of value expectations and latent-state belief updates to further contextualize the meaning of present results. It will also be important to establish whether results extend to non-IPV related PTSD.

Overall, results provide some evidence that model-based RL processes that are altered during fear conditioning are related to PTSD symptoms among women with assault-related PTSD. While research has primarily been devoted to examining relatively simplistic learning processes, model-free and hybrid models cannot capture higher-level, context-related learning (e.g., learning when a stimulus is dangerous versus safe), the latter of which can be captured by the model-based model and may be particularly important in the acquisition and revision of human fear. Although disruptions in model-based processes that scaled with PTSD symptoms occurred within brain regions or networks that were not anticipated (e.g., reduced encoding within visual processing regions during latent-state belief updates), our results provide preliminary evidence of model-based RL-related impairments in PTSD that are separate from other learning processes (e.g., value estimation). Our latent-state model also identified a pattern of value estimation encoding that is distinct from latent-state update encoding that may disrupt normative fear and safety learning among individuals with assault-related PTSD. Critically, exposure-based therapy, which is a gold-standard treatment for PTSD, depends on learning processes to extinguish fear (Hermans et al., 2005), and RL deficits identified in this study may affect treatment response. Given that even the best available treatments for PTSD have limited efficacy, with remission occurring for approximately half of individuals who receive treatment (Morina, Wicherts, Lobbrecht, & Priebe, 2014; Resick, Nishith, Weaver, Astin, & Feuer et al., 2002; Schnurr et al., 2007), it is proposed that future research examine the role of RL impairment in treatment-related outcomes among individuals with PTSD (IPV and non-IPV) using a model-based framework.

Supplementary Material

1747963_Sup_info

Open Practices Statement.

None of the data or materials for the experiments reported here is available, and none of the experiments was preregistered.

Footnotes

1

The terminology “model-based” and “model-free” refer to whether a learner develops an internal model of the environment, which occurs during model-based, but not model-free, RL. The distinction between model-free and model-based RL are explained in more detail elsewhere (e.g., Daw, Niv, & Dayan, 2005; Daw, Gershman, Seymour, Dayan, & Dolan, 2011; Gläscher, Daw, Dayan, & O’Doherty, 2010; Wunderlich, Dayan, & Dolan, 2012; Van Otterlo & Wiering, 2012).

2

SCR responses did not differ between the CS+ and CS- cues during habituation, t(146)=1.28, p=.202. Conditioned responses to the CS+ were significantly larger during acquisition than baseline, t(146)=7.93, p<.001.

References

  1. Alvarez JA, & Emory E (2006). Executive function and the frontal lobes: A meta-analytic review. Neuropsychology Review, 16(1), 17–42. [DOI] [PubMed] [Google Scholar]
  2. Bach DR (2014). A head-to-head comparison of SCRalyze and Ledalab, two model-based methods for skin conductance analysis. Biological Psychology, 103, 63–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bach DR, & Friston KJ (2013). Model‐based analysis of skin conductance responses: Towards causal models in psychophysiology. Psychophysiology, 50(1), 15–22. [DOI] [PubMed] [Google Scholar]
  4. Bach DR, Flandin G, Friston KJ, & Dolan RJ (2010). Modelling event-related skin conductance responses. International Journal of Psychophysiology, 75(3), 349–356. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bach DR, Friston KJ, & Dolan RJ (2013). An improved algorithm for model-based analysis of evoked skin conductance responses. Biological Psychology, 94(3), 490–497. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Beierholm UR, Anen C, Quartz S, & Bossaerts P (2011). Separate encoding of model-based and model-free valuations in the human brain. Neuroimage, 58(3), 955–962. [DOI] [PubMed] [Google Scholar]
  7. Boll S, Gamer M, Gluth S, Finsterbusch J, & Büchel C (2013). Separate amygdala subregions signal surprise and predictiveness during associative fear learning in humans. European Journal of Neuroscience, 37(5), 758–767. [DOI] [PubMed] [Google Scholar]
  8. Breslau N, Kessler RC, Chilcoat HD, Schultz LR, Davis GC, & Andreski P (1998). Trauma and posttraumatic stress disorder in the community: The 1996 Detroit Area Survey of Trauma. Archives of General Psychiatry, 55(7), 626–632. [DOI] [PubMed] [Google Scholar]
  9. Brown VM, Zhu L, Wang JM, Frueh BC, King-Casas B, & Chiu PH (2018). Associability-modulated loss learning is increased in posttraumatic stress disorder. Elife, 7, e30150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Brownell R (2000). Expressive and receptive one-word picture vocabulary tests (EOWPVT, ROWPVT). New York: Psychological Corporation. [Google Scholar]
  11. Calhoun VD, Adali T, Pearlson GD, & Pekar JJ (2001). A method for making group inferences from functional MRI data using independent component analysis. Human Brain Mapping, 14(3), 140–151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cisler JM, Bush K, Steele JS, Lenow JK, Smitherman S, & Kilts CD (2015). Brain and behavioral evidence for altered social learning mechanisms among women with assault-related posttraumatic stress disorder. Journal of Psychiatric Research, 63, 75–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Cisler JM, Begle AM, Amstadter AB, Resnick HS, Danielson CK, Saunders BE, & Kilpatrick DG (2012). Exposure to interpersonal violence and risk for PTSD, depression, delinquency, and binge drinking among adolescents: Data from the NSA‐R. Journal of Traumatic Stress, 25(1), 33–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Cisler JM, Esbensen K, Sellnow K, Ross M, Weaver S, Sartin-Tarm A, … & Kilts CD (2019). Differential roles of the salience network during prediction error encoding and facial emotion processing among female adolescent assault victims. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 4, 371–380. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Cisler JM, Privratsky AA, Sartin-Tarm A, Sellnow K, Ross M, Weaver S, … & Kilts CD (2020). l-DOPA and consolidation of fear extinction learning among women with posttraumatic stress disorder. Translational Psychiatry, 10(1), 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cochran AL & Cisler JM (2019). A flexible and generalizable model of online latent-state learning. PLoS Computational Biology, 15, e1007331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Cools R, Clark L, & Robbins TW (2004). Differential responses in human striatum and prefrontal cortex to changes in object and rule relevance. Journal of Neuroscience, 24(5), 1129–1135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Cox RW (1996). AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research, 29, 162–173. [DOI] [PubMed] [Google Scholar]
  19. Culbreth AJ, Westbrook A, Daw ND, Botvinick M, & Barch DM (2016). Reduced model-based decision-making in schizophrenia. Journal of Abnormal Psychology, 125(6), 777. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Davis T, Goldwater M, & Giron J (2017). From concrete examples to abstract relations: The rostrolateral prefrontal cortex integrates novel examples into relational categories. Cerebral Cortex, 27(4), 2652–2670. [DOI] [PubMed] [Google Scholar]
  21. Daw ND, Niv Y, & Dayan P (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8, 1704–1711. [DOI] [PubMed] [Google Scholar]
  22. Daw ND, Gershman SJ, Seymour B, Dayan P, & Dolan RJ (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69, 1204–1215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Doll BB, Bath KG, Daw ND, & Frank MJ (2016). Variability in dopamine genes dissociates model-based and model-free reinforcement learning. Journal of Neuroscience, 36(4), 1211–1222 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Dunsmoor JE, & Murphy GL (2015). Categories, concepts, and conditioning: how humans generalize fear. Trends in Cognitive sciences, 19(2), 73–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Erdeniz B, Rohe T, Done J, & Seidler R (2013). A simple solution for model comparison in bold imaging: the special case of reward prediction error and reward outcomes. Frontiers in Neuroscience, 7, 116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Fogelson N, Shah M, Scabini D, & Knight RT (2009). Prefrontal cortex is critical for contextual processing: evidence from brain lesions. Brain, 132(11), 3002–3010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Frans Ö, Rimmö PA, Åberg L, & Fredrikson M (2005). Trauma exposure and post‐traumatic stress disorder in the general population. Acta Psychiatrica Scandinavica, 111(4), 291–290. [DOI] [PubMed] [Google Scholar]
  28. Garfinkel SN, Abelson JL, King AP, Sripada RK, Wang X, Gaines LM, & Liberzon I (2014). Impaired contextual modulation of memories in PTSD: an fMRI and psychophysiological study of extinction retention and fear renewal. Journal of Neuroscience, 34(40), 13435–13443. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Gläscher J, Daw N, Dayan P, & O’Doherty JP (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66, 585–595. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Haaker J, Gaburro S, Sah A, Gartmann N, Lonsdorf TB, Meier K, … & Kalisch R (2013). Single dose of L-dopa makes extinction memories context-independent and prevents the return of fear. Proceedings of the National Academy of Sciences, 110(26), E2428–E2436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Hermans D, Dirikx T, Vansteenwegenin D, Baeyens F, Van den Bergh O, & Eelen P (2005). Reinstatement of fear responses in human aversive conditioning. Behaviour Research and Therapy, 43(4), 533–551. [DOI] [PubMed] [Google Scholar]
  32. Homan P, Levy I, Feltham E, Gordon C, Hu J, Li J, … & Schiller D (2019). Neural computations of threat in the aftermath of combat trauma. Nature Neuroscience, 22(3), 470–476. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Huys QJ, Maia TV, & Frank MJ (2016). Computational psychiatry as a bridge from neuroscience to clinical applications. Nature Neuroscience, 19(3), 404–413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Iverson KM, Dick A, McLaughlin KA, Smith BN, Bell ME, Gerber MR, Cook N, & Mitchell KS (2013). Exposure to interpersonal violence and its associations with psychiatric morbidity in a U.S. national sample: A gender comparison. Psychology of Violence, 3(3), 273–287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Jiang J, Beck J, Heller K, & Egner T (2015). An insula-frontostriatal network mediates flexible cognitive control by adaptively predicting changing control demands. Nature Communications, 6(1), 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Jovanovic T, Kazama A, Bachevalier J, & Davis M (2012). Impaired safety signal learning may be a biomarker of PTSD. Neuropharmacology, 62(2), 695–704. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Jovanovic T, Norrholm SD, Blanding NQ, Davis M, Duncan E, Bradley B, & Ressler KJ (2010). Impaired fear inhibition is a biomarker of PTSD but not depression. Depression and Anxiety, 27(3), 244–251. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Jovanovic T, Norrholm SD, Fennell JE, Keyes M, Fiallos AM, Myers KM, … & Duncan EJ (2009). Posttraumatic stress disorder may be associated with impaired fear inhibition: Relation to symptom severity. Psychiatry Research, 167(1–2), 151–160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Kaczkurkin AN, Burton PC, Chazin SM, Manbeck AB, Espensen-Sturges T, Cooper SE, … & Lissek S (2017). Neural substrates of overgeneralized conditioned fear in PTSD. American Journal of Psychiatry, 174(2), 125–134. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Kelley LP, Weathers FW, McDevitt‐Murphy ME, Eakin DE, & Flood AM (2009). A comparison of PTSD symptom patterns in three types of civilian trauma. Journal of Traumatic Stress, 22(3), 227–235. [DOI] [PubMed] [Google Scholar]
  41. Kessler RC, Aguilar-Gaxiola S, Alonso J, Benjet C, Bromet EJ, Cardoso G, … & Koenen KC (2017). Trauma and PTSD in the WHO world mental health surveys. European Journal of Psychotraumatology, 8(sup5), 1353383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Kessler RC, Chiu WT, Demler O, & Walters EE (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 617–627. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Kilpatrick DG, Resnick HS, Milanak ME, Miller MW, Keyes KM, & Friedman MJ (2013). National estimates of exposure to traumatic events and PTSD prevalence using DSM‐IV and DSM‐5 criteria. Journal of Traumatic Stress, 26(5), 537–547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Klein I, Paradis AL, Poline JB, Kosslyn SM, & Le Bihan D (2000). Transient activity in the human calcarine cortex during visual-mental imagery: An event-related fMRI study. Journal of Cognitive Neuroscience, 12, 15–23. [DOI] [PubMed] [Google Scholar]
  45. Le Pelley ME (2004). The role of associative history in models of associative learning: A selective review and a hybrid model. The Quarterly Journal of Experimental Psychology Section B, 57(3b), 193–243. [DOI] [PubMed] [Google Scholar]
  46. Lee SW, Shimojo S, & O’Doherty JP (2014). Neural computations underlying arbitration between model-based and model-free learning. Neuron, 81(3), 687–699. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Leech R, & Sharp DJ (2014). The role of the posterior cingulate cortex in cognition and disease. Brain, 137(1), 12–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Letkiewicz AM, Cochran AL, & Cisler JM (2020). Frontoparietal network activity during model-based reinforcement learning updates is reduced among adolescents with severe sexual abuse. Journal of Psychiatric Research. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Li J, Schiller D, Schoenbaum G, Phelps EA, & Daw ND (2011). Differential roles of human striatum and amygdala in associative learning. Nature Neuroscience, 14(10), 1250–1252. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Lissek S, & van Meurs B (2015). Learning models of PTSD: Theoretical accounts and psychobiological evidence. International Journal of Psychophysiology, 98(3), 594–605. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Lopresto D, Schipper P, & Homberg JR (2016). Neural circuits and mechanisms involved in fear generalization: implications for the pathophysiology and treatment of posttraumatic stress disorder. Neuroscience & Biobehavioral Reviews, 60, 31–42. [DOI] [PubMed] [Google Scholar]
  52. Martin NA, & Brownell R (2011). ROWPVT-4: Receptive One-Word Picture Vocabulary Test.
  53. McDannald MA, Lucantonio F, Burke KA, Niv Y, & Schoenbaum G (2011). Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. Journal of Neuroscience, 31(7), 2700–2705. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Mihatsch O, & Neuneier R (2002). Risk-sensitive reinforcement learning. Machine Learning, 49(2–3), 267–290. [Google Scholar]
  55. Morina N, Wicherts JM, Lobbrecht J, & Priebe S (2014). Remission from post-traumatic stress disorder in adults: a systematic review and meta-analysis of long-term outcome studies. Clinical Psychology Review, 34(3), 249–255. [DOI] [PubMed] [Google Scholar]
  56. Pacella ML, Hruska B, & Delahanty DL (2013). The physical health consequences of PTSD and PTSD symptoms: A meta-analytic review. Journal of Anxiety Disorders, 27(1), 33–46. [DOI] [PubMed] [Google Scholar]
  57. Palminteri S, Justo D, Jauffret C, Pavlicek B, Dauta A, Delmaire C, … & Pessiglione M (2012). Critical roles for anterior insula and dorsal striatum in punishment-based avoidance learning. Neuron, 76(5), 998–1009. [DOI] [PubMed] [Google Scholar]
  58. Polak AR, Witteveen AB, Reitsma JB, & Olff M (2012). The role of executive function in posttraumatic stress disorder: A systematic review. Journal of Affective Disorders, 141(1), 11–21. [DOI] [PubMed] [Google Scholar]
  59. Preuschoff K, Quartz SR, & Bossaerts P (2008). Human insula activation reflects risk prediction errors as well as risk. Journal of Neuroscience, 28(11), 2745–2752. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Price RB, Brown V, & Siegle GJ (2019). Computational modeling applied to the dot-probe task yields improved reliability and mechanistic insights. Biological Psychiatry, 85(7), 606–612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Privratsky AA, Bush KA, Bach DR, Hahn EM, & Cisler JM (2020). Filtering and model-based analysis independently improve skin-conductance response measures in the fMRI environment: Validation in a sample of women with PTSD. International Journal of Psychophysiology, 158, 86–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Raij T, Nummenmaa A, Marin MF, Porter D, Furtak S, Setsompop K, & Milad MR (2018). Prefrontal cortex stimulation enhances fear extinction memory in humans. Biological Psychiatry, 84(2), 129–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Redish AD, Jensen S, Johnson A, & Kurth-Nelson Z (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychological Review, 114(3), 784. [DOI] [PubMed] [Google Scholar]
  64. Resick PA, Nishith P, Weaver TL, Astin MC, & Feuer CA (2002). A comparison of cognitive-processing therapy with prolonged exposure and a waiting condition for the treatment of chronic posttraumatic stress disorder in female rape victims. Journal of Consulting and Clinical Psychology, 70(4), 867. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Ross MC, Lenow JK, Kilts CD, & Cisler JM (2018). Altered neural encoding of prediction errors in assault-related posttraumatic stress disorder. Journal of Psychiatric Research, 103, 83–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Schönberg T, Daw ND, Joel D, & O’Doherty JP (2007). Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. Journal of Neuroscience, 27(47), 12860–12867. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Schönberg T, O’Doherty JP, Joel D, Inzelberg R, Segev Y, & Daw ND (2010). Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson’s disease patients: evidence from a model-based fMRI study. Neuroimage, 49(1), 772–781. [DOI] [PubMed] [Google Scholar]
  68. Steiger F, Nees F, Wicking M, Lang S, & Flor H (2015). Behavioral and central correlates of contextual fear learning and contextual modulation of cued fear in posttraumatic stress disorder. International Journal of Psychophysiology, 98(3), 584–593. [DOI] [PubMed] [Google Scholar]
  69. Stein MB, Kennedy CM, & Twamley EW (2002). Neuropsychological function in female victims of intimate partner violence with and without posttraumatic stress disorder. Biological Psychiatry, 52(11), 1079–1088. [DOI] [PubMed] [Google Scholar]
  70. Stephan KE, & Mathys C (2014). Computational approaches to psychiatry. Current Opinion in Neurobiology, 25, 85–92. [DOI] [PubMed] [Google Scholar]
  71. Suarez-Jimenez B, Albajes-Eizagirre A, Lazarov A, Zhu X, Harrison BJ, Radua J, … & Fullana MA (2020). Neural signatures of conditioning, extinction learning, and extinction recall in posttraumatic stress disorder: a meta-analysis of functional magnetic resonance imaging studies. Psychological Medicine, 50(9), 1442–1451. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Van Otterlo M, & Wiering M (2012). Reinforcement learning and Markov decision processes. In Reinforcement Learning (pp. 3–42). Springer, Berlin, Heidelberg. [Google Scholar]
  73. Wang Q, Luo S, Monterosso J, Zhang J, Fang X, Dong Q, & Xue G (2014). Distributed value representation in the medial prefrontal cortex during intertemporal choices. Journal of Neuroscience, 34(22), 7522–7530. [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Weathers FW, Bovin MJ, Lee DJ, Sloan DM, Schnurr PP, Kaloupek DG, … & Marx BP (2018). The Clinician-Administered PTSD Scale for DSM–5 (CAPS-5): Development and initial psychometric evaluation in military veterans. Psychological Assessment, 30(3), 383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Woon FL, Farrer TJ, Braman CR, Mabey JK, & Hedges DW (2017). A meta-analysis of the relationship between symptom severity of posttraumatic stress disorder and executive function. Cognitive Neuropsychiatry, 22(1), 1–16. [DOI] [PubMed] [Google Scholar]
  76. Wunderlich K, Dayan P, & Dolan RJ (2012). Mapping value based planning and extensively trained choice in the human brain. Nature Neuroscience, 15(5), 786–791. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1747963_Sup_info
NIHMS1747963-supplement-1747963_Sup_info.pdf (1.5MB, pdf)

RESOURCES