Dynamic threat–reward neural processing under semi‐naturalistic ecologically relevant scenarios

Abstract

Studies of affective neuroscience have typically employed highly controlled, static experimental paradigms to investigate the neural underpinnings of threat and reward processing in the brain. Yet our knowledge of affective processing in more naturalistic settings remains limited. Specifically, affective studies generally examine threat and reward features separately and under brief time periods, despite the fact that in nature organisms are often exposed to the simultaneous presence of threat and reward features for extended periods. To study the neural mechanisms of threat and reward processing under distinct temporal profiles, we created a modified version of the PACMAN game that included these environmental features. We also conducted two automated meta‐analyses to compare the findings from our semi‐naturalistic paradigm to those from more constrained experiments. Overall, our results revealed a distributed system of regions sensitive to threat imminence and a less distributed system related to reward imminence, both of which exhibited overlap yet neither of which involved the amygdala. Additionally, these systems broadly overlapped with corresponding meta‐analyses, with the notable absence of the amygdala in our findings. Together, these findings suggest a shared system for salience processing that reveals a heightened sensitivity toward environmental threats compared to rewards when both are simultaneously present in an environment. The broad correspondence of our findings to meta‐analyses, consisting of more tightly controlled paradigms, illustrates how semi‐naturalistic studies can corroborate previous findings in the literature while also potentially uncovering novel mechanisms resulting from the nuances and contexts that manifest in such dynamic environments.

We examined the neural mechanisms of simultaneous threat and reward processing under distinct temporal profiles using a semi‐naturalistic experimental paradigm. Our findings revealed a highly distributed system related to threat imminence and a smaller system related to reward imminence, neither of which involved the amygdala, which otherwise largely overlapped with the broader literature (meta‐analyses).

Practitioner Points.

• Using a semi‐naturalistic environment consisting of simultaneous threats and rewards, we uncovered a highly distributed set of regions (system) related to threat imminence and a much smaller system related to reward imminence.

• Unexpectedly, neither system involved the amygdala.

• There was broad correspondence of these two systems with corresponding meta‐analyses consisting of thread and reward experiments using more controlled paradigms, underscoring the value of examining affect in dynamic, interactive environments.

1. INTRODUCTION

Experimental designs in affective neuroscience traditionally adopt a reductionist approach to achieve the required control over experimental factors. This approach enables researchers to contrast specific conditions, without unwanted confounding variables, that ostensibly reflect underlying affective processes, leading to inferences about function‐to‐structure alignment (Henson, 2005). Using this approach, researchers have mostly contrasted stimuli with negative and positive affective salient features to neutral stimuli. Earlier work strongly implicated the amygdala (Pourtois et al., 2013; Sander et al., 2003) as an important site for affective processing. More recent work has also examined the role of the bed nucleus of the stria terminalis (Lebow & Chen, 2016), the visual cortex (Li et al., 2019; Markovic et al., 2014), the cingulate cortex (Shackman et al., 2011), and the insula (Uddin, 2015), among others. In sum, the use of reductionist experimental paradigms has provided almost all of our considerable knowledge regarding the brain's sensitivity to affective salient features and scenarios in the environment.

However, the success of this approach has relied almost exclusively on tightly constrained, unnaturalistic, artificial stimuli and environments and has left us with little knowledge of how the brain processes affect in more naturalistic settings. It also seems there has been little consideration given to whether constrained paradigms adequately characterize the complexity and interactions of the world that the human brain has evolved to solve (Krakauer et al., 2017), and it may be a mistake to assume that neural patterns and their resulting behavior(s) in tightly controlled settings are representative of what happens in the natural world (Cisek, 2019, 2022). Another major limitation of the reductionist approach is that findings may reify the selectivity of specific brain regions and neural circuits with respect to the specific process being investigated (Friston et al., 2006; Pessoa, 2023). Although practical considerations—for instance, BOLD signal cannot be measured with a mobile fMRI device—justify some constraints on environments in affective experiments, they do not likely justify total reliance on the extreme level of constraint that is typically used. A shift toward more naturalistic, ecologically relevant paradigms (which we refer to as semi‐naturalistic) for examining how the brain addresses complexity in the environment therefore seems warranted. Tightly controlled experiments can provide a foundation for explanations of neural functioning, but greater naturalism and relevance of the experimental scenarios is needed to assess their real‐world plausibility.

Early adopters of more naturalistic experimental conditions used movie and story‐telling paradigms (for a review in the affective literature, see Saarimäki, 2021). This approach, often called naturalistic viewing, introduced more temporal dynamics and complexity in the stimuli than previous tightly controlled, temporally static paradigms (Hasson et al., 2004). One issue with naturalistic viewing, however, was that its initial value was seen as a way to enhance the common resting‐state paradigms in functional connectivity research, particularly due to its ability to reduce head motion and limit participant fatigue (Vanderwal et al., 2019). While still an important methodological contribution, naturalistic viewing used in this way entailed passive viewing/listening with no overt behavior. From an evolutionary vantage point, this method is limited for understanding the brain in its natural context, as a major feature of the brain's evolutionary development has been to guide behavior responses to environmental changes (Anderson & Chemero, 2016; Cisek, 2019; Nastase et al., 2020). Therefore, naturalistic viewing was a step in the direction of semi‐naturalistic environments, but lacked an important feature of real‐world environments—agency—that could be implemented in the fMRI experiment relatively easily. An important component of naturalism is allowing participants to move and interact with the environment. Adding this component to an environment leads to many distinctions between constrained and semi‐naturalistic environments that, in turn, grant the semi‐naturalistic environment much greater complexity. First, because such environments are interactive, they instill the participant with agency (Bach et al., 2014; Mobbs et al., 2007, 2009). Second, because the participant (or their avatar) is moving, the stimuli are non‐stationary through time with respect to the participant's (or avatar's) position. This non‐stationarity can have many influences on the participant's perception of a stimulus, one of which will be its general salience, but also its affective salience (Meyer et al., 2019). Third, participants will routinely face decisions that involve many choices, certainly more than two (Gold et al., 2015). Fourth, decisions are expanded temporally and often embedded in larger overall goals (Orasanu & Connolly, 1993). This feature leads to a few other effects: (a) long‐term goals constrain immediate decisions; (b) decisions that are made may not be enacted immediately; (c) multiple decisions may need to be enacted in a specific sequence; (d) decisions later in the sequence may be further constrained by earlier decisions/actions (i.e., form a hierarchical decision tree); and (e) decisions can be modified/changed before they are enacted. Combining these features leads to a level of complexity in semi‐naturalistic environments that is purposefully unavailable in constrained environments.

Results from studies using semi‐naturalistic environments have in some cases confirmed our understanding built on more reductive approaches, but in other cases have advanced our understanding by challenging those findings. For example, the amygdala has been associated with fear for decades, from studies of Kluver–Bucy syndrome, where fear reactivity is reduced due to amygdala lesions (Klüver & Bucy, 1939; LeDoux, 2007). Recent semi‐naturalistic studies, however, have challenged the theory that the amygdala processes an abstract concept like “fear.” For instance, researchers found that activity in a subpopulation of neurons in the basolateral amygdala nucleus varied over time in the presence of constant threat. The variations in activity were tied to the rats' behaviors, but did not distinguish between traditional approach‐ and avoidance‐related behaviors such as foraging or fleeing. (Amir et al., 2015, 2019). These results suggest a more nuanced role of the amygdala in affective salience processing—rather than activity reflecting an abstract concept like “fear,” it may represent the motivation to act in the face of a threat. Results like these underscore the value of examining affect in dynamic, interactive environments, which greatly motivated our work.

Another major advantage of semi‐naturalistic experimental paradigms is the extended temporal dimension they present, where individual “trials” may occur over many seconds or even minutes, unlike tightly controlled paradigms where trials typically last only several seconds. The temporal evolution during extended periods of threat and reward has revealed insights regarding anticipatory responses (Hur et al., 2020; Somerville et al., 2013), and recently, that such responses are related to the imminence of specific environmental features (Murty et al., 2023).

However, affective neuroscience studies, semi‐naturalistic or not, typically study threat and reward elements independently, rarely incorporating both concurrently in their paradigms. Although semi‐naturalistic paradigms involving threat or reward processing offer a unique opportunity for examining the temporal progression of responses to these features (Mobbs et al., 2020), examining the effects from their coexistence remains understudied and how the brain resolves the conflict between two salient yet oppositely‐valenced features remains an area for further examination (Choi et al., 2014). Such an endeavor is necessary, given that both threatening and rewarding features exist concurrently in natural environments, forcing organisms to contend with this joint overlap to ensure survival. Thus, despite our existing knowledge of the neural substrates that process threats and rewards, our understanding of the integration of threat and reward features when they are jointly present and interacting remains limited.

To study the integration of simultaneous threats and rewards in a semi‐naturalistic environment, we created a modified version of the PACMAN game (PACPAL) that contained simultaneous threat and reward environmental features. Due to the limited number of studies in the literature that have used semi‐naturalistic environments, particularly with salient features, our hypotheses were largely exploratory. As with previous work, we hypothesized that some aspects of affective processing in semi‐naturalistic environments would be consistent with what has been learned in constrained environments; however, we also anticipated that the complexity of the semi‐naturalistic environment would reveal additional mechanisms that were previously hidden. More specifically, our aim was to examine the brain mechanisms of affect (threat, reward) processing under specific temporal profiles (extended, discrete) when threat and reward features were simultaneously present in the environment.

Before continuing, it is worth clarifying several terms and concepts used in the literature and this paper, and operationalizing the variables used in our experimental design. The term threat can be broadly defined (e.g., physical, abstract, imaginary), but we wanted to limit our definition to physical threats. In a natural environment, physical threats are motivationally significant because they cause harm. In our study, the threat is not to the participant, but to the participants' avatar, but is analogous to a real physical threat, because it produces harm and reduces survivability in the game. Likewise, we define reward as something in the environment that increases survivability of the participants' avatar. These terms encapsulate key features of affective salience—the inclination to orient to certain features of an environment based on their motivational significance, learned through previous affective history with those features (Todd et al., 2012). From an evolutionary perspective, “motivational relevance” can be more concretely defined as “homeostatic relevance,” that is, the anticipated benefit/detriment of an environmental feature to the participant's internal status and, ultimately, survivability (Seeley, 2019). This latter definition is more aligned to the evolutionary steps the brain has taken throughout its refinement across the animal kingdom (Cisek, 2019). Again, here, it is not the survivability of the participant, but rather their avatar, that defines threats and rewards. Although the game context does not allow for the threats and rewards to directly affect the participant, survivability of the avatar is nevertheless extremely motivating for participants and has a strong influence on their behavior during gameplay. It has been suggested that participants engaged in gameplay may even internalize the motivations felt by their avatars, a phenomenon referred to as “avatar identification” (Allen & Anderson, 2021; Song & Fox, 2016). Additionally, for the remainder of this paper we use salience in lieu of affective salience for the sake of parsimony.

Importantly, such salience fits into the broader category of emotion, a term that can be difficult to define (Cabanac, 2002). Furthermore, there remains a longstanding proclivity in the psychological sciences for using “epistemically sterile” terms to describe mental functions and processes, such as emotion, cognition, perception, and executive functioning (Pessoa et al., 2022). An exemplar can be found in a recent rebuke to the William James remark “Everyone knows what attention is…,” as the definition of attention is both vague and colloquial and logically circular in that it represents both the phenomena being studied and the process that brings about its own existence (Hommel et al., 2019). Rather than rely on such broad terms as emotion and attention, we operationally defined the salience of threats and rewards based on measurements derived from the experimental environment. Threats were defined as features of the environment that caused detriment, whereas rewards were defined as features of the environment that caused benefit. In the PACPAL game, threats were “ghosts” and rewards were “dots.” The salience of the threats and rewards was defined based on the imminence (inverse of distance) of ghosts and dots, respectively, to the participant avatar. We use “imminence” in accordance with a prominent model of prey and predator interactions, which closely resembles our experimental paradigm (Fanselow & Lester, 1988). Our variables, therefore, constitute states of the environment that lead to varying levels of perceived salience. It is worthwhile noting that when we use the term threat, we do not necessarily mean fear or anxiety, which are internal states resulting from threat (for a roundtable discussion on this distinction, see Grogans et al., 2023).

2. MATERIALS AND METHODS

Participants played a modified version of the classic PACMAN arcade game in a scanner. The experiment was displayed on a Mitsubishi LCD projector (model XL30U), with a resolution of 1920 × 1200 pixels. The viewing distance from the display mirror to eyelid was 11.5 cm, and the distance from the screen to the mirror was 79 cm, giving a total viewing distance of 90.5 cm. When projected in this manner, the size of the game presentation subtended approximately 13° of visual angle.

2.1. Participants

Forty‐three adults aged 18–41 (20 females; average age = 23.9 years; SD = 4.81 years) with normal or corrected‐to‐normal vision were recruited to participate in the study, which was approved by the Institutional Review Board of Indiana University. One participant was excluded from analyses because a modification to the game was implemented following participation, and another was excluded due to processing errors from poor data collection. An additional participant had excessive motion during the final three runs of the experiment, as determined by MRIQC quality assurance checks (Esteban et al., 2017); therefore, the participant was excluded from analyses. Another three participants had a subset of their data excluded from analysis due to software error that failed to sync the experiment with the scanner, resulting in unreliable event timings. Therefore, three participants had a subset of their data excluded and three were completely excluded, resulting in a final cohort of 40 participants (19 females; average age = 24.1, SD = 4.93).

2.2. Experimental paradigm

Participants completed six 10‐min runs of the modified PACMAN game, where they were instructed to play to the best of their abilities (Figure 1). During each trial, a single dot was displayed in the maze for participants to collect, resulting in a $0.05 bonus cumulatively added to participants' $30 base participation pay. A bonus pay floor ($5) and ceiling ($10) were set; however, participants were uninformed of these until completing the experiment, in order to encourage sustained effort. Fifteen dots could be collected during each trial, with only one revealed at a time until collected. If all 15 dots were collected, the trial ended for a minimum of 3 s before proceeding to the next.

FIGURE 1.

Example schematic of the PACPAL gameplay where different environmental variables were measured to correlate with BOLD signal. Variables of note include the closest‐ghost‐to‐participant‐imminence (CGPI), the participant‐to‐dot‐imminence (PDI), closest‐ghost‐to‐dot‐imminence (CGDI), and a health (H) measurement that can also be interpreted as an urgency signal.

During each trial, two ghosts traversed the maze, alternating in 15 s intervals between two modes of movement that were indicated by their color, red or green. Two parameters described the ghosts' movement: closest direction % and speed options %. When red, the ghosts probabilistically sought to chase the participant by selecting the direction at a maze intersection that minimized their distance (increased imminence) to the participant's current location. Ghost speeds alternated between three set values upon reaching an intersection: slower than participant, participant speed, and faster than participant. After pairs of runs, the values of closest direction % and speed options % changed to account for practice effects and to reduce potential complacency during gameplay (Table 1). When green, the ghosts sought to orient to the revealed dot, selecting the direction (75% probability) at a maze intersection that brought them closer to the dot. During this period, ghost speeds alternated between slower than participant and participant speed. Making contact with a ghost, regardless of color, meant “death,” resulting in a $0.10 bonus deduction and trial termination for a minimum of 3 s. Earned money never dropped below $0.00 during gameplay, and participants received the bonus pay minimum of $5 if their gameplay earnings did not exceed $5.00.

TABLE 1.

Probabilistic chase direction and speed options across experiment runs when ghosts are red (parameters values are different and do not change when ghosts are green). The closest direction % parameter denotes the likelihood of each ghost at intersection selecting the direction that moves it closest to the participant's current location (i.e., chase). The ghosts speed options % parameter denotes the likelihood of each ghost at intersection selecting one of three speeds in relation to the participant's speed. These parameter value changes were included to reduce complacency during the experiment.

Runs	Closest direction %	Ghosts speed options %
1–2	90	Slower (33.3)
		Same (33.3)
		Faster (33.3)
3–4	95	Slower (20)
		Same (20)
		Faster (40)
5–6	100	Slower (16.6)
		Same (33.3)
		Faster (50)

Another modification to the original PACMAN game was the addition of a health bar, decreasing at a rate of 0.1 pixel/s, regardless of participant movement. A $0.10 bonus deduction and minimum trial termination of 3 s occurred if health reached zero; however, collecting a dot resulted in a health increase of 25 pixels, with no upper health limit.

The original PACMAN game was played with a joystick, but due to the unavailability of an MRI‐safe joystick, a button box was used instead, with participants using their left middle finger for upward movement, left index finger for downward movement, right index finger for rightward movement, and right middle finger for leftward movement. Continual movement required holding down a button, during which time participant speed was constant. A 10‐min practice session occurred in the scanner prior to the experiment, to familiarize participants with the paradigm and button mapping.

2.3. Imaging data

Imaging data were collected on a Siemens Tesla 3 T Prisma, whole‐body MRI system using a 64‐channel head coil. A T1‐weighted anatomical volume was acquired (TR = 2400 ms, TE = 2.68 ms, flip angle = 8°, inversion time = 1060 ms, and 0.8 × 0.8 × 0.8 mm isometric voxel size). This was followed by a spin‐echo field mapping sequence to reduce magnetic field inhomogeneities (Jezzard & Clare, 1999), comprising two brief acquisitions with opposite phase encoding directions. Functional data consisted of using a T2*‐weighted BOLD interleaved EPI sequences (TR = 1000 ms, TE = 30 ms, multiband factor = 4, 48 slices, and 3 mm isotropic voxels). Each functional BOLD acquisition consisted of 600 volumes, with the first six discarded to obtain steady‐state magnetization. Each functional acquisition was preceded by a single‐band reference (SBRef) with matching parameters.

2.4. Imaging analysis

Imaging data were preprocessed using fMRIPrep 22.0.1 (Esteban et al., 2019). The anatomical and functional preprocessing sections are described verbatim from the fMRIPrep reports, recommended by the developers to ensure transparency and future reproducibility.

2.4.1. Anatomical

A total of 1 T1‐weighted (T1w) images were found within the input BIDS dataset. The T1‐weighted (T1w) image was corrected for intensity non‐uniformity with N4BiasFieldCorrection (ANTs 2.3.3), and used as T1w‐reference throughout the workflow (Tustison et al., 2010). The T1w‐reference was then skull‐stripped with a Nipype implementation of the antsBrainExtraction.sh workflow (from ANTs), using OASIS30ANTs as target template. Brain tissue segmentation of cerebrospinal fluid (CSF), white matter (WM), and gray matter (GM) was performed on the brain‐extracted T1w fast (FSL 5.0.9). Brain surfaces were reconstructed using recon‐all (Freesurfer), and the brain mask estimated previously was refined with a custom variation of the method to reconcile ANTs‐derived and FreeSurfer‐derived segmentations of the cortical GM of Mindboggle (Klein, 2017). Volume‐based spatial normalization to the MNI152NLin6Asym standard space (FSL's MNI ICBM 152 non‐linear 6th Generation Asymmetric Average Brain Stereotaxic Registration Model) was performed through nonlinear registration with antsRegistration (ANTs 2.3.3), using brain‐extracted versions of both T1w reference and the T1w template.

2.4.2. Functional

For each of the six BOLD runs found per subject (across all tasks and sessions), the following preprocessing was performed. First, a reference volume and its skull‐stripped version were generated by aligning and averaging one SBRef. A deformation field to correct for susceptibility distortions was estimated based on fMRIPrep's fieldmap‐less approach. The deformation field is that resulting from co‐registering the BOLD reference to the same‐subject T1w‐reference with its intensity inverted. Registration was performed with antsRegistration (ANTs 2.3.3), and the process regularized by constraining deformation to be nonzero only along the phase‐encoding direction, and modulated with an average fieldmap template. Based on the estimated susceptibility distortion, a corrected EPI (echo‐planar imaging) reference was calculated for a more accurate co‐registration with the anatomical reference. The BOLD reference was then co‐registered to the T1w reference using bbregister (FreeSurfer) which implements boundary‐based registration (Greve & Fischl, 2009). Co‐registration was configured with six degrees of freedom. Head‐motion parameters with respect to the BOLD reference (transformation matrices, and six corresponding rotation and translation parameters) were estimated before any temporal filtering using mcflirt (FSL 5.0.9). First, a reference volume and its skull‐stripped version were generated using a custom methodology of fMRIPrep. The BOLD time‐series were resampled onto their original, native space by applying a single, composite transform to correct for head‐motion and susceptibility distortions. These resampled BOLD time‐series are referred to as preprocessed BOLD in original space, or just preprocessed BOLD. The BOLD time‐series were resampled into standard space, generating a preprocessed BOLD run in MNI152NLin6Asym space. First, a reference volume and its skull‐stripped version were generated using a custom methodology of fMRIPrep. Three global signal time‐series were calculated based on the preprocessed BOLD, extracted within the CSF, the WM, and the whole‐brain masks. Confound time series derived from head motion estimates and global signals were expanded with the inclusion of temporal derivatives and quadratic terms for each (Satterthwaite et al., 2013). All resamplings can be performed with a single interpolation step by composing all the pertinent transformations (i.e., head‐motion transform matrices, susceptibility distortion correction when available, and co‐registrations to anatomical and output spaces). Gridded (volumetric) resamplings were performed using antsApplyTransforms (ANTs), configured with Lanczos interpolation to minimize the smoothing effects of other kernels (Lanczos, 1964).

2.5. fMRI analyses

fMRI analyses were conducted using FSL's fMRI Expert Analysis Tool (FEAT). The fMRIPrep‐processed data were spatially smoothed (4 mm FWHM), while additional preprocessing steps in FEAT were skipped, including normalization, as these steps had already occurred in fMRIPrep and the processed data were already aligned to FSL's MNI152 standard space.

The unconstrained nature of our paradigm allowed us to assess how continuous changes in the environment from moment‐to‐moment affected brain activation, in addition to some discrete, categorical events. To be clear, the design did not include independent variables, therefore, a “standard” GLM analysis examining the contributions of categorical independent variables (i.e., blocks or trials of specific experimental conditions) was not possible. Instead, continuous and categorical dependent variables extracted from the environment were used to “conditionalize” each volume (Figure 2). The interpretation of these continuous dependent variables (which we will call extended explanatory variables [EVs]) is similar to interpreting a correlation or, even moreso, a multiple regression.

FIGURE 2.

Visualization of the relations between design matrix explanatory variables (EVs) the gameplay environment. The EVs are shown at every volume over the course of an entire scan run. Gameplay environment screenshots are shown, detailing the gameplay environment at two example volumes. Only the extended threat and reward EVs (CGPI, PDI) are included as examples. (a) We chose the first screenshot example at volume 114 of the 600 volume run (1‐s repetition time [TR]) of gameplay where the participant‐to‐dot‐imminence (PDI) is greater than the closest‐ghost‐to‐participant‐imminence (CGPI). This is reflected in the standardized values of the design matrix. (b) We chose the second screenshot example at volume 493 of the gameplay where closest‐ghost‐to‐participant‐imminence (CGPI) is greater than the participant‐to‐dot‐imminence (PDI).

Eleven gameplay EVs were included in the design matrix. These were closest‐ghost‐to‐participant‐imminence (CGPI), CGPI‐derivative (CGPID), participant‐to‐dot‐imminence (PDI), PDI‐derivative (PDID), closest‐ghost‐to‐dot‐imminence (CGDI), CGDI‐derivatives (CGDID) health (H), two ghost colors denoting their behavior (GC_red, GC_green), caught‐by‐ghosts (Caught), and collect the dot (CollectDot). The imminence EVs, CGPI, PDI, and CGDI were produced by inverting and normalizing in‐game grid path distance between the participant avatar and either the closest ghost (CGPI) or the dot (PDI), or between the dot and its closest ghost (CGDI). Because distances changed every volume based on movements of the avatar and the ghosts, and because we did not discretize the values, these EVs were continuous, not categorical. CGPI and PDI were considered to reflect continuous variations in threat and reward, respectively, whereas CGDI was an additional environmental variable not directly linked to affective processing. CGDI was considered to reflect a different aspect of threat than CGPI, but one that was less relevant to our research question. The derivatives of the CGPI, PDI, and CGDI EVs, CGPID, PDID, and CGDID, reflected the rate of change in the imminence values (velocity) throughout gameplay. The health (H) EV reflected the health level of the participants' avatar, which also varied continuously across volumes. Lower health was equated with higher urgency. Like threat and reward, urgency is motivating, but rather than directly motivating either avoidance (ghost) or approach (dot), urgency (health) acts as a context that modifies the prioritization of avoidance and approach behaviors. For a typical participant, when possible, they plan to navigate toward the dot. When ghosts intervene, participants may change their plan to try to avoid them. It is this balance between approach and avoidance behaviors that is influenced by health, with participants more apt to risk capture to acquire a dot if their urgency to improve their health is high.

The remaining EVs were categorical/discrete (i.e., not continuous). The ghost behavior EVs (GC_red, GC_green) specified the ghosts' movement/behavior mode and color: red when they were orienting to the participant and green when they were orienting to the dot. CollectDot events specified the moment (volume) when participants consumed a dot, whereas Caught events specified the moment (volume) when participants were caught by a ghost. Importantly, intertrial interval (ITI) periods were not included (were left unspecified) in the design matrix, thus preventing the design matrix from being overspecified and possibly rank deficient.

Given the semi‐naturalistic aspects of our paradigm, where the gameplay itself determined the value of the EVs throughout the course of the experiment, we expected that there would be higher than usual correlations (Figure S1a) between the EVs. Based on this, we anticipated variance inflation factors (VIF, an approximation of multicollinearity) to be elevated relative to a more standard task‐based fMRI experimental design, where categorical independent variables are often designed to be orthogonal. To limit collinearity, we applied a general rule of limiting VIFs to under 10 (Neter et al., 1989). Of particular importance was that including interaction terms (e.g. CGPI × PDI) in the design matrix did elevate VIF to an unacceptable degree. Although a number of interaction terms would have been interesting to explore, especially ones that may have captured coactivation of approach and avoidance, our final design matrix did not include any interactions terms to avoid it being rank deficient. The resulting model contained no VIF exceeding 1.52 (Figure S1b), indicating that multicollinearity was not a concern (Mumford et al., 2015).

It is worthwhile expanding on the rationale for inclusion of two threat and two reward EVs that, on the surface, may seem to be highly interdependent. Although CGPI and Caught EVs were both considered “threat” EVs and PDI and CollectDot were both considered “reward” EVs, we expected the CGPI and PDI EVs to represent extended, dynamically changing aspects of threat or reward, respectively, including phenomena like expectancy. They were modeled as continuous variables. On the other hand, we expected Caught and CollectDot EVs to represent the discrete moments of unusually heightened threat or reward, respectively. They were modeled as categorical variables. Based on the differences in temporal pattern between extended and discrete EVs, we felt that they would capture aspects of threat and reward that were different enough to warrant their separate inclusion in the design matrix. This basis was partially born out by the relatively low correlation between extended and discrete EVs (Figure S1a) and their relatively low VIF values when jointly modeled (Figure S1b).

A first‐level GLM analysis was performed to fit the design matrix with all of the EVs described above. EVs were entered into FSL using the FEAT EV Basic shape of “Custom (1 entry per volume),” which allows one to specify a specific value at every volume by pointing to a text file, and convolved with a double‐gamma HRF. Additional nuisance regressors included CSF and WM time series along with their corresponding derivatives, a combined CSF and WM mask, six motion parameters, and their derivatives, fMRIPrep‐generated cosine confounds that apply a high‐pass filter with a 128‐s cutoff, and six regressors specifying the six non‐steady state volumes at the beginning of each run. First‐level contrasts were constructed to examine the spectrum of threat and reward (i.e., high, low) and interactions between the two affectively salient states, in particular, CGPI, PDI, Caught, CollectDot, CGPI—PDI, and Caught—CollectDot. Specifically, the CGPI and PDI contrasts reflect the outputs of the identically‐named EVs, Caught and CollectDot denote the contrasts of the discrete EVs with non‐gameplay rest periods (ITIs) as the reference, and CGPI—PDI and Caught—CollectDot reflect contrasts of threat and reward variables regardless of temporal profile.

Higher level analyses consisted of two parts. A second‐level GLM analysis was performed with a fixed‐effects model, combining multiple scan runs within‐subject. This procedure ignored participants' cross‐scan variances from the first‐level, with the assumption that within‐subject variance is stable. A third, group‐level analysis was then performed with a mixed‐effects model (FLAME 1) procedure that treated participants as a random effect. Variances from lower‐level analyses were passed to the group‐level, ensuring better inferences with regards to the broader population. Whole‐brain multiple comparisons tests were performed using FSL's cluster thresholding method, where clusters were defined as contiguous voxels that were thresholded with z > 4.05 and the resulting clusters were tested for p < .05 significance. We chose a stricter voxel‐wise threshold than the FSL default of z > 3.1, to reduce cluster sizes so that neighboring clusters did not become conjoined, thereby making it possible to find centers of mass for these clusters.

2.6. Meta‐analyses

Two activation likelihood estimation meta‐analyses were conducted with text extraction from abstracts in the NeuroSynth database (Yarkoni et al., 2011), using the NiMARE automated meta‐analysis package (Salo et al., 2022). Each analysis contained a salience condition (threat, reward), restricted to studies involving only “threat” or “reward” respectively. The threat meta‐analyses included 228 studies, while the reward meta‐analyses included 922 studies. Each study included in the meta‐analyses was assumed to have a sample size of N = 20, as this information is not discernible from the NeuroSynth database. Family‐wise error corrections were performed using 10,000 Monte Carlo iterations for constructing a cluster‐size null distribution, with a cluster‐defined p‐value threshold of 0.05. It should be noted that generating automated meta‐analysis somewhat complicates interpretation, which is discussed later.

3. RESULTS

Of the gameplay EVs included in the design matrix, the main analyses were based on four EVs, CGPI, PDI, caught‐by‐ghosts (Caught,) and CollectDot, due to the expectation that they exemplified threat or reward aspects of the semi‐naturalistic environment. The remaining EVs (CGPID, PDID, CGDI, CGDID, H, GC_red, GC_green) were considered less directly related to threat or reward, but also captured important characteristics of the environment that should be accounted for in the model. They were included in the model to capture nuisance variance attributable to those characteristics, but their relevance to threat and reward processing was considered indirect. For completeness, the results for those EVs are presented in the Supplementary Materials (Figures S2–S4). Two of the threat and reward EVs (CGPI, PDI) had an extended temporal profile. Because these were continuous variables, the first‐level betas derived from the model fit can be interpreted as a partial correlation between imminence and BOLD, which would be similar to a contrast of high versus low imminence (Figures 3 and 4a). The other two threat and reward EVs (Caught, CollectDot) had a discrete temporal profile. Because these were categorical variables, their contributions can be considered a contrast with the ITI (baseline) as a reference (Figures 3 and 4b). The maps in Figures 3 and 4 give a broad overview of the regions recruited by threat and reward aspects of gameplay.

FIGURE 3.

Statistical maps (z > 4.05, p < .05 cluster corrected) of the threat explanatory variables (EVs): (a) closest‐ghost‐to‐participant‐imminence (CGPI; extended time period) and (b) caught‐by‐ghosts (Caught; discrete time period). Despite the Caught contrast essentially representing a task‐on versus task‐off comparison, its similarity to CGPI, essentially a contrast of high versus low threat imminence, underscores how both reveal the whole network of brain regions being recruited during threat processing, regardless of their temporal differences. Notably, robust activation was observed ventral attention (salience) network regions to high threat immense, whereas default mode network (DMN) network regions were more sensitive to low threat imminence. Dotted circles denote regions observed in our meta‐analyses but not our results. Regions were color‐coded based on their seven‐network designation as described in Yeo et al. (2011). L = left, R = right. Color bars are denoted in z‐values.

FIGURE 4.

Statistical maps (z > 4.05, p < .05 cluster corrected) of the reward EVs. (a) participant‐to‐dot‐imminence (PDI; extended time period) and (b) CollectDot (discrete time period). Notably, the temporal profile of the contrasts appeared to affect the neural distribution, particularly the attenuation observed in PDI. Dotted circles denote regions observed in our meta‐analyses but not our results. Regions were color‐coded based on their seven‐network designation as described in Yeo et al. (2011). L = left, R = right. Color bars are denoted in z‐values.

To examine differences between threat versus reward processing we separately contrasted threat and reward for extended (CGPI, PDI) and discrete (Caught, CollectDot) EVs (Figure 5). The statistical map (z > 4.05, p < .05 cluster corrected) of the CGPI—PDI contrast revealed greater sensitivity to threat in dorsolateral prefrontal cortex (dlPFC), anterior/fronto insular cortex (AIC/FIC), premotor cortex, mid/precentral gyrus, precuneus, occipital fusiform gyrus, lateral occipital cortex, and occipital pole, whereas the nucleus accumbens and supplementary motor area were more sensitive to reward (Figure 5a). At first glance, the vmPFC, mPFC, and PCC also appear to be more sensitive to reward, however this is largely due to the strong negative correlation with CGPI rather than a strong positive correlation with PDI (Figures 3 and 4a). The Caught—CollectDot contrast was largely identical, with the exception of no cluster present in the dlPFC but periaqueductal gray sensitivity to Caught (Figure 5b). These results suggest that, in a semi‐naturalistic environment with both threat and reward features, there is a distributed set of regions exhibiting spatially dependent sensitivity to extended threat than reward features and sensitivity to extreme threat than reward events.

FIGURE 5.

Statistical map contrasts (z > 4.05, p < .05 cluster corrected) between (a) extended threat and reward EVs (closest‐ghost‐to‐participant‐imminence [CGPI], participant‐to‐dot‐imminence [PDI]) and (b) discrete threat and reward EVs. A threat bias is observed, as most neural regions with the exception of the default mode network (DMN) display greater sensitivity to threat than reward. This was in spite of the lack of physical sensations of aversion, underscoring the evolutionary importance of threat detection in one's environment. Positive z‐scores may simply reflect less strongly negative effects. Dotted circles denote regions observed in our meta‐analyses but not our results. Regions were color‐coded based on their seven‐network designation as described in Yeo et al. (2011). L = left, R = right. Color bars are denoted in z‐values.

Finally, we performed two meta‐analyses on previous studies that used less‐naturalistic threat and reward stimuli/environments than our study. Based on the emphasis in the literature on threat and reward recruiting specific regions such as the amygdala and basal ganglia, our expectation was that the meta‐analyses would show less distributed results than we found with our more‐naturalistic task. However, there was considerable overlap between our results and meta‐analyses for both threat and reward EVs (Figure 6), regardless of whether the EVs were extended or discrete (although see our discussion of lower correspondence with the PDI contrast). Perhaps the most notable discrepancy was the absence of amygdala in our findings compared to the meta‐analyses.

FIGURE 6.

Term‐based meta‐analyses (z > 3.09, p < .05 cluster corrected), derived from the NeuroSynth database. (a) Threat meta‐analysis resulting from all NeuroSynth abstracts containing the word “threat.” (b) Reward meta‐analysis resulting from all NeuroSynth abstracts containing the work “reward.” Regions were color‐coded based on their seven‐network designation as described in Yeo et al. (2011). L = left, R = right. Color bars are denoted in z‐values.

4. DISCUSSION

An important avenue for affective neuroscience research is to establish experimental paradigms that contain semi‐naturalistic, ecologically relevant scenarios, to better ensure that relations uncovered between neural mechanisms and behaviors are as close as possible to how they exist in nature. Using a semi‐naturalistic paradigm, we sought to examine the brain mechanisms of affect (threat, reward) processing when threat and reward features and events were jointly present in the environment. Our results revealed three key findings: (1) a highly distributed set of regions (system) related to threat imminence and much smaller system related to reward imminence, (2) unexpectedly, neither system involved the amygdala, and (3) both systems overlapped with systems determined through meta‐analysis of more controlled experiments. In total, these findings suggest that, in an environment co‐active with threats and rewards, approach and avoidance behaviors are produced by largely overlapping neural systems. They also reveal congruences between semi‐naturalistic and more‐controlled studies of affect, suggesting that semi‐naturalistic paradigms may be useful for uncovering novel mechanisms of affective processing that may result from the nuances and contexts that only manifest in more dynamic, interactive environments.

A key finding was a distributed system of many brain regions displaying sensitivity to threat (Figure 5). This was accompanied by a smaller distributed system for reward. Both systems were sensitive to discrete threat or reward events (being caught by ghosts and collecting a dot) and also sensitive to continuously varying imminence of threats and rewards (CGPI and PDI). Despite the smaller number of regions in the reward system, the systems were largely overlapping. These findings appear to correspond to a phenomenon often referred to as a threat/negativity bias, where threat (or more broadly affectively negative) features command greater perceptual and cognitive consideration (Baumeister et al., 2001; Cacioppo & Gardner, 1999). Threat and reward represent salient features of the environment and produce affective responses, but those responses are oppositely valenced. Their commonality with respect to salience may help explain how the brain's detection and processing of threat and reward features seems to involve similar anatomical and functional regions, like the anterior insula (Craig, 2009; Liu et al., 2011), and also analogous opioid and dopamine systems (Leknes & Tracey, 2008). Our results also reveal that threat and reward produce similar dynamic changes in neural activity patterns as levels of salience fluctuate over time. However, brain regions that show this pattern, such as AIC/FIC and dACC, tended to respond more strongly or more extensively to threat features. This threat bias makes sense from an evolutionary perspective, as failure for an organism to detect and respond to threat features, like predators, in its environment is more likely to result in death than failure to respond to reward features, like food, which may result in hunger (Mobbs et al., 2015). Oftentimes, salience studies examine threat and reward features in isolation, which is different from naturalistic contexts in which both exist and must be accounted for to reach an appropriate decision and act accordingly. Our results offer some clarification to this naturalistic dilemma, suggesting that greater neural processing and monitoring is dedicated to threat features.

However, might this threat bias finding in our specific experiment be due to some unintended confounding factor/influence, given the unconstrained and complex nature of the experimental paradigm? One possible alternative explanation may be that the extended reward (PDI) results were attenuated by the CGPI effects (and vice versa), given that threat or reward features did not exist exclusively in the environment. For example, moments of high reward imminence could simultaneously occur during moments of equally high threat imminence, such as instances where both participant and ghost(s) are close to the dot. Indeed, some degree of overlap exists and cannot be separated; however, we assessed the strength of this relationship by calculating the correlations between our EVs, finding that these values were modest, outside of the ghost behavior modes, and not substantially elevated. Additionally, our threat and reward findings during discrete time periods displayed remarkable similarities in patterns and regional effects, suggesting that reward effects were not being lost to the threat results.

Another possible counterpoint could be that the threat and reward metrics were not equally subjectively salient, as we did not systematically assess the degree to which the threat and reward features were perceived as such. Instead, threat and reward metrics were measured from the state of the environment, which directly relied on the expected behaviors of participants during gameplay in response to salient features (i.e., avoiding ghost threats, seeking dot rewards). This approach aligns with recent proposals for refocusing neuroscience toward the study of complex, naturalistic behaviors to better uncover the brain's vast functionalities (Pessoa et al., 2022). Furthermore, the threat bias results were found despite the fact that we did not attempt to induce threat using physical sensations often associated with aversion, such as electric shocks, jarring audio, or disturbing images. Indeed, direct, physical sensations are not always necessary to manipulate or induce salience; human defense and reward circuits are capable of inference rather than explicit experiences to determine the degree and severity of salient features (Baczkowski et al., 2023).

It is also worth considering whether the observed threat bias could simply be attributed to the framework of gains and losses in our paradigm. Specifically, gains and losses were structured such that obtaining dots earned $0.05 per dot and a health increase, whereas being caught incurred a penalty of $0.10 and the entire loss of health (game trials ended in participant avatar “death”). At first glance, it may appear that an individual in‐game threat engenders a harsher penalty relative to the benefits of obtaining an individual reward, thereby causing or significantly contributing to the negativity bias. However, once a dot is collected a new one appears, meaning that participants could earn more than $0.05 in an individual trial. This was often the case, as the average in‐game final bonus was $4.15, demonstrating that on average the accumulated monetary rewards per trial exceeded the incurred penalty. Admittedly, we did not vary these gains and losses, as determining the effect of these variations on threat and reward processing was not our objective. Our choices were guided by the theory that, in nature, the likelihood of an organism's demise is greater when failing to avoid and escape predator threats rather than failing to obtain rewards such as food (Fanselow & Lester, 1988). Nevertheless, future studies that incorporate variations in incentivization schedules will add to our overall understanding of covarying threat and reward processing.

Perhaps the most notable finding was the lack of amygdala sensitivity not only to threats, but also to rewards, that is, the amygdala did not appear to respond to salient events or to imminence‐dependent salient features (Figure 5). This finding is contradictory to the classical understanding of the important role the amygdala plays in salience processing, particularly Pavlovian fear conditioning (Adolphs et al., 1995; Davis, 1992; Davis & Whalen, 2001; LeDoux, 2003). However, some studies examining the role of the amygdala in the processing of threats have done so in a more ecological manner, and typically do not find strong amygdala responses to threat, but find a variety of results from weak/no responses to the opposite response (Meyer et al., 2019; Murty et al., 2023; Somerville et al., 2010). Certain ecological paradigms involving virtual predators have noted the presence of amygdala activation when threat imminence is high (Mobbs et al., 2007, 2009), whereas others have noted a lack of amygdala engagement under dynamic changes of threat monitoring (Somerville et al., 2010). A recent study of threat proximity found an interaction of distance by proximity effect in the right amygdala, whereby activation was observed when threat was proximal yet retreating (Meyer et al., 2019). Paré and Quirk (2017) have noted the overreliance of fear conditioning tasks for describing the role of the amygdala in affective salient processing, arguing that behavioral movements provide a more reliable correlation with activation (Paré & Quirk, 2017). Research has further demonstrated amygdala sensitivity to transitions between exploratory and defensive behavioral states during an open‐world foraging task (Gründemann et al., 2019). This is not to say that the amygdala is uninvolved in salience processing, but that it may be more sensitive to transient periods of threat compared to extended (Davis et al., 2010; Somerville et al., 2013). Indeed, semi‐naturalistic paradigms often encompass extended periods of affectively salient qualities rather than discrete periods in more static paradigms. It has also been demonstrated that extended periods of arousal can induce habituation, resulting in attenuation of amygdala activation over the course of an experiment (Breiter et al., 1996; Fischer et al., 2000; Wright et al., 2001). Recent work in our laboratory supports this phenomenon of amygdala habituation and the seeming lack of salience differentiation in neural profiles of amygdala activity using a gradual reveal paradigm, where threat stimuli are presented dynamically over an extended period of time (Levitas et al., 2023). Finally, several meta‐analyses have also found notable absences of amygdala activation, albeit, in the context of fear conditioning (Fullana et al., 2016; Visser et al., 2021). This mixture of amygdala activation findings, including ours, illustrates the need to consider this discrepancy and the role of the amygdala in threat (and reward) processing.

As discussed earlier, a key premise of the study was that assessing salience in semi‐naturalistic environments would reveal findings that were undiscoverable with more constrained environments. Although we did not use more constrained experimental conditions in our study, the literature has many examples of such conditions. Thus, one way to assess the similarity of neural mechanisms across more constrained studies and our study was to attempt a meta‐analysis of many previous studies. Based on the prevailing notion that processing of affective salience resides in the amygdala, we had anticipated that the meta‐analyses would largely emphasize the amygdala, with perhaps some contribution from regions of the salience network (Seeley et al., 2007). However, the results of the threat and reward meta‐analyses were more distributed than we anticipated (Figure 6) and the overlap with our findings for both threat and reward systems was also greater than expected. Like our results, the meta‐analyses were dominated by regions of the salience network, primarily the AIC/FIC and dACC (Seeley et al., 2007). This partial correspondence in findings between the salience literature and our experimental paradigm serve as a form of verification that semi‐naturalistic environments can elicit similar brain activation patterns as classical, more‐constrained task paradigms.

Historically, an advantage of tightly constrained paradigms has been the ability to examine specific aspects of salience (and more broadly, perception and cognition) while alleviating the possibility of confounding variables. More specifically, tightly constrained paradigms control the independence of variables, whereas more unconstrained paradigms cede this control for the sake of greater naturalism, thereby increasing the possibility of confounds and having variables become interdependent, which complicates interpretability of results. By targeting a select feature for examination, resulting activation can be attributed to that specific affective, perceptual, or cognitive feature (Friston et al., 1994). That our findings, particularly regarding extended threat (CGPI) and reward (PDI), contain similar activations across the brain with the threat and reward meta‐analyses suggests that semi‐naturalistic task paradigms are similarly able to characterize regions sensitive to salient stimuli/environments, even if they are inherently more difficult to fully interpret.

Just as some degree of overlap was expected between our findings and the meta‐analysis, some degree of divergence was also expected, given that the former were produced using a dynamic environment that engenders examination of context‐dependent neural processing and sensitivity to inherently unfolding temporal dynamics (Fanselow & Lester, 1988; Meyer et al., 2019; Mobbs et al., 2007). The most notable difference occurred in the amygdala, which had robust representation in the meta‐analyses but was completely lacking in our findings. As previously discussed, the role of the amygdala in threat and reward processing has become increasingly nuanced as recent studies employ more ecologically valid techniques and paradigms, resulting in various contexts in which the amygdala is or is not implicated. That our results (semi‐naturalistic) and the meta‐analyses (consisting of more tightly controlled studies) diverge with regard to the amygdala further underscores the need for studying this prominent brain region in ways that better conform to the natural world.

Furthermore, although the gameplay involved alternating ghost modes (behaviors) of hunting (orienting to the participant) versus protecting (orienting to the dot) and may appear to be related to threat and reward processing, these were not variables of interest and only included as nuisance regressors rather than EVs of interest. These two ghost behaviors were conceived and implemented to add variety to the game and better mimic naturalistic scenarios such as when a predator is hunting versus uninterested in prey. Including ghost behaviors in our model did enable us to examine their effects; however, they did not reveal any notable findings (Figure S4). While possibly surprising at first, this is likely due to the conception of the ghost behaviors. When ghosts were red (hunting), they oriented their movement to the participants' avatar. When green (protecting), the ghosts oriented their movement to the dot. In this latter case, the participants were also orienting their movements to the dot to collect their reward. Thus, regardless of ghost behavior (hunting vs. protecting), the ghosts posed a threat that needed to be contented with to accomplish the primary goal, which was to collect the dot.

This constitutes a limitation of our experimental paradigm, as there was no true “safe” period to compare to the threat periods. Given the desire to collect monetary dots while simultaneously avoiding the ghost threats, participants were often in close proximity to environmental threats. Although the co‐occurrence of environmental threats and rewards was intentional, it did not enable participants to seek shelter and wait for opportune moments to seek the dots. Future work will be needed to assess threat and reward processing under semi‐naturalistic scenarios where a safe space or period of time is present, to more fully encompass the range of contexts in a natural environment.

Finally, while the use of continuous variables as predictors in a GLM used to model BOLD signals is not new, it is only common in specific fields where continuous measures are taken, and thus may be unfamiliar. One particularly good example is correlating continuous measures of pupil dilation with BOLD signal (Schneider et al., 2016; Yellin et al., 2015). To do this, the dilation measure is resampled at the BOLD acquisition rate and then convolved with the HRF. This is the same treatment given to the extended predictors in our models. Despite the precedent set by work with pupil dilation, it is possible that the assumption of linearity of the BOLD signal may have been violated in our analyses. We think this is unlikely for the following reasons. As pointed out in previous articles (Monti, 2011; Poline & Brett, 2012) the GLM may not be an ideal model for fMRI analyses and assumptions of the GLM are often violated. The most pertinent question, however, is whether those violations produce false positives. In Monti's (2011) treatment of linearity assumptions, he notes that convolution with the HRF in cases of violation of linearity produces a model that expects larger signals than are actually produced. In general, poor model fits produce weak inferential statistics, which increases false negatives. In the specific case of a model that expects smaller signals than are actually produced, some false positives could result. However, with a model that expects larger signals than are actually produced, only false negatives can result. Given the robustness of our findings, we find it unlikely that our models are producing a preponderance of false negatives, which suggests that the model fits were satisfactory, which further suggests that assumptions of linearity were unlikely to have been violated.

5. CONCLUSIONS

In this semi‐naturalistic study, we examined the mechanisms of threat and reward processing under extended and discrete temporal profiles, all during the simultaneous presence of threat and reward features and events. We uncovered a distributed system of regions sensitive to threat imminence and a less distributed system related to reward imminence, both of which exhibited overlap yet neither of which involved the amygdala. There was broad correspondence of these two systems—with meta‐analyses consisting of thread and reward experiments using more controlled paradigms—with the notable exception of the amygdala. Overall, these results suggest a shared system for salience processing that shows a heightened sensitivity toward environmental threats compared to rewards when both are simultaneously present in an environment. The results also illustrate how semi‐naturalistic studies can substantiate previous findings in the literature while also potentially uncovering novel mechanisms resulting from the nuances and contexts that manifest in such dynamic environments.

AUTHOR CONTRIBUTIONS

Daniel J. Levitas: Conceptualization, methodology, software, formal analysis, investigation, data curation, writing—original draft, writing—review and editing, visualization, and project administration. Thomas W. James: Conceptualization, methodology, formal analysis, writing—original draft, writing—review and editing, and supervision.

CONFLICT OF INTEREST STATEMENT

The authors declare no competing interests.

Supporting information

FIGURE S1. A pairwise correlation of all included EVs in our model. A pronounced negative correlation between the ghost color modes and behaviors (GC_red, GC_green) was observed due to the gameplay, as the ghosts only existed in one specific mode at a time. Otherwise, no two EVs exhibited exceptionally high correlations (highest was 0.36, between CGDI and PDI). B each EV and its corresponding variance inflation factor. The higher a VIF, the greater the multicollinearity induced by the inclusion of the corresponding EV into the model.

FIGURE S2. Statistical map contrasts (z > 4.05, p < 0.05 cluster corrected) of the derivative EVs. A closest‐ghost‐to‐player‐imminence‐derivative (CGPID) represents the change and degree to which the imminence of the closest‐ghost‐to‐player was increasing (more close, red) vs decreasing (less close, blue). B player‐to‐dot‐imminence‐derivative (PDID) represents the change and degree to which the imminence of the player‐to‐dot was increasing or decreasing. Since player speed was constant during gameplay, this simply denotes whether the player was moving toward or away from the dot. Finally, C closest‐ghost‐to‐dot‐imminence‐derivative (CGDID) represents the change and degree to which the imminence of the closest‐ghost‐to‐dot was increasing or decreasing. Dotted circles denote regions observed in our meta‐analyses but not our results. Regions were color‐coded based on their 7‐network designation as described in (Yeo et al., 2011). L = left, R = right. Color bars are denoted in z‐values.

FIGURE S3. Statistical map contrasts (z > 4.05, p < 0.05 cluster corrected) of the remaining two excluded EVs. A closest‐ghost‐to‐dot‐imminence (CGDI) denotes the imminence of the closest ghost to the dot. B Health denotes the health level of participants during gameplay. Dotted circles denote regions observed in our meta‐analyses but not our results. Regions were color‐coded based on their 7‐network designation as described in (Yeo et al., 2011). L = left, R = right. Color bars are denoted in z‐values.

FIGURE S4. Statistical map contrasts (z > 4.05, p < 0.05 cluster corrected) of the ghost behaviors. A red ghost color (GC_red) represents when ghosts were orienting to the participant, with the brighter colors in the statistical map referring to this period and bluer colors when not. B green ghost color (GC_green) represents when the ghosts were orienting to the do, with the brighter colors in the statistical map referring to this period and bluer colors when not. C GC_red—GC_green (red—green) contrast examines the different neural sensitivity between these separate periods. Dotted circles denote regions observed in our meta‐analyses but not our results. Regions were color‐coded based on their 7‐network designation as described in (Yeo et al., 2011). L = left, R = right. Color bars are denoted in z‐values.

Levitas, D. J. , & James, T. W. (2024). Dynamic threat–reward neural processing under semi‐naturalistic ecologically relevant scenarios. Human Brain Mapping, 45(4), e26648. 10.1002/hbm.26648

DATA AVAILABILITY STATEMENT

The source code for the PACMAN game that influenced our paradigm can be found at https://itsourcecode.com/free-projects/python-projects/pacman-in-python-code/. The modified game code (PACPAL) can be found at the following GitHub repository: https://github.com/dlevitas/PACMAN_MRI_study.