Effects of Penetrating Traumatic Brain Injury on Event Segmentation and Memory

Abstract

Penetrating traumatic brain injury is associated with deficits in cognitive tasks including comprehension and memory, and also with impairments in tasks of daily living. In naturalistic settings, one important component of cognitive task performance is event segmentation, the ability to parse the ongoing stream of behavior into meaningful units. Event segmentation ability is associated with memory performance and with action control, but is not well assessed by standard neuropsychological assessments or laboratory tasks. Here, we measured event segmentation and memory in a sample of 123 male military veterans aged 59–81 who had suffered a traumatic brain injury as young men, and 34 demographically similar controls. Participants watched movies of everyday activities and segmented them to identify fine-grained or coarse-grained events, and then completed tests of recognition memory for pictures from the movies and of memory for the temporal order of actions in the movies. Lesion location and volume were assessed with computed tomography imaging. Patients with traumatic brain injury were impaired on event segmentation. Those with larger lesions had larger impairments for fine segmentation and also impairments for both memory measures. Further, the degree of memory impairment was statistically mediated by the degree of event segmentation impairment. There was some evidence that lesions to the ventromedial prefrontal cortex (vmPFC) selectively impaired coarse segmentation; however, lesions outside of a priori regions of interest also were associated with impaired segmentation. One possibility is that the effect of vmPFC damage reflects the role of prefrontal event knowledge representations in ongoing comprehension. These results suggest that assessment of naturalistic event comprehension can be a valuable component of cognitive assessment in cases of traumatic brain injury, and that interventions aimed at event segmentation could be clinically helpful.

1 Effects of Penetrating Traumatic Brain Injury on Event Segmentation and Memory

In everyday life, human cognitive systems must confront the fact that the stream of experience is continuous, dynamic and complex. In the face of this dynamic complexity, perception and comprehension parse the ongoing experience stream into meaningful events, a capability known as event segmentation (Radvansky & Zacks, 2014; Zacks, Speer, Swallow, Braver, & Reynolds, 2007). For example, when watching someone prepare a turkey sandwich, an observer would likely segment the activity into events such as gathering the ingredients, arranging turkey and condiments on bread, and tidying up. Activity can be segmented at a range of time-scales, and fine-grained events tend to correspond to subdivisions of coarse-grained events; for example, tidying up might break down into putting away the ingredients, washing up dishes, and wiping objects dry. Event segmentation ability is not well characterized by standard neuropsychological or laboratory tasks. This is true in part because such tasks impose a rigid, discrete task structure that renders event segmentation trivial, and thereby masks individual differences in event segmentation ability. However, event segmentation is important for the online control of action (Bailey, Kurby, Giovannetti, & Zacks, 2013; Cooper & Shallice, 2006; Schwartz, 1995) and for subsequent memory (Boltz, 1992; Radvansky, Tamplin, & Krawietz, 2010; Sargent et al., 2013; Schwan, Garsoffky, & Hesse, 2000). Deficits in sequencing actions and in episodic memory are frequently associated with focal brain injury (Cohen & Eichenbaum, 1995; Fogassi et al., 2005; Sirigu, Zalla, Pillon, Grafman, Dubois, et al., 1995). Therefore, in the present study we investigated the chronic effects of penetrating traumatic brain injury on the segmentation and memory of everyday events in a sample of older male military veterans.

1.1 Perception and Memory of Events

Research on the behavioral and neurophysiological correlates of event segmentation reveal it to be a capability at the center of everyday comprehension. Healthy adult observers are able to segment movies of everyday activity into meaningful events with minimal training (Newtson, 1976). Their segmentation judgments show strong test-retest reliability and interobserver agreement (Speer, Swallow, & Zacks, 2003). Event segmentation emerges early in development (Baldwin, Baird, Saylor, & Clark, 2001; Hespos, Grossman, & Saylor, 2010; Hespos, Saylor, & Grossman, 2009; Sharon & Wynn, 1998) and shows modest declines in healthy aging (Zacks, Speer, Vettel, & Jacoby, 2006; but see Sargent et al., 2013). Coarsegrained events and fine-grained events tend to cluster hierarchically such that the boundaries of coarse-grained events coincide with nearby fine-grained event boundaries (Zacks, Tversky, & Iyer, 2001) and fall slightly later than the end of a group of fine events, enclosing them (Hard, Tversky, & Lang, 2006). Implicit measures of event segmentation indicate that it is an ongoing concomitant of normal perceptual processing (Hard, Recchia, & Tversky, 2011; Zacks, Braver, et al., 2001), requiring neither intention nor attention.

Event segmentation may be an important component of encoding everyday activity for subsequent memory. Evidence suggests that effective event segmentation allows for the formation of effective memory structures. Event boundaries are especially memorable, and cueing event boundaries can improve memory and learning (Boltz, 1992; Newtson, 1976; Zacks & Tversky, 2003). Instructions to segment activity at a fine grain rather than a coarser grain can improve some kinds of memory (Hanson & Hirst, 1989, 1991; Lassiter & Slaw, 1991). One large lifespan individual differences study assessed whether individual differences in event segmentation could account for differences in memory for everyday activity (Sargent et al., 2013). Participants completed tasks assessing segmentation and memory for movies of everyday activity, and also completed a psychometric battery assessing processing speed, working memory capacity, semantic knowledge, and episodic memory for words and pictures.

Segmentation was found to be a significant predictor of event memory, accounting for substantial variability in memory above and beyond individual differences in the psychometric measures. This result suggests that, for naturalistic everyday activities, impairments in event segmentation could account for everyday memory deficits that are a significant component of the clinical presentation of TBI.

1.2 Information-processing Mechanisms of Event Segmentation

What are the functions and mechanisms of event segmentation? One account is given by Event Segmentation Theory (EST; Zacks et al., 2007). According to EST, segmentation arises because perceptual systems make predictions about how the immediate environment will evolve, and these predictions take advantage of sequential dependencies in everyday environments by constructing a working memory representation of the current event (an event model) and updating it at boundaries between events. In EST, the system constantly monitors the accuracy of its predictions and updates its event model when prediction error increases transiently. When prediction error increases, the current event model representation is destabilized and a new model is formed by integrating the current sensory information with information from memory. One particularly important kind of memory is knowledge about categories of events that are related to the current situation. Event knowledge is represented in structured event complexes (Grafman, 1995). The SEC construct is closely related to the constructs of the script and event schema. An SEC captures information about how a particular category of event typically unfolds, including information about the objects, settings, actors, and sequences of actions. Event models are constituted in part by activating relevant knowledge representations in long term memory. In this regard they are similar to the working memory representations proposed by accounts such as long-term working memory theory (Ericsson & Kintsch, 1995), the episodic buffer (Baddeley, 2000), and Cowan’s (1999) short term store. For event models, one important form of long-term knowledge that is activated to constitute an event boundary is the SEC. Thus, EST proposes that the function of event segmentation is to improve perceptual predictions by constructing event models. In information-processing terms, EST proposes that event segmentation arises from the interaction of perceptual mechanisms with mechanisms of working memory, cognitive control, semantic memory, and episodic memory. The monitoring of prediction error is a form of cognitive control, in that one cognitive system modulates processing in another in response to task demands. SECs are a form of semantic memory, and their activation contributes to event model content. Finally, representations of related previous events are part of episodic memory, and these too contribute to event model content.

1.3 Neural Mechanisms of Event Segmentation

Event segmentation can be characterized in terms of neural mechanisms as well as information-processing mechanisms. Of particular interest is the prefrontal cortex (PFC), which may include components that play at least two distinct roles in event model maintenance and updating. First, recurrent activity in regions of the PFC, particularly dorsolateral ones, has been shown to underlie various forms of working memory maintenance in humans and in nonhuman primates (D’Esposito & Postle, 2002). One reasonable possibility is that maintaining event models depends on the dorsolateral prefrontal cortex (dlPFC), possibly in coordination with other structures that maintain long-term representations that are activated when an event model is instantiated. SECs have been associated with parts of the PFC providing a second potential role for the PFC in event segmentation (Koechlin, Corrado, Pietrini, & Grafman, 2000; Krueger, Barbey, & Grafman, 2009). The involvement of the PFC in maintaining SECs has received direct support from studies of patients with PFC damage (Crozier et al., 1999; Grafman, Sirigu, Spector, & Hendler, 1993; Humphreys & Forde, 1998; Sirigu, Zalla, Pillon, Grafman, Agid, et al., 1995), and from neuroimaging studies (Crozier et al., 1999; Krueger et al., 2009; Krueger, Moll, Zahn, Heinecke, & Grafman, 2007; Partiot, Grafman, Sadato, Flitman, & Wild, 1996). The neuroimaging studies provide specific support for the role of the ventromedial PFC (vmPFC) in representing the social aspects of event knowledge. Relatedly, medial PFC is more associated with predictable and routinely reinforced event sequences, whereas lateral PFC is more associated with variable sequences (Koechlin et al., 2000; Krueger et al., 2007). These findings suggests that patients with lesions affecting the vmPFC might be particularly impaired in segmenting activities that involve multiple interacting participants. A related possibility is that patients with vmPFC lesions might have particular difficulty with segmentation at a coarse grain, because coarse segmentation has been more strongly associated with changes in goals and causes, whereas fine segmentation has been more strongly associated with changes in physical features such as motion and objects. (Dickman, 1963; Zacks, 2004; Zacks, Speer, & Reynolds, 2009).

Updating an event model entails up-regulating processing in the pathways from sensory inputs to the substrates of event models. Functional MRI studies show evidence of phasic increases in activity throughout a broad collection of regions in the posterior parts of the cortex, particularly at the juncture of the parietal, temporal and occipital lobes on both the medial and lateral surfaces of the brain (Speer, Reynolds, & Zacks, 2007; Whitney et al., 2009; Zacks, Braver, et al., 2001). These brain regions are modulated by many aspects of perceptual processing and task performance, and a reasonable possibility is that their activation at event boundaries reflects phasic up-regulation during event model updating. The observation of these phasic responses was one motivation for the regions of interest investigated in the current study, as described below.

In order to assess the effects of penetrating TBI on event segmentation, we tested the ability of people with TBI and controls to segment everyday activities into meaningful events and to subsequently remember those activities. We hypothesized that TBI would be associated with impairments of event segmentation and memory. Further, we hypothesized that memory impairments would be partially statistically mediated by impairments in segmentation. Finally, we tested whether brain regions that have been associated with event representations and event segmentation would be specifically associated with impaired event segmentation or memory.

2 Method

2.1 Participants

Participants were drawn from the Vietnam Head Injury Study (VHIS) registry (Raymont et al., 2008). The VHIS registry is a longitudinal study that includes a large sample of American male veterans who suffered penetrating traumatic brain injury (pTBI), mostly from combat, while serving in the Vietnam War and non-injured control veterans, who experienced combat but did not suffer brain damage. The VHIS registry consists of four phases described in detail elsewhere (Raymont, Salazar, Krueger, & Grafman, 2011). Phase I was the recruitment period for the registry, Phase II occurred between 1981 and 1984 at the Walter Reed Army Medical Center and involved administration of a neuropsychological battery, Phase III occurred approximately 20 years later between 2003 and 2006 and consisted of both neuropsychological testing and computed tomography (CT) acquisition at the National Navy Medical Center in Bethesda, MD, and Phase IV (2008–2012) was completed as a follow-up assessment, consisting of a week-long testing battery at the National Institute of Neurological Disorders and Stroke, Bethesda, MD.

Patients with focal penetrating traumatic brain injury (pTBI; n = 123) and noninjured controls (NC; n = 34) were male combat veterans who served during the Vietnam War (Glass, Krueger, Solomon, Raymont, & Grafman, 2013; Raymont et al., 2011). To ensure that veterans were eligible to participate in the VHIS Phase IV testing, a phone interview prior to arrival and a neurological exam at the test site were conducted to screen all participants for psychological and neurological exclusion symptoms. All participants gave their written informed consent and the Institutional Review Board (IRB) at NINDS approved all study procedures. Participants completed a battery of normed psychological tests, including the Beck Depression Inventory (Beck, Steer, & Brown, 1996), the Boston Naming Test (Kaplan, Goodglass, & Weintraub, 1983), the Delis-Kaplan Executive Function System (D-KEFS; Delis, Kaplan, & Kramer, 2001), the Visual Object and Space Perception Battery (Warrington & James, 1991), and the Wechsler Memory Scale III-abbreviated (Wechsler, 1997). From the D-KEFS, we examined scores on the category sorting task, letter and category fluency, and the trailmaking test. For all the D-KEFS measures, scaled scores were used. For the trailmaking test we examined the difference between the scaled number-letter sequencing and number sequencing conditions; higher scores indicate stronger executive control over sequence switching. Finally, participants completed the Armed Forces Qualifying Test (AFQT). The AFQT has been shown to correlate strongly with measures of overall IQ (Raymont et al., 2008). Importantly, premorbid AFQT scores were also available from participants’ service records. Performance of the pTBI and NC groups on relevant measures are given in Table 1.

Table 1.

Demographic Statistics

	pTBI				NC
	Mean	Range	SD	Missing	Mean	Range	SD	Missing
Age	63.35	59, 81	2.91	0	63.28	59, 79	3.85	2
Years of Education	14.64	10, 20	2.15	0	15.03	12, 18	2.15	2
Pre-injury general intelligence (AFQT; percentile)	65.48	1, 99	23.19	12	72.87	40, 95	16.67	11
Post-injury general intelligence (AFQT; percentile)	55.53	1, 98	25.47	0	71.12	19.79	15–94	2
Depression (BDI, total score)	7.49	0, 35	7.54	0	10.97	0, 29	8.67	2
Word naming (BNT, total score)	53.58	14, 60	7.23	0	55.69	46, 60	3.84	2
Sorting (D-KEFS, scaled score)	10.89	1, 17	3.19	0	12.78	6, 17	2.81	2
Letter Fluency (D-KEFS, scaled score)	8.76	1, 19	3.66	0	10.41	4, 19	3.58	2
Category Fluency (D-KEFS, scaled score)	9.72	1, 18	3.24	0	11.78	2, 19	3.86	2
Trailmaking (D-KEFS, difference of scaled)	−0.43	−10, 8	2.7	1	−1.16	−8, 3	2.32	2
Neurobehavioral Rating Scale	36.93	27, 117	12.64	0	36.03	27, 65	8.85	2
Visual Screening (VOSP, total score)	19.44	14, 20	1.02	0	19.53	12, 20	1.5	2
Long-term memory (WMS, scaled score)	38.5	9, 62	11.7	0	44.48	13, 60	10.89	1

pTBI: penetrating traumatic brain injury; NC: noninjured control; AFQT: Armed forces qualifying test; BDI: Beck Depression Inventory; BNT: Boston Naming test; D-KEFS: Delis-Kaplan Executive Function System; VOSP: Visual Object and Space Perception; WMS: Wechsler Memory Scale-abbreviated.

2.2 Stimuli

We created movies of four everyday activities: making breakfast, doing the laundry, preparing for a party, and finding a book at the library. The breakfast and laundry activities were selected to be relatively high in familiarity, whereas the party and library activities were selected to be low in familiarity, based on previously collected norms (Rosen, Caplan, Sheesley, Rodriguez, & Grafman, 2003). For each activity, we filmed two versions: the social version had two actors who cooperated to complete the task, and the nonsocial version had only one actor who completed the task alone. This manipulation was motivated by the finding that the medial PFC is associated with representing the social aspects of events (Koechlin et al., 2000; Krueger et al., 2007). Representative frames from the movies are shown in Figure 1. The movie durations ranged from 235 s to 376 s.

Figure 1.

Example frames from movies of four everyday activities performed with one actor (nonsocial; left column) or two actors (social; right column).

2.3 Experimental Design

The flow of data collection procedures is shown in Table 2. In a first session, each participant performed a segmentation task and two memory tasks with each activity. Participants were asked to push a button to identify either the largest units that were meaningful to them (coarse grain) or the smallest (fine grain). The segmentation task was followed by a test of recognition memory for frames from the movie just segmented, followed by a test of memory for temporal order of the activity depicted. Each activity was represented either by the social or nonsocial movie. In a second session separated by one to four days, the participant segmented the same movies at whatever grain had not been tested during the first session. Assignment of activities to the social or nonsocial condition, order of activity presentation, and order of segmentation grain were counterbalanced across participants.

Table 2.

Overview of Data Collection

Prior to study: Premorbid Armed Forces Qualifying Test (AFQT) Phase III (2003–2006): Computer Tomography (CT) imaging Phase IV (2008–2012): Psychometric battery Beck Depression Inventory, Boston Naming Test, the Delis-Kaplan Executive function System, Visual Object and Space Perception Battery, and the Wechsler Memory Scale III-abbreviated. Session 1 For each of 4 movies: Segmentation at coarse or fine grain (counterbalanced) Recognition memory Order Memory Session 2 For each of 4 movies: Segmentation at fine or coarse grain

2.4 Event Processing Tasks

Each participant completed three event processing tasks involving movies of everyday events: segmentation of the movies, recognition memory for still pictures, and a test of memory for temporal order. First, they segmented each movie into meaningful units (Newtson, 1973). Participants were instructed to mark off the activity in the movie into the largest units [coarse segmentation] or smallest units [fine segmentation] that seemed natural and meaningful to them, by pressing the space bar on a computer keyboard. They were instructed that there was no right or wrong answer; the experimenter wanted to know how they perceived the movies. At the beginning of the session, each participant was given the opportunity to practice the task with a brief (145 s) movie of a man assembling a model using Legos. The experimenter monitored to ensure that the participant pressed the button at least 3 times for coarse segmentation or at least 6 times for fine segmentation; if not, he was reminded of the instructions and given the opportunity to repeat the practice. (This was done for 42 pTBI and 15 NC.)

Immediately after the first segmentation of each movie, participants completed a two-alternative forced-choice recognition memory test for still frames taken from the movie (Zacks et al., 2006). For each movie, 20 still frames were selected and paired randomly with 20 frames taken from a similar movie that we had filmed using the same actor or actors in the same location. Participants were instructed to choose the picture from the movie they had just seen, and to do so as quickly as possible while remaining accurate. The picture pairs were presented in random order.

After the recognition tests, participants were given an order memory test. For this test, they were given 12 cards printed with pictures of distinctive points in the movie they had just seen. The cards were shuffled and presented in random order, and each participant was asked to sort them into the order in which they had occurred in the movie. The experimenter recorded the order given and the time taken to complete the task. Order error was scored as the mean absolute deviation of each card’s ordinal position from the correct position, and thus ranged from 0 (perfect ordering) to 6 (cards arranged backwards).

2.5 Event Segmentation Measures

From the segmentation data we calculated three variables of interest: unit size, segmentation agreement and hierarchical alignment. Unit size is simply mean duration of the events identified for each viewing. Segmentation agreement measures the degree to which a given observer’s segmentation agrees with the normative segmentation of the group as a whole. Agreement was calculated as described by Kurby and Zacks (2011). Briefly, each movie is divided into 1-s intervals and each participant’s segmentation is coded as to whether the participant segmented during each interval. The individual segmentation functions are cumulated by taking the mean across participants (without regard for group), and then each individual’s segmentation function is correlated with the group norm. Finally, the correlation is scaled to a 0–1 variable such that 0 is the worst possible score given the number of event boundaries identified by the participant, and 1 is the best possible score. Hierarchical alignment assesses the degree to which fine-grained units are chunked together into larger structures. To the extent that finegrained events are grouped hierarchically, each coarse unit boundary should fall close to one of the fine unit boundaries. Hierarchical alignment was calculated as described by Zacks, Tversky and Iyer (2001). Briefly, for each coarse event boundary we identified the participant’s closest fine boundary and measured the absolute distance between them. Those distances were compared to the mean distance expected under a null model in which the coarse and fine segmentation time series are independent; the amount that the observed distance is lower than the null model expectation is the measure of hierarchical alignment.

2.6 Computed Tomography Imaging

Computed tomography (CT) scans acquired as part of Phase III of the VHIS were used to analyze patients' brain lesion locations (see Raymont et al., 2008 for details of acquisition and processing). Structural images were reconstructed with an in-plane voxel size of 0.4 × 0.4 mm and an overlapping slice thickness of 2.5 mm at a 1.0 mm. Lesions were traced using the Analysis of Brain Lesion (ABLe) software (Solomon, Raymont, Braun, Butman, & Grafman, 2007), and projected to Montreal Neurological Institute atlas space. The atlas-registered lesion images were used to calculate the percentage of each Brodmann area (BA) affected, and to visualize the lesions for coding using the Caret software package (Van Essen, 2005).

For seven patients, CT data were unavailable. For the remaining 114, one of the authors (C.L.) classified the region or regions affected by each subject’s lesion. The regions were selected based on our a priori hypotheses regarding cortical systems important for event segmentation and memory, and also based on the observed distribution of brain lesions (see Figure 2). The regions of interest included two in the PFC, ventromedial prefrontal cortex (vmPFC) and dorsolateral prefrontal cortex (dlPFC), and one in the posterior cortex: right superior/posterior (RSP). The division of PFC into dlPFC and vmPFC reflects the possibility that vmPFC could be selectively important for social or coarse-grained segmentation. The selection of RSP reflected previous finding of strong fMRI responses in this region at event boundaries (Speer et al., 2007; Zacks, Braver, et al., 2001); the left hemisphere was excluded because few patients had lesions affecting this region. Lesions were classified according to the following criteria:

Figure 2.

Distribution of lesion overlap projected onto an MNI-registered anatomical image and thresholded to show only voxels where more than 4 patients’ CT images showed damage.

• vmPFC: 40% or more of any of BA 10, 11, 12, 32 affected or 25% or more of any two of those BAs. Patients with smaller lesions completely enclosed within vmPFC also were included.

• dlPFC: 40% or more of any of BA 8, 9, 10, 45, 46 affected or 25% or more of any two of those BAs. Patients with smaller lesions completely enclosed within dlPFC also were included.

• RSP: 40% or more of right BA 5, 22, 37, 39, 40 affected, or 25% or more of any two of those BAs, or more than 20% of BA 7 or 19.

Of the 114 patients with CT data, 36 had lesions in the regions of interest. Of these, 16 had lesions affecting multiple regions of interest. The distribution of lesions is shown in Table 3.

Table 3.

Distribution of Lesion Locations in Penetrating Traumatic Brain Injury Patients

Region of Interest	Count
vmPFC only	2
dlPFC only	8
RSP only	10
vmPFC + dlPFC	13
dlPFC + RSP	2
vmPFC + dlPFC + RSP	1
Total vmPFC	16
Total dlPFC	24
Total RSP	13
Other	80
CT not available	7

vmPFC: ventromedial prefrontal cortex; dlPFC: dorsolateral prefrontal cortex; RSP: right superior posterior cortex; CT: computed tomography.

From the traced lesions we also calculated the total percentage volume loss for each participant. This global measure of lesion severity was used as a covariate in the analyses.

During the data analysis phase we also considered a voxel-symptom lesion mapping (VLSM) approach. However, despite the reasonable sample size obtained here, the distribution of lesions did not provide sufficient coverage for an adequately powered analysis in our regions of greatest interest. Therefore, we restricted the analyses to region-based statistics to conserve statistical power.

3 Results

All analyses were conducted using an alpha level of .05. Data were modeled with linear mixed models, using the lme4 (http://cran.r-project.org/web/packages/lme4) and lmerTest (http://cran.r-project.org/web/packages/lmerTest) packages in R. Statistical tests were performed using the Satterthwaite approximation for the degrees of freedom. Outliers were trimmed by removing observations ± 3 SD from the mean (event unit size: 7 observations, 0.6%; segmentation agreement: 4 observations, 0.3%; recognition memory: 4 observations, 0.3%).

Five patients in the pTBI group were unable to complete the event segmentation and memory tasks. Of these, four had lesions affecting vmPFC and one had a lesion affecting dlPFC.

3.1 Global Effects of TBI on Event Segmentation and Memory

To assess the overall effects of TBI on event cognition and memory, we fit linear mixed models that included fixed effects of group (pTBI or NC) and socialness of the activity. We also included percentage brain volume lost as a covariate, and random effects of activity and participant. For event unit size and segmentation agreement, observations were made at both the coarse and fine grains and so this variable was also included as a repeated measure; for the memory variables, grain was treated as a between-participants variable because memory was tested after the first viewing. Brain volume loss was allowed to interact with segmentation grain but not with socialness.

Segmentation agreement

As can be seen in Figure 3, pTBI had poorer segmentation agreement than NC, F(1, 148.98) = 31.4, p < .001. There was also an interaction such that the group difference was larger for coarse than for fine segmentation, F(1, 1026.9) = 24.1, p < .001 (panel A). Finally, there was an interaction between grain and lesion size such that larger lesions were associated with poorer segmentation, but only for fine segmentation, F(1, 1024.8) = 4.3, p = .04 (panel B). Agreement was higher for fine than for coarse segmentation, F(1, 1026.9) = 223.1, p < .001; however, this main effect is not generally interpretable because the larger number of fine segmentation observations produces more reliable measures and thereby higher segmentation agreement. In sum, segmentation agreement was impaired in people with TBI, and this was especially true for coarse segmentation, and for fine segmentation in patients with large lesions.

Figure 3.

A. Event segmentation agreement was poorer for patients with TBI than controls, and this difference was larger for coarse than for fine segmentation. B. However, larger lesion volume was associated with greater impairment for fine but not coarse segmentation. (For all figures, boxplots show the first, second, and third quartiles as the box, the range trimmed to 1.58 times the inter-quartile range as the whiskers, and the outliers as dots. Scatterplots show linear fits with 95% confidence intervals.)

Hierarchical alignment

The model fits revealed no significant fixed effects, largest F = 1.01.

Event unit size

Our training procedure was designed to reduce individual differences in segmentation grain but was not guaranteed to eliminate them, and previous studies have found differences in unit size in clinical samples (Bailey, Zacks, et al., 2013; Zacks et al., 2006; Zalla, Labruyère, & Georgieff, 2013; Zalla, Pradat-Diehl, Monmart, & Sirigu, 2000; Zalla, Verlut, Franck, Puzenat, & Sirigu, 2004). Because the distribution of unit sizes was highly skewed (skewness = 5.3), the unit sizes were log-transformed prior to outlier removal and analysis. Both pTBI and NC were able to modulate the grain of their segmentation in response to the experimenter’s instructions, resulting in coarse units that were longer (control: mean 35.2 s, SD 42.3 s; pTBI: mean 39.4 s, SD 43.7 s) than fine units (control: mean 13.3 s, SD 12.4 s; pTBI: mean 13.7 s, SD 11.9 s), resulting in a main effect of grain, F(1, 1016.77) = 811.2, p < .001. There was a significant interaction of lesion volume with grain such that patients with larger lesions had shorter coarse units but not fine units, F(1, 1015.21) = 13.4, p < .001. However, this was accompanied by a significant interaction between grain and group such that, after correcting for lesion volume, pTBI had slightly longer coarse units, F(1, 1016.78) = 6.19, p = .01. The presence of these opposing effects suggests caution in interpreting this pattern. In any case, it is clear that both pTBI and NC were able to follow the instructions to modulate their segmentation grain. No other effects were significant, largest F = 2.11.

Recognition memory

Overall, pTBI recognized fewer pictures than NC; however, this was mostly due to pTBI with large lesions. Thus, there was a large effect of lesion volume, F(1, 143.7) = 12.50, p < .001, and no significant effect of group once lesion volume was accounted for, F(1, 143.7) = 0.22, p = .64. Movies with two actors were remembered slightly less well, leading to a significant effect of socialness, F(1, 478.26) = 8.52, p = .003. These effects are illustrated in Figure 4. No other effects were significant, largest F = 1.72.

Figure 4.

A. Overall the pTBI group recognized fewer pictures than the control group, and pictures from movies with two actors (social) were recognized less well than movies with one actor (nonsocial). B. Patients with larger lesions due to TBI had poorer recognition memory for movies of everyday activities, which accounted for the group difference seen in A.

Order memory

Overall, pTBI made more order errors than NC; however, as with recognition memory this was mostly due to pTBI with large lesions. As a result, the only significant effect was a large effect of lesion size such that patients with larger lesions made more order errors, F(1, 147.0) = 32.5, p < .001; the main effect of group was not significant, F(1, 147.0) = 0.26, p = .61 (see Figure 5). The effect of socialness was marginally significant, F(1, 495.1) = 2.95, p = .09; all other Fs <= .52.

Figure 5.

A. Overall the pTBI group made more order memory errors than the control group. B. Patients with larger lesions due to TBI made more errors, which accounted for the group difference seen in A..

3.2 Might Segmentation Mediate Effects of Lesion Volume on Memory?

In most process models of memory, memory performance depends causally on operations at encoding. Given that we observed strong effects of lesion volume on one encoding measure (fine segmentation agreement) and two memory performance measures (recognition and order memory), we asked whether the effects of lesion volume on memory might be mediated by segmentation agreement. To do so, we used the mediation package in R (Tingley, Yamamoto, Hirose, Keele, & Imai, 2014) to assess whether accounting for the relationship between lesion volume and fine segmentation statistically reduced the relationship between lesion volume and memory. As can be seen in Figure 6, segmentation agreement was correlated with both memory measures, more strongly for fine than coarse segmentation. For both memory measures, the analysis indicated a significant degree of mediation. For recognition memory, fine segmentation agreement mediated 18.9% of the lesion volume effect (quasi-Bayesian 95% confidence interval: 0.05, 0.45); for order memory, fine segmentation agreement mediated 18.4% of the lesion volume effect (quasi-Bayesian 95% confidence interval: 0.07, 0.34).

Figure 6.

Correlations between mean participant scores on fine and coarse segmentation agreement, recognition memory, and order memory.

3.3 Regional Specificity

Prior to assessing affects of region-specific injury on event segmentation and memory, we tested whether any of the demographic or psychometric measures were associated with region-specific lesions. Surprisingly, RSP lesions were associated with higher levels of education, t(112) = 4.047, p < .001, and long-term memory, t(112) = 2.24, p = .03, and marginally lower levels of depressive symptoms, t(112) = −1.91, p = .06. One might speculate as to why the association between RSP lesions and education occurred, but given this association the effect of RSP lesions on long-term memory is not surprising. Controlling for education eliminated the effect of RSP lesions on long-term memory, t(112) = 0.98, p = .33. To control for the potential confound between education and RSP lesions, we included education in all subsequent models (along with its interaction with segmentation grain, for the segmentation agreement and unit length models). Education was z-scored prior to model fitting. None of the other psychometric variables had a significant association with lesion location.

To evaluate the regional specificity of lesion effects on the dependent measures, we fit linear mixed models similar to those for the group analysis, but instead of the group variable we included fixed effects of lesions in vmPFC, dlPFC, RSP, and lesions outside of our areas of interest. Note that this approach allowed us to model the fact that a number of patients had lesions affecting multiple regions of interest (16 of 35 with lesions in our regions of interest). Again, brain volume loss was allowed to interact with segmentation grain but not with socialness (and not with the lesion indicator variables).

Segmentation agreement

The model indicated that segmentation agreement was significantly poorer in patients with lesions in vmPFC [F(1, 146.2) = 5.62, p = .02], dlPFC [F(1, 147.3) = 7.18, p = .008], RSP [F(1, 144.01) = 4.41, p = .04], or outside the areas of interest [F(1, 144.49) = 36.4, p < .001]. The effects of vmPFC lesions and lesions outside the interest areas interacted with segmentation grain such that impairments were greater for coarse than for fine segmentation [vmPFC x grain F(1, 1001.5) = 8.1, p = .003; other x grain F(1, 987.2) = 20.7, p < .001]. This pattern is depicted in Figure 7. Consistent with the previous analysis comparing pTBI to NC, segmentation agreement was higher for fine than for coarse segmentation, F(1, 998.7) = 39.3, p < .001. Neither the main effect of socialness nor its interactions with region were statistically significant, largest F = 2.1.

Figure 7.

Segmentation agreement was significantly impaired in patients with lesions in vmPFC, dlPFC, RSP, and outside the a priori regions of interest, and the impairments were larger for coarse than for fine segmentation in vmPFC and in regions outside the regions of interest. Points are estimates from mixed linear models for hypothetical participants with lesions in only one region of interest; error bars are 95% confidence intervals.

Hierarchical alignment

As with the model comparing pTBI to NC, the model fit revealed no significant fixed effects, largest F = 2.15.

Event unit size

The model provided little evidence of regional specificity of effects on event unit size. Lesions of vmPFC and outside the regions of interest were marginally associated with longer unit sizes; F(1, 146.9) = 3.62, p = .06 and F(1, 145.5) = 2.97, p = .09, respectively. As expected, the effect of grain was highly significant, F(1, 973.7) = 291.6, p < .001. No other main effects or interactions of interest approached significance, largest F = 2.13.

Recognition memory

The region-specific model clarified the results of the model comparing pTBI to NC, indicating that lesion volume was the primary predictor of effects of TBI on recognition memory, F(1, 132.8) = 9.63, p = .002. The effect of socialness was again significant, F(1, 454.2) = 6.74, p = .010. No other effects were significant, largest F = 1.32.

Order memory

For order memory as well as for recognition memory, the region-specific model did not provide evidence that the effects of TBI were region-specific. The only significant effect was a large effect of lesion size such that patients with larger lesions made more order errors, F(1, 143.3) = 22.7, p < .001. The interaction between dlPFC lesions and socialness approached significance, F(1, 431.8) = 2.73, p = .10. No other effects of interest were significant, largest F = 1.82.

4 Discussion

4.1 Traumatic Brain Injury Impairs Event Segmentation and Memory

The key results are summarized in Table 4. The most important finding was that penetrating TBI was associated with substantial impairment in patients’ ability to segment activity into normative events. This was especially true for those with larger lesions and for coarse-grained segmentation. This finding adds to the clinical picture of cognitive deficit in pTBI. Impaired event segmentation could be a clinically significant feature, because impairments in segmenting the ongoing stream of behavior appropriately could lead to difficulty organizing one’s actions in sequential tasks and in remaining appropriately oriented to an ongoing task or situation and in adapting to execution errors in planning.

Table 4.

Summary of Key Results

Segmentation agreement Impaired in pTBI More impaired with larger lesions Effects on coarse segmentation somewhat regionally selective

Recognition memory Impaired in pTBI with larger lesions

Order memory Impaired in pTBI with larger lesions

Impairment in memory for everyday activity also was associated lesion volume. This finding is not surprising given the long history associating brain injury with memory impairment (Schacter & Tulving, 1994). However, quantitative assessments of memory for everyday activity in brain injury patients are rare. More importantly, the mediation analyses provided preliminary evidence that impairments in the encoding operation of event segmentation account for some of the memory deficits observed as a result of pTBI. This result reinforces the importance of ongoing comprehension during encoding for subsequent memory in brain injury. It converges with psychometric studies of healthy adults (Sargent et al., 2013) to suggest that event memory depends importantly on functions that are not simply episodic memory as captured by typical laboratory tests (Rubin & Umanath, 2015). This has implications for the assessment and treatment of such memory deficits. When evaluating a patient’s concerns regarding memory for everyday activity, it may prove valuable to assess encoding operations including event segmentation. If appropriate representations are not being formed during encoding, effective memory is unlikely. Further, it may prove valuable to design memory interventions aimed at initial segmentation and encoding in order to improve patients’ ability to retrieve and use event information later.

4.2 Limited Evidence for Functional Specificity

Based on the finding that the vmPFC is selectively involved in representing the social attributes of event representations, we hypothesized that the segmentation of activities staged with two actors would be selectively impaired by lesions to this region. This was not supported by the data. We also hypothesized that vmPFC lesions would affect coarse segmentation more than fine segmentation, because coarse segmentation has been associated with goal and cause relationships that are also associated with social event representations. We did find evidence for this effect. However, this finding should be interpreted with caution because the same pattern was also observed for patients with lesions outside our a priori regions of interest. Given that the parsing of activity into hierarchically structured relations between events and sub-events is important for comprehension, it would be valuable to further assess the role of vmPFC in coarsegrained segmentation.

Surprisingly, coarse segmentation was not sensitive to the size of patients’ lesions; patients with smaller lesions were impaired equally to those with larger lesions. In contrast, for fine segmentation the degree of impairment increased with lesion size. One possibility is that this lesion size effect reflects the degree of compromise to some cognitive resource that is widely distributed across the brain. One speculative possibility is that making fine-grained predictions about activity depends heavily on activation of knowledge representations as part of event models, and these knowledge representations are widely distributed in the cortex. The domain of knowledge impairment may be relatively location-dependent (Patterson, Nestor, & Rogers, 2007), but its contribution to event model quality may be similarly independent of domain. Another possibility is that with greater lesion size comes a likelihood that more components of the event segmentation mechanism will be impaired, resulting in more severe deficits. That is, rather than reflecting a graded resource, the lesion size effect may reflect the summation of probabilistic effects on multiple individual mechanisms. A final possibility that cannot be ruled out is that fine segmentation is more reliable than coarse segmentation, and therefore has a better chance of showing statistically reliable differences.

Both pTBI and NC were able to modulate the grain of their segmentation in response to the experimenter’s instructions. Despite the use of a training procedure designed to reduce differences across individuals in segmentation grain, there was some indication that patients with TBI identified larger coarse units than controls. However, this effect was accompanied by an effect of lesion size such that patients with larger lesions formed coarse events that were closer to the length of controls’ coarse events. This pattern is perplexing and merits further investigation.

4.3 Potential Mechanisms

The large effect of vmPFC lesions on coarse segmentation is consistent with the view that the medial PFC is selectively involved in the representation of SECs. SEC knowledge is more likely to represent features of events on the scale corresponding to coarse events in this dataset (several tens of seconds in general) than the scale corresponding to fine events (10 to 15 seconds). For example, in the “making breakfast” activity, an SEC might well represent a step such as “getting a frying pan,” but may not represent the individual actions of “opening the cupboard,” “taking down the pan,” and “closing the cupboard.” (Presumably, those actions are captured by other representations such as motor schemata for targeting and grasping objects.)

The substantial effects of lesion size on fine segmentation agreement and on memory emphasize that segmentation agreement constitutes a final common pathway for a processing cascade that involves sensory and perceptual processing, prediction, error monitoring, and memory updating. An important question is whether segmentation is a cause of effective comprehension of an activity, a consequence of comprehending the activity, or a shared consequence of a common cause. Recent data from healthy adults suggests that segmentation is at least in part a cause, because intervening to improve segmentation can improve memory (Flores, Bailey, Eisenberg, & Zacks, 2014; Gold & Zacks, 2014). In future research it would be valuable to probe these processing steps individually in patients with focal TBI to assess them directly.

4.4 Limitations

In interpreting the present results it is important to note the basic limitations of the lesion-based method. Lesions are not randomly distributed. As a result, for many of the areas of the cortex there were few or no patients with lesions affecting the area. Some of these include regions of a priori relevance for event segmentation and memory; for example, lesions of the right superior posterior cortex were well represented, but not the left. Further, lesions in a particular area may be accompanied by damage to fibers of passage that is not detectable in the CT imaging. Also, lesions to a particular region may be associated with premorbid individual differences. (We partially controlled for this by assessing effects of the lesion variables on education and premorbid AFQT score, and controlling for education statistically.) In view of these issues, one must interpret null effects conservatively and be conscious of potential confounding effects.

The lesion methods used here are powerful for studying effects on cortical structures (and their associated white matter tracts), but provide little information about subcortical structures. EST proposes a central role for midbrain and striatal structures in event segmentation (Zacks, Kurby, Eisenberg, & Haroutunian, 2011; Zacks et al., 2007). However, these structures are too small and central to be studied with the clinical lesion methods used here.

Finally, we note that, at the point of testing, our sample was mostly in the seventh and eighth decades of life. Thus, these results may not generalize to event perception and memory in younger adults. However, it is worth noting that previous studies of age effects on event segmentation and its relationship to memory show decreases in overall performance but stable relations amongst measures (Kurby & Zacks, 2011; Sargent et al., 2013; Zacks et al., 2006).

4.5 Conclusion

In this study, patients with pTBI showed substantial impairments in comprehension and memory for movies of everyday activity. Such stimuli make for an attractive middle ground between highly simplified psychometric tasks with rigid trial structures on the one hand, and highly subjective self-report or informant-report measures of tasks of daily living on the other. Characterizing how the cognitive components of naturalistic event comprehension and memory are impaired by brain injury holds the promise both to inform models of healthy cognition and to suggest interventions to diagnose and remediate clinically significant difficulties negotiating the everyday world.

Abbreviations

EST

event segmentation theory

PFC

prefrontal cortex

dlPFC

dorsolateral prefrontal cortex

vmPFC

ventromedial prefrontal cortex

VHIS

Vietnam head injury study

pTBI

penetrating traumatic brain injury

CT

computed tomography