Assessing the Coherence of Narratives of Traumatic Events with Latent Semantic Analysis

Abstract

Objective:

Memories of traumatic events are thought to be less coherent than memories for non-traumatic events, and expressive writing about traumatic events may help create more organized, coherent memories. Investigations of traumatic memories have been hampered by limitations in conceptualizing and measuring memory organization and coherence. The objective of this investigation is to compare the coherence of written narratives of trauma and neutral memories, and examined changes in coherence of trauma narratives as a function of repeated expressive writing about the trauma using an atheoretical, computational method for assessing the coherence of text.

Method:

Participants (N=246) wrote three times about either their daily activities (neutral condition) or the most traumatic event of their lives. Latent Semantic Analysis (LSA), a method of calculating meaning from text based on semantic association between words, was used to assess narrative coherence as the average semantic association between contiguous sentences.

Results:

Neutral narratives were more coherent than trauma narratives overall, but neutral narratives decreased while trauma narratives increased in coherence from the first to the final session.

Conclusions:

These findings provide evidence that memories for trauma experiences are less coherent than neutral memories, and suggest that writing helps to create a more coherent representation of a traumatic event, highlighting a potential mechanism for expressive writing's effects. Results demonstrate the utility of LSA for examining coherence of memories of traumatic events.

Expressive writing about a personally traumatic event over time can have positive mental and physical health effects (Lepore, Greenberg, Bruno, & Smyth, 2002). Several hypotheses about the effects of expressive writing suggest that it is beneficial because it helps create a more coherent and meaningful memory out of a disorganized and incomprehensible experience (Pennebaker & Seagal, 1999; Smyth, True, & Souto, 2001). In order to study the structure and content of trauma narratives, researchers have relied on self-report of memory disorganization (O’Kearney, Hunt, & Wallace, 2011), rater-coding of trauma narratives for variables such as fragmentation and organization (Foa, Molnar, & Cashman, 1995), or computer-assisted counting of words in different content categories that appear in the narratives (Jaeger, Lindblom, Parker-Guilbert, & Zoellner, 2014).

An important distinction in this literature has been between cohesion, which typically involves surface features like specific word choice and reflects connectedness of temporal and causal relationships, and coherence, which involves deeper conceptual organization throughout the entire document (Foltz, 2007; Crespo & Fernández-Lansac, 2016; O’Kearney & Perrott, 2006). However, despite clinical and theoretical accounts that describe trauma memory in PTSD as lacking coherence, the literature on narrative coherence and related constructs in trauma memories is inconclusive (Crespo & Fernández-Lansac, 2016; O’Kearney & Perrott, 2006). Reviewers attribute this in part to “limitations in how disorganization or fragmentation are conceptualized and operationalized” and recommend as a “priority for future PTSD narrative research…to develop measures of narrative organization that are valid but less transparent than the present ones” (O’Kearney & Perrott, 2006, p. 90).

The current study introduces Latent Semantic Analysis (LSA) as a method to examine the coherence of expressive writing. LSA is a computational method for modeling semantic meaning from text (Landauer, 2007). By deriving the meaning of a word as a function of its relationships to other words occurring across a large body of texts, LSA builds a network of meanings in which the meaning of each word is characterized by loadings on a set of dimensions. Once LSA is “trained” on enough texts to extract meaning from words, then texts of any length—from a single word to long passage—can be compared for their semantic similarity. LSA has been used to summarize and grade student essays (Landauer, 2007), as well as discriminate between people with and without schizophrenia based on the coherence of answers to interview questions (Foltz, 2007). Both of these applications involve LSA’s approach to computing coherence, which is to calculate the semantic similarity (expressed as a correlation) between each successive pair of sentences (e.g., the first and second, the second and third, etc.) in a document (Dennis, 2007). Coherence in this study is defined as the mean of the correlations between each successive sentence, which represents how well each sentence flows into the next to support the overall meaning of a text.

LSA’s approach to coherence is very different than approaches that involve expert-coded ratings of fragmentation, or self-reported ratings of organized thoughts. Coherence is operationalized as the average semantic relationship between each consecutive sentence; thus there is no holistic evaluation of the organization of the narrative, nor any assessment of the quality of thought within individual sentences. As an extreme example of the contrast between approaches, in LSA two consecutive identical sentences are considered perfectly correlated, contributing to high coherence, whereas in a commonly-used narrative coding system, repetition of thought is “the most direct index of fragmentation” (Foa et al., 1995, p. 681). Thus, LSA may provide a related but distinct operationalization of memory organization compared to other methods.

The goals of this study are to 1) introduce LSA as a method for evaluating the coherence of expressive writing about traumatic events and 2) provide preliminary evidence for the validity and utility of LSA as a measure of the coherence of trauma narratives. Because trauma memories are considered to be fragmented and not well organized, descriptors related to coherence, we hypothesize that the coherence of trauma narratives will be lower than the coherence of narratives about neutral topics (Jaeger et al., 2014). Further, given the theory that expressive writing provides physical and mental health benefits by organizing and bringing coherence to memories of traumatic events, we hypothesize that trauma narratives will become more coherent over repeated writing sessions.

Method

Participants and Procedures

The narratives analyzed here were collected by Konig, Eonta, Dyal, & Vrana (2014) from undergraduates at a large, public mid-Atlantic university in the United States. The sample (N=246) had an average age of 21 years and was 72% female with diverse ethnicity (48% Caucasian, 28% African American, 27% Asian, 13% other). They were not pre-screened for either endorsing potentially traumatic experiences or for posttraumatic symptoms. Following a standard protocol for expressive writing studies (Baikie & Wilhelm, 2005), participants wrote for twenty minutes each on three days within a two-week period. In the expressive writing condition (N=113), participants wrote for all three days with the instruction to write about “the most traumatic, upsetting experience of your entire life” (for more information about the specific trauma topics in the expressive writing essays, see Table 4 in Konig et al., 2014), using as much emotional expression as possible. In the neutral topic condition (N=133), participants wrote objectively without emotions or opinions on the first day about the details of their previous day, on the second day about what they had done earlier that same day, and on the last day about their plans for the next week. Before the first day of writing, participants were given forty-five minutes of either response training, stimulus training, or no training, adapted from previous studies of emotional imagery (Miller et al., 1987; Peasley-Miklus, Panayoitou, & Vrana, 2016). This manipulation was included to test hypotheses about response training’s effect on the physiological response to expressive writing (see Konig et al., 2014, for results) but is not relevant to the current hypotheses and will not be discussed further.

Data Preparation and Analysis

Coherence scores were extracted with the Sentence Comparison tool on the LSA website (http://lsa.colorado.edu) using a semantic space based on a corpus of readings up to first-year college level, with the maximum (300) number of factors chosen to represent the data (Landauer, 2007). This tool produced a coherence score for each document by calculating the mean of the correlation between each successive pair of sentences (Dennis, 2007). A 2 (writing topic) x 3 (session) ANOVA was conducted with session as a repeated measures variable and coherence scores as the dependent variable.

Results

Writing about a neutral topic was significantly more coherent (mean=0.373, SE=0.006) than expressive writing about a trauma (mean=0.326, SE=0.006), F (1, 206)=30.93,p<.0001, η_p²=. 131. However, the coherence of expressive writing increased across sessions, whereas the coherence of writing about a neutral topic decreased (see Figure 1), Topic x Session F (1.92, 395.9)=8.30, ε=.961,p<.0001, η_p²=.039. The linear component of this interaction was significant, F (1, 206)=13.92,p<.0001, η_p²=.063. Analyses of the session effect separately for each writing topic found a significant increasing linear trend (p=.002) in coherence for expressive writing and a significant decreasing linear trend (p=.005) for neutral writing.

Figure 1.

Coherence by writing topic and session. Vertical lines represent standard errors.

Discussion

This study found that, as predicted, neutral narratives were more coherent than trauma narratives overall, but that trauma narratives increased in coherence while neutral narratives decreased in coherence over repeated writing sessions. These results are consistent with two widely held beliefs about traumatic memories and expressive writing: that memories of traumatic experiences are not coherently organized, and that expressive writing facilitates organizing the narrative of the traumatic event, allowing the individual to organize and assimilate the event. These beliefs are held despite inconclusive findings regarding both the structure and organization of trauma narratives (Crespo & Fernández-Lansac, 2016; O’Kearney & Perrott, 2006) and the cognitive adaptation explanation for the effects of expressive writing (Sloan & Marx, 2004). The lack of conclusive evidence has been attributed at least in part to measurement issues, leading to a call for the development of more reliable and valid ways of evaluating trauma narratives for coherence and organization (Brewin & Holmes, 2003; O’Kearney & Perrott, 2006; Zoellner et al., 2002). The LSA-derived coherence metric produced results consistent with these hypotheses using an atheoretical, bottom-up approach; data are derived only from the semantic association of contiguous sentences. Because the texts are computer-processed, derivation of this coherence measure is fast and automatic. It is not as time- and resource-intensive as human coding, nor subject to bias as is self-reported memory disorganization.

This computational approach to assessing memory coherence is methodologically and conceptually distinct from previous approaches to assessing narrative coherence, and a potential area of future research is to investigate how, if at all, it relates to other measures of narrative coherence. Two studies with relatively small sample sizes suggest different approaches to coherence are correlated, though effects are in the small to moderate range, and not highly related to health indices (O’Kearney et al., 2011; Rubin, Deffler, Ogle, Dowell, Graesser, & Beckham, 2016). Similarly, a recent principal components analysis suggested that existing measures of coherence contain multiple factors (Adler et al., 2018). Thus, future work should compare LSA’s measure of coherence to these factors in order to fully conceptualize the construct, and examine how coherence relates to other aspects of narratives such as cohesion, fragmentation, content, and organization (Crespo & Fernández-Lansac, 2016).

Several points need to be considered when interpreting LSA coherence data. First, ideal coherence levels for expressive writing may fall somewhere in the middle of the scale: Whereas very low correlations indicate a lack of coherence, very high scores may indicate that no new information is being presented from sentence to sentence (Foltz, 2007). The lack of new information may indicate a lack of emotional and cognitive processing, which would likely reduce the therapeutic benefit of expressive writing (Sloan & Marx, 2004). Further, the order of words within the narrative, which would be evidence of thought organization or disorganization both theoretically and as an element in coding systems, is completely disregarded in LSA (Landauer, 2007). For additional considerations about using and interpreting LSA, and a methodological tutorial for behavioral scientists, see Vrana, Vrana, Penner, Eggly, Slatcher, and Hagiwara (2018).

This study has several limitations. First, the instructions given for each writing session, based on standard expressive writing procedures, were slightly different, and this may have affected the coherence scores. For example, for the final expressive writing session participants were reminded that “this is the last day and so you might want to wrap everything up.” This may have prompted participants to write more coherent narratives, or, as a more expansive instruction, may have encouraged a more divergent approach that would have lowered the coherence. Similarly, there is no obvious explanation for why neutral narratives decreased in coherence over the three sessions. This also may have been due to slightly different instructions for each writing session, or it may be a function of participant disinterest in repeatedly writing a similar, unexciting essay; anecdotal evidence suggested that neutral narrative writers became bored with the study over time. Although the majority of college-aged people have experienced one or more potentially-traumatic events (Vrana & Lauterbach, 1994), another limitation is that the participants were not selected for traumatic experiences or posttraumatic symptoms. To the extent that greater posttraumatic symptom severity is related to less coherent traumatic memories (Crespo & Fernández-Lansac, 2016; O’Kearney & Perrott, 2006), narrative coherence in this sample may be higher than among individuals diagnosed with PTSD; in addition, increases in coherence related to repeated expressive writing of trauma narratives may be underestimated. These hypotheses should be tested in a clinically traumatized sample in future work.

Clinical Impact Statement.

This study shows that expressive writing about a traumatic experience is less coherent than writing about everyday activities. However, repeatedly writing about a trauma results in a more coherent narrative, suggesting that the physical and mental health benefits of expressive writing occur at least in part because writing helps make logical sense out of the traumatic memory. This study measured coherence using a readily-available web-based computer program, suggesting that its measurement is practical for clinical and research purposes.