Abstract
Growing evidence supports that the coping strategies that individuals utilize are a key predictor of distress following trauma. However, there is limited longitudinal research examining the relationship between psychological distress and coping over time, and even less research examining the possibility of reciprocal relationships between distress and coping, despite the fact that prior theoretical work posits such a relationship. The current study modeled the relationship between distress (PTSD and general distress) and maladaptive coping over time in a sample of 368 college women exposed to the mass shooting at Virginia Tech (VT). Participants completed web surveys regarding their distress, shooting-related coping, and shooting-related PTSD 2 months, 6 months, and 1 year following the shooting. They also completed measures of their psychological distress prior to the shooting as part of an unrelated study. A structural cross-lagged model with latent variables supported a reciprocal relationship between maladaptive coping and general psychological distress over time. In contrast, the cross-lagged model evaluating the relationship between PTSD and maladaptive coping supported that PTSD symptoms predicted coping over time, but there was no reciprocal relationship between coping and PTSD. Implications of the findings for future work examining adjustment following traumatic events are discussed.
Keywords: PTSD, psychological distress, coping, longitudinal methods, mass trauma, school violence
Although many unfortunately will be faced with a traumatic experience during their lives, individuals vary greatly in the extent to which these experiences result in the development and persistence of psychological distress and disrupted functioning (e.g., Bonanno, Galea, Bucciarelli, & Vlahov, 2006; Bonanno et al., 2008). This recognition has fueled efforts to identify what factors promote both poor and good adjustment outcomes following traumatic events. Efforts to identify factors associated with poor adjustment have increasingly supported that post-trauma factors may be highly influential in predicting the development and persistence of distress, even after controlling for demographic factors and aspects of the traumatic experience.
One particular post-trauma factor that appears to be strongly associated with the development and persistence of distress is the coping strategies that individuals use to manage the experience (e.g., Benotsch et al., 2000; Littleton, Horsley, John, & Nelson, 2007). Specifically, engaging in high levels of avoidant and ruminative coping strategies has consistently emerged as a strong predictor of distress following a number of traumas including multiple forms of interpersonal violence, major accidents, serious illness, mass shootings, natural disasters, and combat, with little evidence that heavy reliance on these strategies are ever associated with positive outcomes (Benotsch et al., 2000; Jenkins, 1997; Littleton et al., 2007; Norris et al., 2002; Penley, Tomaka, & Weibe, 2002). Avoidant coping strategies are those focused on avoiding actively managing a traumatic experience or avoiding managing one’s reactions to the experience (Littleton et al., 2007). Examples of such strategies include withdrawing from others, attempting to disengage or suppress thoughts and feelings about the experience, and even denying that the experience occurred, or denying one’s thoughts or feelings about the experience (Snyder & Pulvers, 2001). Ruminative coping involves repetitively and passively thinking about a traumatic experience, such as imagining ways in which the traumatic experience could have been avoided or prevented, or imagining how things would be better if the trauma had not occurred (Branscombe, Wohl, Owen, Allison, & N’gbala, 2003; Nolen-Hoeksema, Wisco, & Lyubomirsky, 2008). These avoidant and ruminative coping strategies stand in stark contrast to more adaptive approach coping strategies, which are actively focused on either managing the traumatic experience or one’s reaction to it (Littleton et al., 2007). Examples of such strategies include seeking support from others, engaging in problem solving or planning regarding the trauma and its sequelae, and attempting to restructure maladaptive thoughts about the trauma (Snyder & Pulvers, 2001).
Snyder and Pulvers (2001) developed a general model to describe how individuals cope with stressful and traumatic experiences, drawing heavily on prior work by other coping theorists including Lazarus and Folkman (1984). Their model posits that, assuming that the individual is not so overwhelmed by a traumatic experience that he or she engages in extreme numbing or avoidance, he or she first makes an initial appraisal of the severity of the experience itself. This initial appraisal of the event’s severity is influenced by a number of factors including the event’s perceived importance for the individual’s life, the extent to which it is perceived as potentially disrupting multiple areas of the individual’s life, the extent to which the event is perceived as a chronic stressor, the extent to which the individual perceives him or herself as having control over the experience, and the perceived severity of the event. Once the individual has made an initial appraisal of the severity of the experience, the individual then makes an appraisal of the coping resources that he or she has to manage the experience and whether the resources available are sufficient to manage the experience. The individual also makes an appraisal regarding whether he or she possesses any characteristics that could interfere with his or her ability to successfully cope with the experience.
Snyder and Pulvers (2001) posit that these primary and secondary appraisal processes then affect the coping strategies that the individual utilizes. Specifically, they propose that individuals’ predominant coping responses can be classified as either approach or avoidant. Individuals primarily utilize avoidance coping strategies when they first appraise an experience as highly stressful and then further appraise themselves as having insufficient resources to successfully handle the experience, and/or appraise themselves as having qualities that will actively interfere with their ability to successfully manage the experience. As a result of these appraisals, the individual is left with a perceived dearth of active options to manage the experience. The individual then engages primarily in reactive coping strategies focused on actively avoiding confronting the stressful or traumatic experience and/or attempting to suppress or deny thoughts or feelings about it. Unfortunately, these attempts to deny and suppress thoughts and emotions about the stressor or trauma are generally not successful in the long-term and instead are followed by a paradoxical increase in thoughts and emotions about the experience, as demonstrated by the thought suppression literature (e.g., Wegner, Schneider, Carter, & White, 1987). As a result, the individual then engages in non-productive and highly distressing rumination about the stressor or trauma. The individual also experiences an increase in negative thoughts and emotions about both the experience and his or her inability to successfully cope with it. The negative thoughts and emotions brought about by the maladaptive coping efforts then feed into further utilization of maladaptive avoidance coping efforts because the person now has to manage both the original stressful event and negative self-related thoughts and emotions. Thus, the individual engages in repeated cycles of avoidance attempts followed by stressor- and self-focused ruminative coping. As a result, he or she experiences persistent distress regarding both the initial experience and his or her inability to successfully manage it.
In contrast, if the individual appraises the event as a significant stressor, but perceives him or herself as having sufficient coping resources to manage it, then he or she engages primarily in active approach coping. Individuals who make these appraisals then utilize a number of active strategies to manage the experience, minimize its impact, and to actively manage their own negative thoughts and emotions as a result. Use of these approach strategies over time serve to ameliorate the negative impact of the experience. Thus, individuals who utilize primarily approach coping strategies are likely to note positive changes over time in both the experience itself and in their ability to successfully manage their thoughts and emotions. The individual therefore experiences an amelioration of distress and an increase in his or her own sense of efficacy to handle the experience and future stressful events.
Extensive research supports various elements of Snyder and Pulvers’ coping model. For example, the work of Dunmore, Clark, and Ehlers supports the importance of primary appraisals in both the development and persistence of PTSD symptomatology (Dunmore, Clark, & Ehlers, 1999, 2001; Ehlers & Clark, 2000). In addition, the work of Benight and colleagues support the importance of appraisals of coping efficacy in predicting both distress post-trauma, as well as the coping strategies (e.g., approach or avoidant) that individuals utilize in managing trauma (Benight & Bandura, 2004; Benight et al., 1999). Finally, extensive research supports a strong relationship between reliance on avoidant trauma coping strategies and psychological distress post-trauma (Littleton et al., 2007).
As previously stated, research has clearly supported the role of maladaptive avoidant and ruminative coping in predicting distress (e.g., Littleton et al., 2007); however, Snyder and Pulver’s (2001) model posits a reciprocal relationship between these two constructs, rather than a unidirectional one (i.e., maladaptive coping leads to distress). Although longitudinal and prospective research in these domains is quite limited, there is some extant research that supports the possibility of a reciprocal relationship between distress and maladaptive coping. Benotsch and colleagues (2000) evaluated the relationship between avoidance coping and PTSD symptoms in a sample of Gulf War veterans assessed twice, approximately 1 year apart. They found that, as expected, avoidance coping at the first assessment predicted time two PTSD symptoms (after controlling for time one PTSD). In addition, time one PTSD symptoms predicted avoidance coping at time two (after controlling for time one coping). Similarly, although not modeling the directionality of influence, Gutner, Rizvi, Monson, and Resick (2006) found in a sample of female victims assessed 1 and 3 months after a physical or sexual assault, that individuals whose PTSD symptoms increased between assessments similarly reported increases in their maladaptive avoidant and ruminative coping. Finally, Benight and colleagues (1999) assessed individuals affected by Hurricane Andrew at two time points (1–4 months and 8–12 months post-storm) and found using SEM that initial distress predicted initial coping efforts, and that both initial distress and coping predicted longer-term distress. However, this study did not model a reciprocal relationship between these variables over time (i.e., initial distress predicting longer-term coping). Overall, however, these studies support that there is a potential reciprocal relationship between distress and coping, although it is also clear that more systematic evaluation of these relationships are needed.
Therefore, the goal of the current study was to examine the relationship between psychological distress and maladaptive coping over time in a sample of college women affected by the 16 April 2007 mass shooting at Virginia Tech (VT). This shooting involved a lone gunman (who later killed himself) and resulted in the death of 32 individuals and the wounding of an additional 25 people (Associated Press, 2007). Participants in the current study were enrolled as students at VT at the time of the shooting and completed surveys 2 months, 6 months, and 1 year after the shooting that assessed their psychological distress and coping. In addition, participants completed a survey prior to the shooting that assessed their psychological distress.
In the current study, the relationship between psychological distress (general distress and PTSD symptoms) and maladaptive coping was assessed through structural cross-lagged regressions. These models involved two latent variables (LVs; e.g., distress and maladaptive coping) where each LV was predicted by the same variable at the previous time point, as well the other LV at the previous time point. Thus, a reciprocal relationship between the two LVs over time was modeled. The fit of the structural cross-lagged model was then compared to models where one variable predicts the other variable over time (e.g., coping predicts psychological distress), but there was no reciprocal relationship between the two variables (e.g., psychological distress predicts coping). The fit of the cross-lagged model was also compared to a model where each LV was only predicted by the same LV at the previous time point. Thus, we could directly compare models of reciprocal influences to unidirectional models, as well as models where the two constructs do not directly influence each other. In addition, pre-shooting psychological distress was included in the general distress model as a predictor of initial maladaptive coping as well as post-shooting distress. The current study represents the first to our knowledge to assess the relationship between distress and maladaptive coping utilizing more than two assessments, the first to include assessment of multiple forms of distress, and the first to examine pre-trauma distress as a predictor of initiation of maladaptive coping following trauma.
Method
Participants
Participants were drawn from a sample of undergraduate VT women who had previously taken part in a multi-university survey (data collected in fall 2006 and spring 2007). They were initially recruited to participate in a study of college women’s negative sexual experiences through announcements on the Psychology department research participant web page and received course credit (n= 843 VT women). Of these, 363 women responded to recruitment emails about a study related to adjustment following the campus shooting and completed at least one of three online surveys (conducted 2 months, 6 months, and 1 year after the shooting). Participants were 19.4 years of age on average (SD = 1.3 years, range = 18–27 years) when they completed the initial pre-shooting survey. Eighty-six percent characterized their ethnicity as White/European American, 5% as Asian American, 3% as Black/African-American, 2% as Latina, 2% as multi-ethnic, and 1% did not indicate their ethnicity or marked other.1
Prior analyses using this data-set supported that there were few meaningful differences between women who completed the two-month post-shooting survey and those who did not (Littleton, Grills-Taquechel, & Axsom, 2009). Specifically, women who completed the two-month post-shooting survey were slightly older, t (831) =3.16, p<.005, d=.23, and reported slightly less social support, t (840) =3.09, p<.005, d=.22, than non-completers of this survey.
Procedure
Participants initially received course credit to take part in an online survey of women’s negative sexual experiences, open to all women 18 years of age and older. As part of this survey, measures (administered in equivalent format to their paper-and-pencil versions) were completed regarding current depressive and anxiety symptoms. Email addresses were collected to award course credit. Of note, the victimization rate found in this study was similar to other recent surveys of college women (e.g., Hammond & Calhoun, 2007; Mohler-Kuo, Dowdall, Koss, & Wechsler, 2004), thus, there was no evidence that victims were more likely to ‘‘select’’ to participate.
Approximately 2 months after the shooting (during summer 2007), all VT women who completed the initial survey were sent an email inviting them to participate in an online survey related to risk and resilience following the shooting. Participants were provided with an electronic consent form that delineated the types of information to be collected, the fact that participants’ responses would be linked with their responses to the prior survey, and compensation information. The post-shooting survey assessed exposure to the shooting, current distress, and shooting-related coping. All standardized measures were administered in an equivalent format to their paper-and-pencil versions. Participants had 4 weeks to complete the survey and were sent up to five electronic reminders. Participants were compensated US$20 with either a gift certificate or a donation to a memorial fund for shooting victims. A total of 298 women completed this survey.
Six months after the shooting (during fall 2007), women were sent an email inviting them to complete an additional online post-shooting survey. This email was sent to women regardless of whether they had completed the two-month follow-up. Participants were again given 4 weeks to complete this survey and sent up to five electronic reminders. Participants were again compensated in the form of a gift certificate or donation. A total of 263 women completed this survey, of which 193 had also completed the two-month post-shooting survey.
One year after the shooting (during spring 2008), women who had completed at least one of the prior post-shooting surveys (n= 368) were sent an email inviting them to complete a final post-shooting survey. Participants were again given 4 weeks to complete this survey and sent up to five electronic reminders. Participants received a US$20 gift certificate for completing this survey and were entered in a drawing to receive one of five US$50 gift certificates. A total of 258 women completed this survey.
All surveys were approved by the university institutional review board and the post-shooting surveys were approved by a university committee developed to ensure ethical conduct in shooting-related research.
Measures administered in each survey
Center for Epidemiologic Studies Depression Scale (CES-D)
The Center for Epidemiologic Studies Depression Scale (CES-D; Radloff, 1977), a 20-item, self-report measure of depressive symptomatology, was administered to assess current depressive symptoms. A sample item is, ‘‘I felt sad.’’ For each item, individuals indicated how often they had felt that way in the past week on a four-point Likert scale bounded by 0 (rarely or none of the time/less than one day) and 3 (most or all of the time/5–7 days). Prior research supports the measure’s internal consistency, test–retest reliability, and validity (Radloff, 1977; Weissman, Sholomskas, Pottenger, Prusoff, & Locke, 1977). Cronbach’s α in the current sample across assessments ranged from .87 to .92.
Four Dimensional Anxiety Scale (FDAS)
The Four Dimensional Anxiety Scale (FDAS; Bystritsky, Linn, & Ware, 1990), a 35-item, self-report measure of the affective, cognitive, behavioral, and physiological components of anxiety, was administered to assess current anxious symptoms. A sample item is, ‘‘Feeling nervous?’’ For each item, individuals indicated how often they had felt in the described manner in the past week on a five-point Likert scale bounded by 1 (not at all) and 5 (extremely). Prior research supports the measure’s internal consistency and validity (Bystritsky et al., 1990; Stoessel, Bystritsky, & Pasnau, 1995). Cronbach’s α of the subscales in the current sample across assessments ranged from .81 to .88.
Post-shooting survey measures
Exposure to the shooting incident
As part of the first post-shooting survey completed (i.e., the two-month or six-month post-shooting survey), participants were asked a number of yes/no questions regarding their direct exposure to several aspects of the shooting (e.g., if they were on campus during the shooting, if they heard gunfire). Participants were placed into one of three exposure groups based on their responses: no direct exposure; moderate direct exposure (on campus, saw police/security); and severe direct exposure (in one of the buildings where the shootings occurred, heard gunfire, saw individuals who had been wounded or killed).
Coping Strategies Inventory (CSI)
The short form of the Coping Strategies Inventory (CSI; Tobin, 2001; Tobin, Holroyd, Reynolds, & Wigal, 1989) was administered to assess strategies used to cope with the shooting. For each survey, participants were asked about the extent to which they had used each strategy in coping with the shooting (since the shooting for the two-month survey, in the past 4 months for the six-month survey, and in the past 6 months for the one-year survey) on a five-point Likert scale anchored by 1 (not at all) and 5 (very much). This measure includes four, four-item maladaptive avoidant and ruminative coping scales (termed disengagement coping scales). These scales are: problem avoidance (I went along as if nothing were happening), social withdrawal (I spent more time alone), wishful thinking (I hoped a miracle would happen), and self-criticism (I blamed myself). Cronbach’s α for these scales in the current study across assessments ranged from .62 to .85.
PTSD Symptom Scale
Symptoms of PTSD associated with the shooting were assessed with the PTSD Symptom Scale, Self-Report (PSS-SR; Foa, Riggs, Dancu, & Rothbaum, 1993). The PSS-SR is a 17-item measure of PTSD symptoms designed to map onto the diagnostic criteria for PTSD. A sample item is, ‘‘Having bad dreams or nightmares about the event.’’ At each assessment, individuals rated how often they had each described symptom in the past week in connection to the shooting on a four-point Likert scale anchored by 0 (not at all or only one time) and 3 (five or more times per week/almost always). Prior research supports the internal consistency and validity of the measure (Foa et al., 1993). Cronbach’s α for the scales in the current study across assessments ranged from .76 to .85.
Analysis plan
Structural cross-lagged regressions were conducted using the Mplus program (version 5.1; Muthén & Muthén, 1998–2007) including the full sample (n= 368). Full maximum likelihood estimation (FIML) was utilized to handle missing data. FIML maximizes the likelihood of the model given the observed data and produces more efficient estimates than other procedures to handle missing data (Wothke, 2000). Three LVs were first created: general psychological distress (scores on the CES-D and subscales of the FDAS), shooting-related PTSD (scores on the subscales of the PSS-SR), and maladaptive coping (scores on the disengagement subscales of the CSI). The adequacy of this measurement model was then evaluated. Next, the extent to which the LVs met criteria for weak factorial invariance across assessments was evaluated; an assumption of the cross-lagged regression models (Ferrer & McArdle, 2003). A model has weak factorial invariance if the loading of each indicator on the LV does not vary across assessments (determined by evaluating the fit of a model where each indicator’s loading on the LV is constrained to be equal across the assessments; Bontempo & Hofer, 2007).
Because the coping measure had not previously been used in mass trauma samples, an exploratory factor analysis (EFA) using an oblique (direct oblimin) rotation was first conducted using the maladaptive coping items. Results of the EFA replicated three of the four maladaptive coping subscales (problem avoidance, social withdrawal, and wishful thinking). However, the items from the fourth coping subscale (self-criticism) did not load on any factor. Thus, the measurement model for maladaptive coping included these three coping subscales as indicators and did not include the self-criticism subscale.
The general distress and PTSD symptom structural cross-lagged models were then evaluated and are summarized in Figure 1 (with the indicators of each LVomitted for ease of interpretation). For these models, the autoregressive components (the effect of the same LV at the previous time point) were set to be constant across assessments (Ferrer & McArdle, 2003). In addition, the residual of each LV was allowed to correlate with the residual of the other LV at the same time point (Ferrer & McArdle, 2003). These residuals were allowed to correlate as it was considered plausible that there existed omitted variables that had causal impacts on both LVs (Hays, Marshall, Wang, & Sherbourne, 1994). In addition, omitting these correlated residuals significantly worsened model fit. It should be noted that the cross-lagged components of the model at each time point were not constrained to be equal, nor were the correlated residuals (Hays et al., 1994). Constraining these components to equality either significantly worsened model fit or led to non-convergence. It should also be noted that shooting exposure variables were not included in the models because prior analyses with this data-set had shown that the two exposure variables – level of direct exposure to the shooting and loss of a friend in the shooting – were unrelated to distress (Littleton et al., 2009). In addition, analyses examining the correlations among the exposure variables supported that these three variables were unrelated to any variables used in the models (all rs <.23, with nearly all rs <.15).
For each model, four fit indices in addition to the model chi square were evaluated. The RMSEA assesses model misfit per degree of freedom, with values of approximately .06 or less indicative of good fit (Hu & Bentler, 1999). The CFI and TLI measure improvement in fit when the model tested is compared to an independence model, with values closer to .95 indicative of good fit (Hu & Bentler, 1999). The SRMR is a measure of the mean absolute correlation residual, that is, the difference between the observed and predicted correlations overall, with values less than .10 considered acceptable (Kline, 2005). In addition to evaluating each fit index individually, it is also important to examine the overall pattern of fit indices (Hu & Bentler, 1999). For example, using a combination of a CFI value of .96 or greater and a SRMR value of .09 or lower minimizes type I and II error rates. Similarly, using a combination of a TLI value of .95 or greater and a SRMR value of .06 or lower minimizes error rates, as does using a combination of a RMSEA value of .06 or lower and a SRMR value of .09 or lower (Hu & Bentler, 1999). The fit of each structural cross-lagged model was compared to two models, each omitting one set of cross-lagged paths (i.e., the lags from distress to coping or the lags from coping to distress). The fit of the structural cross-lagged model was also compared to a model omitting both sets of cross-lagged paths. Because these alternative models represent nested models, their relative fit was compared by means of a chi square difference test. If the chi square difference test was non-significant, this suggests that the full cross-lagged model did not fit the model as well as a more parsimonious model. However, because retaining the nested model is tantamount to rejecting a null hypothesis on the basis of non-significant effects (Keppel, 1991), the full model was rejected in favor of the more parsimonious model only if the Bayesian Information Criteria (BIC) was smaller in the case of the more parsimonious model (Schwarz, 1978).
Results
Exposure to the shooting and distress among participants
Participants varied in their direct exposure to the shooting with 25% reporting no direct exposure to the shooting, 45% reporting moderate direct exposure (e.g., on campus, saw police/security), and 30% reporting severe direct exposure (e.g., heard gunfire, in one of the buildings during the shooting). No participants reported seeing the gunman or being fired upon. Distress was common across assessments. A total of 28% of participants scored above the clinical cut-off of 14 on the PSS-SR at the two-month assessment, 23% scored about the cut-off at the six-month assessment, and 27% scored above the cut-off at the one-year assessment (Coffey, Gudmundsdottir, Beck, Palyo, & Miller, 2006). Similarly, 19% scored above the cut-off of 23 for probable depression on the CES-D at the two-month assessment, 22% at the six-month assessment, and 24% at the one-year assessment (Myers & Weissman, 1980). Although depression scores were somewhat higher than some prior samples, participants’ mean depression scores were similar to a recent college sample (Shean & Baldwin, 2008).
Participant flow and missing data
Of the 368 participants in the current study, 165 (45%) completed all three of the follow-up surveys, 43 (12%) completed only the six month and one-year surveys, 50 (14%) completed only the two-month and one-year surveys, 28 (8%) completed only the two and six-month surveys, 55 (15%) completed only the two-month survey, and 27 (7%) completed only the six-month survey. Bonferroni-adjusted comparisons of the scores of participants who completed one, two, or all three follow-up surveys revealed no significant differences on any study variables. Missing data on study variables were minimal, ranging from 0% missing to less than 1% missing.
Measurement models
Descriptive statistics for variables to be used in the measurement and subsequent cross-lagged models are summarized in Table 1. Correlations among the observed variables can be obtained at http//:www.heatherlittleton.com/asc.pdf. Results of the measurement model for the general distress LV at each time point supported that the models displayed adequate fit to the data (pre-shooting assessment: χ2 (5) = 26.70, p<.001, CFI = .98, TLI = .96, RMSEA = .11, SRMR = .02; two-month assessment: χ2 (5) = 9.20, p = .10, CFI = .99, TLI = .99, RMSEA = .05, SRMR = .02; six-month assessment: χ2 (5) = 17.68, p<.001, CFI = .99, TLI = .97, RMSEA = .10, SRMR = .02; one-year assessment: χ2 (5) = 12.27, p<.05, CFI = .99, TLI = .98, RMSEA = .08, SRMR = .02). In addition, each indicator was strongly associated with the distress LV at each time point with the standardized loadings for each indicator ranging from .66 to .94. The other two measurement models (PTSD symptoms and maladaptive coping) were just identified, that is the number of free parameters equaled the number of known values resulting in zero degrees of freedom and the inability to calculate tests of model fit. However, supporting the fit of the PTSD models, each indicator was strongly associated with the LV, with the standardized loadings ranging from .66 to .89. Similarly, supporting the fit of the maladaptive coping models, each indicator was moderately to strongly associated with the LV, with standardized loadings ranging from .39 to .80.
Table 1.
Variable | Pre-shooting survey (n= 368)
|
Two-month survey (n= 298)
|
Six-month survey (n= 263)
|
One-year survey (n= 258)
|
||||
---|---|---|---|---|---|---|---|---|
M (SD) | Range | M (SD) | Range | M (SD) | Range | M (SD) | Range | |
CES-D | 14.63 (9.89) | 0–47 | 14.05 (9.44) | 0–46 | 14.39 (9.56) | 0–48 | 15.01 (10.13) | 0–48 |
FDAS-physiological | 16.97 (6.52) | 10–44 | 16.84 (6.22) | 10–48 | 16.58 (6.12) | 10–48 | 17.02 (6.16) | 10–39 |
FDAS-cognitive | 13.47 (5.00) | 10–45 | 14.28 (5.48) | 10–37 | 13.80 (5.15) | 10–46 | 14.28 (5.48) | 10–40 |
FDAS-behavioral | 21.53 (6.42) | 10–43 | 21.55 (6.71) | 10–44 | 21.13 (6.45) | 10–43 | 21.61 (7.01) | 10–44 |
FDAS-emotional | 10.57 (4.06) | 5–25 | 10.82 (4.18) | 5–25 | 10.88 (4.21) | 5–23 | 10.84 (4.03) | 5–23 |
PSS-re-experiencing | – | – | 3.38 (2.98) | 0–13 | 2.55 (2.52) | 0–15 | 2.67 (2.75) | 0–15 |
PSS-avoidance | – | – | 3.35 (3.45) | 0–18 | 2.78 (3.21) | 0–15 | 3.45 (4.10) | 0–21 |
PSS-arousal | – | – | 4.30 (3.83) | 0–15 | 3.67 (3.34) | 0–15 | 4.00 (3.76) | 0–15 |
CSI-problem avoidance | – | – | 7.16 (3.18) | 4–20 | 9.37 (3.90) | 4–20 | 10.20 (3.95) | 4–20 |
CSI-social withdrawal | – | – | 7.60 (3.41) | 4–20 | 7.75 (3.26) | 4–20 | 7.95 (3.50) | 4–20 |
CSI-wishful thinking | – | – | 12.37 (3.53) | 4–20 | 11.77 (3.40) | 4–20 | 11.68 (3.72) | 4–20 |
Structural cross-lagged models
The fit of the weak factorial invariance models for the three LVs are summarized in Table 2. The relative fit indices of these models all suggested that they had adequate fit and thus we evaluated the structural cross-lagged regression models. The standardized path coefficients for these models are summarized in Figure 1 and the model fit indices are summarized in Table 3. In addition, the chi square difference test for the three alternative models are also summarized in Table 3: the model omitting the lags from symptoms to coping, the model omitting the lags from coping to symptoms, and the model omitting both sets of cross lags.
Table 2.
Psychological distress | PTSD symptoms | Maladaptive coping | |
---|---|---|---|
χ2 (df) | 332.05 (149) | 55.52 (20) | 31.85 (20) |
CFI | .96 | .98 | .98 |
TLI | .95 | .96 | .97 |
RMSEA | .06 | .07 | .04 |
90% CI RMSEA | .05–.06 | .05–.09 | .01–.07 |
SRMR | .09 | .05 | .06 |
Note: 90% CI RMSEA = 90% confidence interval for RMSEA.
Table 3.
PTSD symptoms | Psychological distress | |
---|---|---|
χ2 (df) | 320.79 (109) | 548.63 (328) |
CFI | .92 | .96 |
TLI | .88 | .95 |
RMSEA | .07 | .04 |
90% CI RMSEA | .06–.08 | .04–.05 |
SRMR | .08 | .07 |
Model omitting symptom lags | 32.30 (2)** | 20.64 (2)** |
Model omitting coping lags | 1.84 (2) | 6.74 (2)* |
Model with no cross lags | 35.30 (4)** | 28.96 (4)** |
Note: 90% CI RMSEA = 90% confidence interval for RMSEA; Model omitting symptom lags = chi square difference test comparing model omitting lags predicting coping from psychological symptoms to the full cross-lagged model; Model omitting coping lags = chi square difference test comparing model omitting lags predicting psychological symptoms from coping to the full cross-lagged model; Model with no cross lags = chi square difference test comparing model with no cross lags to the full cross-lagged model.
p<.05,
p<.005.
The first structural cross-lagged model included PTSD symptoms and maladaptive coping. In this model, the lags from coping to PTSD symptoms were not statistically significant, whereas the lags from PTSD to coping were significant. In addition, the chi square difference test comparing the full cross-lagged model to a model removing the lags from coping to PTSD was non-significant. However, the more parsimonous model had a significantly worse BIC value than the full model: 24,065 versus 24,055, and thus the full model was retained. In addition, the more parsimonious model had similar fit indices as the full model. In contrast, the chi square difference test comparing the full cross-lagged model to the model omitting both sets of lags and the model omitting the lags from PTSD symptoms to maladaptive coping were both statistically significant.
The structural model including psychological distress and maladaptive coping demonstrated adequate fit. All the lags from psychological distress to maladaptive coping were statistically significant. In addition, the lag from six-month post-shooting maladaptive coping to one-year post-shooting distress was significant, although the lag from two-month post-shooting maladaptive coping to six-month post-shooting distress was not significant. The chi square difference tests comparing the three alternative models to the full cross-lagged model were all significant, and thus, the full cross-lagged model was accepted as the best fit to the data.
Discussion
Results partially supported the hypothesis that there would be a reciprocal relationship between maladaptive coping and psychological distress. Consistent with Snyder and Pulvers’ (2001) model and prior research, individuals’ level of maladaptive coping prospectively predicted depressive and anxious symptoms at the one-year assessment. Maladaptive coping also appeared to become more influential in predicting distress over time, with coping predicting distress at one-year post-shooting, but not at 6 months. One possibility is that coping may become a more potent predictor of distress over time as a result of the ineffectiveness of avoidance and withdrawal strategies resulting in rumination, self-recrimination, and demoralization as well as a continued cycle of avoidance and withdrawal, rumination, and negative self-focused thoughts. In addition, psychological distress predicted maladaptive coping, both pre- and post-trauma. Thus, individuals who are experiencing psychological distress prior to a trauma may have appraised the shooting incident as a more stressful event and may have been more likely to appraise the shooting as one that they cannot successfully manage. Distressed individuals also may have been more likely to appraise themselves as possessing characteristics that would prevent them from successfully managing the shooting. As a result of these appraisals, the individuals heavily utilized avoidance coping strategies. Experiencing continued depressive and anxious symptoms following the shooting then made it more difficult for them to stop the cycle of avoidance, withdrawal, and rumination by fueling continued appraisals that they cannot successfully manage their thoughts and feelings about the shooting. Finally, the individuals’ continued inability to successfully manage the trauma and the feelings of demoralization that result then fuel continued depressive and anxious symptoms.
Unexpectedly, whereas PTSD symptoms predicted maladaptive coping, maladaptive coping did not predict PTSD symptomatology. Putting this finding in the context of Snyder and Pulvers’ (2001) model, it appears that experiencing PTSD symptoms following the shooting increased the extent to which individuals appraised the shooting as stressful as well as their appraisal that they lacked sufficient resources to cope with the shooting. Experiencing persistent PTSD symptoms may then have led to continued reliance on maladaptive coping and make it harder for individuals to end the cycle of avoidance, withdrawal, and rumination. For example, individuals may respond to unwanted thoughts/dreams/memories related to the shooting through use of cognitive avoidance strategies (e.g., engaging in wishful thinking). These automatic shooting-related thoughts/dreams/memories could also contribute to continued distressing rumination about the shooting. Similarly, symptoms of hyperarousal or attempts to avoid shooting reminders could also fuel shooting-related rumination as well as continued cognitive avoidance and withdrawal. Although the finding that maladaptive coping did not predict PTSD was unexpected, one possibility is that the sequelae of maladaptive coping (e.g., loss of esteem, demoralization, recrimination) may play a stronger role in leading to general symptoms of depression and anxiety as opposed to directly fueling PTSD symptomatology. For example, rumination or wishful thinking about the trauma (conscious/controlled processes) may not necessarily lead to greater re-experiencing or hyperarousal symptoms (non-conscious/uncontrolled processes), whereas it may lead to greater overall distress or other types of impairment. Instead, these post-traumatic symptoms could be more closely related to pre-trauma variables (e.g., trauma history, pre-trauma distress) as well as more immediate appraisal processes (e.g., Benight & Bandura, 2004; Dunmore et al., 1999, 2001; Ehlers & Clark, 2000).
Limitations of the study should be noted. First, only 43% of individuals who participated in the pre-shooting study completed at least one of the post-shooting surveys. However, this response rate was similar to response rates in prior online studies (Cook, Heath, & Thompson, 2000), and there were few differences between women who participated in the post-shooting research and those who did not. We also did not have sufficient power to simultaneously model the potentially complex relationships among distress, maladaptive coping, and PTSD. In addition, the sample was restricted to female students who completed a prior survey, limiting generalizability to the full population of individuals affected by the shooting. However, the majority of participants were enrolled in Introductory Psychology classes, a course that fulfills a general university requirement and thus draws students from across disciplines. The sample also did not include individuals most severely affected by the shooting (e.g., individuals who saw the shooter fire upon others or who were fired on themselves). It is possible that including of these individuals may have strengthened the relationships between the constructs, such as the relationship between coping and PTSD symptoms. In addition, exposure was not related to any of the variables in the model. This could potentially reflect the fact that all participants were repeatedly exposed to the shooting and shooting reminders. For example, there was extensive media coverage of the shooting and qualitative data collected supported that most participants watched multiple hours of shooting-related TV coverage and spent several hours viewing online material about the shooting in the days following the shooting. Qualitative data also supported that over 90% of participants participated in memorial activities related to the shooting. In addition, it should also be noted that a number of studies following mass trauma and terrorist acts have similarly found that exposure is unrelated to distress following these events (Johnson, North, & Smith, 2002; North, Smith, & Spitznagel, 1994; Silver et al., 2004; Tucker, Pfefferbaum, Nixon, & Dickson, 2000). Finally, only self-report data were collected and were not supplemented with clinical interviews.
Another issue that should be noted is the conceptual distinction between avoidance coping and PTSD, particularly the avoidance symptoms of PTSD. Whereas resolving this issue is beyond the scope of the current paper, results suggest that these do represent distinct constructs. For example, with a few exceptions, the correlations between the maladaptive coping subscales and the subscales of the PTSD measure were in the low to moderate range, suggesting that these represent distinct constructs. In addition, it should be noted that only two of the seven avoidance symptoms of PTSD represent conscious avoidance behaviors, with the others representing less consciously controlled processes (e.g., having a sense of a foreshortened future, inability to recall aspects of the trauma). On the other side, avoidance coping behaviors delineate a number of strategies individuals can use to avoid thoughts, feelings, or reminders of the trauma, in addition to simply avoiding trauma reminders or thinking about the trauma, as described in the avoidance symptom criteria for PTSD. These include strategies such as denying the reality of the trauma (e.g., going along as if nothing had happened) and fantasizing that the trauma never occurred (I wished that the situation had never started).
Bearing these limitations in mind, results have a number of implications for clinicians working with individuals who have experienced traumatic or highly stressful events. For one, the high percentage of individuals experiencing clinically elevated PTSD symptoms supports that even individuals indirectly exposed to a mass trauma are potentially vulnerable to experiencing significant and persistent symptomatology. In addition, the fact that distress and PTSD symptoms predicted maladaptive coping over time suggests that early intervention successfully addressing initial distress and PTSD symptoms may potentially prevent individuals from experiencing long-term negative sequelae. Interventions that successfully address PTSD symptoms may also decrease vulnerability to depressive and general anxious symptoms in part by reducing individuals’ reliance on maladaptive coping. Results also suggest that screenings in the aftermath of a mass trauma could be valuable as many who are initially distressed by a trauma will experience persistent distress.
Results also have a number of implications for future research. First, the relationships among post-trauma factors and psychological symptomatology appear complex. Results strongly support that experiencing psychological distress following trauma contributes to continued maladaptive coping. In addition, results suggest that the persistence of maladaptive coping over time contributes to anxiety and depression following a traumatic experience, and thus may be one mechanism for explaining why individuals with PTSD also frequently experience significant depressive and anxiety symptoms. Specifically, PTSD symptoms may fuel continued maladaptive coping, which then leads to exacerbations of depressive and anxious symptoms. Nonetheless, as this is the first known study to explore these relationships over time, future research examining predictors of post-trauma adjustment should continue to use longitudinal methods to capture the complex relationships among these constructs. Research should also focus on identifying factors that both initiate and maintain poor adjustment patterns following trauma, particularly research examining the role of trauma and coping appraisal processes (e.g., Benight & Bandura, 2004; Dunmore et al., 2001). Similarly, future work should evaluate the unidirectional and bidirectional relationships among distress and post-trauma factors (e.g, coping, appraisals). Similarly, future research should focus on identifying factors that lead to positive and negative post-trauma adjustment patterns, including pre-, peri-, and post-trauma processes. Research in these areas will lead to a fuller understanding of individuals’ adjustment patterns following trauma and how to develop effective prevention and intervention programs for victims.
Acknowledgments
Funding was provided by the National Science Foundation, BCS-0737940. We gratefully acknowledge Fred Anthony Miller, who developed the online data collection program.
Footnotes
Among undergraduates enrolled at Virginia Tech during the 2006–2007 academic year who indicated their ethnicity, 82% self-identified as European American, 7.9% as Asian American, 5% as Black or African-American, and 2.6% as Latina/o (http://www.vt.edu/about/factbook/student_overview.php).
References
- Associated Press. Va Tech wounded may heal slowly. 2007 April 28; Retrieved from http://www.nytimes.com/aponline/AP-Virginia-Tech-The-Wounded.html.
- Benight CC, Bandura A. Social cognitive theory of posttraumatic recovery: The role of perceived self-efficacy. Behaviour Research and Therapy. 2004;42:1129–1148. doi: 10.1016/j.brat.2003.08.008. [DOI] [PubMed] [Google Scholar]
- Benight CC, Ironson G, Klebe K, Carver CS, Wynings C, Burnett K, Schneiderman N. Conservation of resources and coping self-efficacy predicting distress following a natural disaster: A causal model analysis where the environment meets the mind. Anxiety, Stress, and Coping. 1999;12:107–126. doi: 10.1080/10615809908248325. [DOI] [Google Scholar]
- Benotsch EG, Brailey K, Vasterling JJ, Uddo M, Constans JI, Sutker PB. War zone stress, personal and environmental resources, and PTSD symptoms in Gulf War veterans: A longitudinal perspective. Journal of Abnormal Psychology. 2000;109:205–213. doi: 10.1037/0021-843X.109.2.205. [DOI] [PubMed] [Google Scholar]
- Bonanno GA, Galea S, Bucciarelli A, Vlahov D. Psychological resilience after disaster: New York City in the aftermath of the September 11th terrorist attack. Psychological Science. 2006;17:181–186. doi: 10.1111/j.1467-9280.2006.01682.x. [DOI] [PubMed] [Google Scholar]
- Bonanno GA, Ho SMY, Chan JCK, Kwong RSY, Cheung CKY, Wong CPY, Wong CVW. Psychological resilience and dysfunction among hospitalized survivors of the SARS epidemic in Hong Kong: A latent class approach. Health Psychology. 2008;27:659–667. doi: 10.1037/0278-6133.27.5.659. [DOI] [PubMed] [Google Scholar]
- Bontempo DE, Hofer SM. Assessing factorial invariance in cross-sectional and longitudinal studies. In: Ong AD, van Dulmen MHM, editors. Oxford handbook of methods in positive psychology. Oxford: Oxford University Press; 2007. pp. 153–175. [Google Scholar]
- Branscombe NR, Wohl MJA, Owen S, Allison JA, N’gbala A. Counter-factual thinking, blame assignment, and well-being in rape victims. Basic and Applied Social Psychology. 2003;25:265–273. doi: 10.1207/S15324834BASP2504_1. [DOI] [Google Scholar]
- Bystritsky A, Linn LS, Ware JE. Development of a multidimensional scale of anxiety. Journal of Anxiety Disorders. 1990;4:99–115. doi: 10.1016/0887-6185(90)90002-Q. [DOI] [Google Scholar]
- Coffey SF, Gudmundsdottir B, Beck JG, Palyo SA, Miller L. Screening for PTSD in motor vehicle accident survivors using the PSS-SR and IES. Journal of Traumatic Stress. 2006;19:119–128. doi: 10.1002/jts.20106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cook C, Heath F, Thompson RL. A meta-analysis of response rates in Web- or Internet-based surveys. Educational and Psychological Measurement. 2000;60:821–836. doi: 10.1177/00131640021970934. [DOI] [Google Scholar]
- Dunmore E, Clark DM, Ehlers A. Cognitive factors involved in the onset and maintenance of posttraumatic stress disorder (PTSD) after physical or sexual assault. Behaviour Research and Therapy. 1999;37:809–829. doi: 10.1016/S0005-7967(98)00181-8. [DOI] [PubMed] [Google Scholar]
- Dunmore E, Clark DM, Ehlers A. A prospective investigation of the role of cognitive factors in persistent posttraumatic stress disorder (PTSD) after physical or sexual assault. Behaviour Research and Therapy. 2001;39:1063–1084. doi: 10.1016/S0005-7967(00)00088-7. [DOI] [PubMed] [Google Scholar]
- Ehlers A, Clark DM. A cognitive model of posttraumatic stress disorder. Behaviour Research and Therapy. 2000;38:319–345. doi: 10.1016/S0005-7967(99)00123-0. [DOI] [PubMed] [Google Scholar]
- Ferrer E, McArdle J. Alternative structural models for multivariate longitudinal data analysis. Structural Equation Modeling. 2003;10:493–524. doi: 10.1207/S15328007 SEM1004_1. [DOI] [Google Scholar]
- Foa EB, Riggs DS, Dancu CV, Rothbaum BO. Reliability and validity of a brief instrument for assessing post-traumatic stress disorder. Journal of Traumatic Stress. 1993;6:459–473. doi: 10.1002/jts.2490060405. [DOI] [Google Scholar]
- Gutner CA, Rizvi SL, Monson CM, Resick PA. Changes in coping strategies, relationship to the perpetrator, and posttraumatic distress in female crime victims. Journal of Traumatic Stress. 2006;19:813–823. doi: 10.1002/jts.20158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hammond CB, Calhoun KS. Labeling of abuse experiences and rates of victimization. Psychology of Women Quarterly. 2007;31:371–380. doi: 10.1111/j.1471-6402.2007.00386.x. [DOI] [Google Scholar]
- Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55. doi: 10.1080/10705519909540118. [DOI] [Google Scholar]
- Hays RD, Marshall GN, Wang EYI, Sherbourne CD. Four year cross-lagged associations between physical and mental health in the Medical Outcomes Study. Journal of Consulting and Clinical Psychology. 1994;62:441–449. doi: 10.1037/0022-006X.62.3.441. [DOI] [PubMed] [Google Scholar]
- Jenkins SR. Coping, routine activities, and recovery from acute distress among emergency personnel after a mass shooting incident. Current Psychology: Developmental, Learning, Personality, Social. 1997;16:3–19. [Google Scholar]
- Johnson SD, North CS, Smith EM. Psychiatric disorders among victims of a courthouse shooting spree: A three-year follow-up study. Community Mental Health Journal. 2002;38:181–194. doi: 10.1023/A:1015269521969. [DOI] [PubMed] [Google Scholar]
- Keppel G. Design and analysis: A researcher’s handbook. 3. Englewood Cliffs, NJ: Prentice Hall; 1991. [Google Scholar]
- Kline RB. Principles and practice of structural equation modeling. 2. New York, NY: Guilford; 2005. [Google Scholar]
- Lazarus RS, Folkman S. Stress, appraisal, and coping. New York, NY: Springer; 1984. [Google Scholar]
- Littleton HL, Grills-Taquechel AE, Axsom D. Resource loss as a predictor of posttrauma symptoms among college women following the mass shooting at Virginia Tech. Violence and Victims. 2009;24:669–687. doi: 10.1891/0886-6708.24.5.669. [DOI] [PubMed] [Google Scholar]
- Littleton HL, Horsley S, John S, Nelson DV. Trauma coping strategies and psychological distress: A meta-analysis. Journal of Traumatic Stress. 2007;20:977–988. doi: 10.1002/jts.20276. [DOI] [PubMed] [Google Scholar]
- Mohler-Kuo M, Dowdall GW, Koss MP, Wechsler H. Correlates of rape while intoxicated in a national sample of college women. Journal of Studies on Alcohol. 2004;65:37–45. doi: 10.15288/jsa.2004.65.37. [DOI] [PubMed] [Google Scholar]
- Muthén L, Muthén B. Mplus user’s guide [Computer software] Los Angeles, CA: Muthén & Muthén; 1998. [Google Scholar]
- Myers JK, Weissman MM. Use of a self-report symptom scale to detect depression in a community sample. American Journal of Psychiatry. 1980;137:1081–1084. doi: 10.1176/ajp.137.9.1081. [DOI] [PubMed] [Google Scholar]
- Nolen-Hoeksema S, Wisco BE, Lyubomirsky S. Rethinking rumination. Perspectives on Psychological Science. 2008;3:400–424. doi: 10.1111/j.1745-6924.2008.00088.x. [DOI] [PubMed] [Google Scholar]
- Norris FH, Friedman MJ, Watson PJ, Byrne CN, Diaz E, Kaniasty K. 60,000 disaster victims speak: Part I. An empirical review of the literature, 1981–2001. Psychiatry. 2002;65:207–239. doi: 10.1521/psyc.65.3.207.20173. [DOI] [PubMed] [Google Scholar]
- North CS, Smith EM, Spitznagel EL. Posttraumatic stress disorder in survivors of a mass shooting. American Journal of Psychiatry. 1994;151:82–88. doi: 10.1176/ajp.151.1.82. [DOI] [PubMed] [Google Scholar]
- Penley JA, Tomaka J, Weibe JS. The association of coping to physical and psychological health outcomes: A meta-analytic review. Journal of Behavioral Medicine. 2002;25:551–603. doi: 10.1023/A:1020641400589. [DOI] [PubMed] [Google Scholar]
- Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1:385–401. doi: 10.1177/014662167700100306. [DOI] [Google Scholar]
- Schwarz G. Estimating the dimension of a model. Annals of Statistics. 1978;6:461–464. doi: 10.1214/aos/1176344136. [DOI] [Google Scholar]
- Shean G, Baldwin G. Sensitivity and specificity of depression questionnaires in a college-age sample. Journal of Genetic Psychology. 2008;169:281–288. doi: 10.3200/GNTP.169.3.281-292. [DOI] [PubMed] [Google Scholar]
- Silver RC, Poulin M, Holman EA, McIntosh DN, Gil-Rivas V, Pizarro J. Exploring the myths of coping with a national trauma: A longitudinal study of responses to the September 11th terrorist attacks. Journal of Aggression, Maltreatment, & Trauma. 2004;9:129–141. doi: 10.1300/J146v09n01_16. [DOI] [Google Scholar]
- Snyder CR, Pulvers KM. Dr. Seuss, the coping machine, and ‘‘Oh, the Places You’ll Go’’. In: Snyder CR, editor. Coping with stress: Effective people and processes. Oxford: Oxford University Press; 2001. pp. 3–29. [Google Scholar]
- Stoessel PW, Bystritsky A, Pasnau RO. Screening for anxiety disorders in the primary care setting. Family Practice. 1995;12:448–451. doi: 10.1093/fampra/12.4.448. [DOI] [PubMed] [Google Scholar]
- Tobin DL. Unpublished manuscript. 2001. User manual for the coping strategies inventory. [Google Scholar]
- Tobin DL, Holroyd KA, Reynolds RV, Wigal JK. The hierarchical factor structure of the coping strategies inventory. Cognitive Therapy and Research. 1989;13:343–361. doi: 10.1007/BF01173478. [DOI] [Google Scholar]
- Tucker P, Pfefferbaum B, Nixon SJ, Dickson W. Predictors of post-traumatic stress symptoms in Oklahoma city: Exposure, social support, peri-traumatic responses. Journal of Behavioral Health Services & Research. 2000;27:406–416. doi: 10.1007/BF02287822. [DOI] [PubMed] [Google Scholar]
- Virginia Tech 2006–2007 on-campus enrollment profile. n.d Retrieved from http://www.vt.edu/about/factbook/student_overview.php.
- Wegner DM, Schneider DJ, Carter SR, III, White TL. Paradoxical effects of thought suppression. Journal of Personality and Social Psychology. 1987;53:5–13. doi: 10.1037/0022-3514.53.1.5. [DOI] [PubMed] [Google Scholar]
- Weissman MM, Sholomskas D, Pottenger M, Prusoff BA, Locke BZ. Assessing depressive symptoms in five psychiatric populations: A validation study. American Journal of Epidemiology. 1977;106:203–214. doi: 10.1093/oxfordjournals.aje.a112455. [DOI] [PubMed] [Google Scholar]
- Wothke W. Longitudinal and multigroup modeling with missing data. In: Little TD, Schnaubel KU, Baumert J, editors. Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples. Mahwah, NJ: Erlbaum; 2000. pp. 219–240. [Google Scholar]