Resilience has been and will always be, but rates declared are inevitably suspect: Reply to Galatzer-Levy and Bonanno

Our central argument is that it is wrong to make any definitive declarations about rates of resilience, not whether these rates are high, low, or in between. We showed that prevalence rates of resilience using growth mixture models (GMM) are inevitably determined by a priori assumptions applied and our findings are cause to reconsider declared rates. We welcome the commentary by Galatzer-Levy and Bonanno (in press) as a way to engage in an open discussion on approaches to studying resilience, and respond to each of their criticisms in turn.

The first criticism raised centered on concerns that we did not perform an exact re-analysis because we included more data points before and after the stressor than they had. Borrowing from principles of measurement, we would argue that just as the reliability of a scale generally increases with more items, additional time points should have led us to more “reliable” findings. And if indeed the addition of more data points engendered substantively different findings, then this would directly support our central contention that trajectories derived from GMM need to be reconsidered. Parenthetically, we note that in follow-up analyses, we re-analyzed our data to replicate prior analyses more exactly and found similar results; see Footnotes 1 and 2 in Infurna & Luthar (in press).¹

Galatzer-Levy and Bonanno then contended that our analytic choices were not optimal with regard to technicalities of model fit, slopes, and entropy, and we continue to maintain that they were. Assumptions we applied each had strong conceptual bases that were explained in non-technical terms and with illustrative figures. A strong refutation of our central argument could perhaps have come from demonstration that (and how) our analyses contained statistical errors, and this has not been done.² Absent demonstrable errors, the disagreement boils down to which choices are best in setting up the GMM. Even assuming that both sets of assumptions (theirs and ours) might be judged as equally well-justified, this, then, would yet again directly support our central argument, that our findings are cause to reconsider rates of resilience as determined by GMM.

We consider, next, specific issues raised regarding model technicalities of the slope variances, model fit, and entropy. First, Galatzer-Levy and Bonanno mention that in our re-analysis we freed all slope variances within and between classes, but we emphasize that we additionally freely estimated the intercept variances within and between classes. This distinction has important implications for the number of trajectories found and proportions of people in each, but is not acknowledged. Second, a critical preliminary step before utilizing GMM is to first find the best single-group representation of change (Ram & Grimm, 2009). The analyses we conducted across Parts A, B, and C illuminate which longitudinal model fit the data best by comparing the 1-class models on key indicators of fit (e.g., BIC, CFI, and RMSEA; see Hu & Bentler, 1999). Taking spousal loss as an example, the model fit for Part C is markedly better based on the lower BIC (39,054 versus 39,517 and 39,292) and a higher CFI (0.923 versus 0.849 and 0.863) and lower RMSEA (0.068 versus 0.091 and 0.088) (see note in Tables 1, 5, and 6 for CFI and RMSEA). These values show that constraining variance parameters severely impacts the overall model fit, just as they support decisions to relax a priori assumptions. Third, Galatzer-Levy and Bonanno heavily emphasize entropy, but there is consensus that multiple fit statistics (in addition to entropy) should be considered when determining the proper number of classes (see Muthén, 2004; Nylund, Asparouhov, & Muthén, 2007; Ram & Grimm, 2009). For example, the likelihood ratio tests (LRT) are particularly informative in determining the correct number of classes, whereas entropy indicates the probability of individuals falling into the classes identified, and as we and others have shown (see Muthén, 2004), can be overestimated based on model specifications. Finally, they note that our trajectories never deviated in response to the stressor by more than a single scale point. In fact, the slope and intercept variances in our analyses indicated significant variability both between- and within-persons. Moreover, the trajectories shown in the figures are model-implied estimates that average across participants within particular groups; if each individual were to be graphed, a great deal of heterogeneity would inevitably be seen.

Galatzer-Levy and Bonanno's third criticism is centered on the advancement of theory. The note that the absence of a sustained grief trajectory in our GMM findings renders our work theoretically uninformative and implausible; here we underscore that our results say nothing about whether we believe such a group exists (and we most certainly do). Our study included one sample and a single item measure of life satisfaction (as in past studies) and although debatable, had feelings of sadness, loss, or loneliness been considered instead or as well, a “grieving” group would most likely have been found, as well as more distinct trajectories. Second, they discuss the decades of research that has consistently produced evidence of resilience being modal, with additional commonly observed groups being those of recovery and chronic distress. However, we note that the decades of research that they refer to have mostly used GMM that have relied on the same methodological assumptions that they support (for recent applications, see Galatzer-Levy & Bonanno, 2014; Maccallum et al., 2015), and not the assumptions we applied that have stronger conceptual grounding. We reiterate Lazelere and colleagues’ (2015) caution here that if researchers apply exactly the same statistical assumptions across studies, the presence of the same biases will yield essentially the same sets of findings (i.e., exact replications).

Respectfully, we contend that our analyses on GMM are actually central to the refinement of theory. For the maturation of any scientific discipline, it is essential is the requirement that central tenets be empirically examined across a variety of research circumstances, in order to identify what Lakatos (1978) has termed the ‘hard core’ of theories, that is, a set of central principles that are impervious to challenge (Luthar & Brown, 2007). What we contend is that statements asserting that “resilience is common” do not belong in the hard core of resilience theory. Besides varying significantly by data analytic techniques as we described in detail, we also discussed briefly how labels of resilience can differ greatly based on measurements used to define resilience; it is practically impossible to make definitive “diagnoses of resilience” because of the range of plausible adjustment difficulties that must be ruled out.

Conclusion

Whatever our respective perspectives, researchers must reconsider suggestions that resilience is common. When declarations in science bear on matters of resources and social policy, standards of evidence must be higher than usual with claims standing up across populations studied and methods employed. We hope that the issues we have raised will be viewed as an opportunity to move resilience research forward, moving away from “exact replications”, and instead, considering alternative approaches to analyses and measurement, each scientifically justifiable. For both empirical and conceptual reasons as we have outlined, we believe that it is unwise for scientists to make any definitive statements about rates of resilience in the face of major life stressors.