Moving Psychopathology Forward

Vaidyanathan et al. (this issue) offer provocative proposals for the striking lack of progress, after decades of basic and clinical research, in understanding the etiology of psychopathology. They engage a number of prominent issues. They do not (and need not) take a stand regarding the relative merits of DSM 5’s categorical premise and its almost exclusively psychological characterization of the manifestations of psychopathology vs. the merits of the Research Domain Criteria (RDoC) initiative of the National Institute of Mental Health, with its embrace of dimensional conceptualizations of contributors to psychopathology and of biological phenomena as playing a critical role in understanding psychopathology. Six years after its announcement in 2009, RDoC is gaining traction but is subject to widespread misunderstanding in several critical ways. First, although it champions dimensional conceptualizations of psychopathology, it does not preclude traditional categorical conceptualizations. Second, the widely distributed RDoC matrix does not represent the final set of concepts RDoC seeks. The current matrix is merely a “draft” (http://www.nimh.nih.gov/research-priorities/rdoc/nimh-research-domain-criteria-rdoc.shtml#toc_matrix), an illustrative example, and is designed to be fully extensible as the literature merits. Third, RDoC is not bad news for psychological concepts or for established methods of studying psychopathology. On the contrary, RDoC offers an explicit, systematic antidote to the naïve reductionism fostered by the Decades of the Brain that began in 1990 (Miller, 1996, 2010; Miller & Rockstroh, 2013). This is clear from examination of the rows of the proposed matrix, nearly all of which are psychological phenomena. Such dominance of psychology in the RDoC matrix reflects the definitional fact that psychopathology is a set of psychological phenomena, not a set of “brain diseases” as became fashionable to claim in the first Decade of the Brain (e.g., Hyman, 1998; Leshner, 1997). Fourth, in the nomological network, among its columns RDoC positions psychological measurements at the same logical level as biological measurements. Notably, in RDoC, psychological events are not assumed to be reducible to biological events, nor do biological events “underlie” psychological events. Psychological and biological symptoms are not merely at different “levels of analysis”. They are qualitatively different phenomena. Psychological and biological events are coincident (reflecting “contingent identity”), the psychological phenomena are not reducible to or logically the same as the biological phenomena (“necessary identity”, Fodor, 1968; Kozak & Miller, 1982; Miller & Kozak, 1993). RDoC is often characterized as a radical departure from the longstanding dominance of categorical assumptions about psychopathology, and it is, though it merely makes space for dimensional models alongside categorical models. More fundamentally radical is its resuscitation of psychology as an equal partner in the clinical science of psychopathology.

The critique of the psychopathology literature that Vaidyanathan et al. (this issue) offer stands regardless of how one might view controversies about DSM V vs. RDoC, about categorical vs. dimensional concepts, or about the roles and relationships between psychological vs. biological phenomena in psychopathology. Vaidyanathan and colleagues press not just for clearly articulated theories but for strong tests of them. In particular, they emphasize the concept of “risky tests”. The key feature of such tests is not low probability of confirmation. Rather, these are tests that put one’s theory(ies) at risk.

In calling for risky tests, Vaidyanathan et al. (this issue) assert the benefits of incorporating data from multiple domains, very much in line with the provisions of RDoC (Cuthbert & Insel, 2013). Vaidyanathan and colleagues illustrate this point by examining research on the startle blink reflex and the error-related negativity component of the event-related brain potential and their joint contribution in clarifying the neurobiology of depression. Not only is their argument persuasive, it builds on such fundamental concepts as convergent and discriminant validity and the multitrait-multimethod approach proposed by Campbell and Fiske (1959). Furthermore, integration of multiple measures within and across RDoC units of analysis will likely lead to refinements of the constructs suggested in RDoC along with the emergence of novel constructs as research progresses. Fostering such a dynamic approach to research is implicit in several of the key points made by Vaidyanathan and colleagues, including consideration of developmental trajectories that lead to psychopathology. Although RDoC provides a useful, well-organized, and generative framework, an important element that may be easily overlooked is its promotion of relationships within and between domains that cross row and column boundaries in the exemplar RDoC matrix. For example, in considering the etiology of substance abuse disorders, Vaidyanathan and colleagues demonstrate the utility of integrating research support from RDoC’s positive valence system, cognitive system, and systems for social processes, providing rich opportunities for designing risky tests.

Much research on genetic contributions to psychopathology has, in retrospect, been unwisely based on the hope that genetic explanations would be simple, straightforward, and comprehensive (Miller, Clayson, & Yee, 2014). It now appears (e.g., Kendler, 2005) that there are no “genes for” psychopathology, in the simple Mendelian sense that, for example, there are genes for Type A and Type B blood. Genes play a substantial role in psychopathology, of course, but individual effects are likely to be very small in the population. Deleterious genes of large effect – i.e., not very sensitive to mediation or modulation by other genes and by environment – would tend to be selected against over time. The lack of large, simple genetic effects means that the role of genes in psychopathology involves either (a) spontaneous mutations of somewhat consistent form that (rarely) arise to drive psychopathology or (b) genetic sources of variance that individually or in combination contribute little variance to risk or manifestations of psychopathology at the population level but which sometimes converge to make a substantial difference in individual cases. As discussed by Golden and Meehl (1979; see also Lykken, McGue, Tellegen, & Bouchard, 1992; Lykken, Tellegen, & Iacono, 1982), a set of rare causal factors or indicators that are uncorrelated in the general population may have a huge impact in the rare cases in which they converge. Thus, in judging the magnitude of the impact of genetic contributions to psychopathology, we need to distinguish between the effect size in the population and the effect size in an individual with a particular set of genes. We need to be able to detect rare variants, not the common variants that small-N studies are powered to find. We need studies with very large Ns not because the effect size (in an individual) is small but because the effect is rare in the population, and for that reason the effect size across the population may be trivially small. Ns will need to be especially large when the clinical phenotype is loosely defined, as is arguably the case for depression or schizophrenia.

Vaidyanathan et al. (this issue) make the point that very large Ns may be needed to detect such rare confluences in a population. For example, about the large, high-profile literature on serotonin and related genes, they speculate: “Indeed, it may turn out that serotonin-related genes do have large effects, but we were unable to measure those effects until now because the relevant variants are very rare and heretofore unmeasured.” Although this will be very demanding research to carry out, they note that the technology to do so is advancing. If some of the endophenotypes that lie causally between genetic factors and clinical manifestations turn out to be considerably more straightforward than the genes that contribute to them (Gottesman & Gould, 2003; Gottesman & Shields, 1972; Insel & Cuthbert, 2009; Lenzenweger, 2010; Miller & Rockstroh, 2013; Miller et al., 2014), the clinical utility of what we learn about genetic factors may yet prove quite high.

As Vaidyanathan and colleagues note, study of endophenotypes has not yet succeeded in identifying genes involved in psychopathology. However, the utility of endophenotypes does not rest primarily on their ability to help identify genes. Their promise is help in identifying causal mechanisms, which obviously are not limited to genes. There are grounds for hope that discovering pieces of the genetic stories in psychopathology, and other parts of the causal chains, will foster psychological and biological treatments that target biochemical and environmental causal factors proximal to psychopathology.

Just as RDoC is opening thinking to dimensional phenomena (without banning categorical concepts), the premise in recent decades that a primary goal is to identify directly biochemical interventions needs reconsideration (without eschewing pharmacological treatment). The clinical benefits of conventional psychological interventions (traditional psychotherapy, cognitive training, etc.) are well established, often with impact matching or exceeding that of medication. Less appreciated is the wide-ranging evidence of psychological interventions altering biochemistry (for selective review, see Miller, 2010). Just as there are no logical grounds to confine our targeting of dysfunction that we conceptualize psychologically to directly psychological treatments, there are no logical grounds to confine our targeting of dysfunction that we conceive biologically to directly biological treatments (Miller, 1996, 2010).

Vaidyanathan et al. (this issue) touch on a crucial question that has received far too little attention, that of causality. Their emphasis is on how rare it is that psychopathology research is designed to address etiological mechanisms – not just correlation but causation. What causal mechanisms that include both biological and psychological phenomena will look like is entirely unclear. Notwithstanding the point made above about the ample evidence that psychology and biology are mutually causal, to date there is not a single demonstration of how that happens. That is, one can provide psychotherapy and show that it alters biology, and one can give medication and observe a reduction in (psychological) symptoms, but there is not a single case of a fully-worked-out mechanism for how such an effect occurs. As noted above and argued elsewhere (Lilienfeld, 2007; Miller, 1996, 2010; Miller & Keller, 2000), the reductionist assumption pervasive in recent decades is not viable. New thinking is needed to understand the relationship between biological and psychological phenomena. Merely asserting that psychology is what the brain does, or arguing that the mind is an emergent property of the brain, is not a description of a mechanism.

Although Vaidyanathan et al. (this issue) present a forceful critique of the psychopathology literature, they also provide two impressive examples of the integration they advocate, bringing to bear multiple types of measures to test theory. They calibrate just how risky the tests have been of those theories. Regretably, few other instances of very risky (decisive) tests stand out in the research literature.

To amplify their call for more theory and more critical testing of theory: we should strive for quantitative models that allow point predictions (Meehl, 1978; Miller, 2004). The archetypal test in the psychopathology literature, the t-test, primarily evaluates a hypothesis that is rarely of theoretical interest: whether two samples were drawn from the same population. For that purpose, whether the group mean difference is likely nonzero has no special value. If the samples reliably differ in standard deviation, median, 37%ile score … any such difference would be grounds for rejecting the null hypothesis of coming from a single population. Merely testing whether group means differ rarely says much about our theories. We are often quite confident a priori that psychopathology groups will differ on a host of measures. The t-test is rarely put to a richer use, to provide a “risky test” of a theory by examining the predicted size of a group difference. That would be particularly informative when testing a theory that makes a prediction about the size of the effect. But that almost never happens in the psychopathology literature.

Thorny problems of task difficulty, task reliability, and task matching are rarely engaged in studies that attempt to demonstrate a differential (specific) deficit (Chapman & Chapman, 1973, 1978, 2001). This alone is a potentially fatal weakness of much of the psychopathology literature. Vaidyanathan et al. (this issue) do not underscore this issue, but it is implicit in their advocacy of decisive tests of theory.

The issue of statistical power also warrants attention. In recent years, a section entitled “power analysis” is de rigueur in dissertation proposals and grant applications, but in our experience these rarely provide an appropriate power analysis. The primary questions that should be addressed are: how small an effect should the proposed work be powered to find, and how well will this be accomplished for each of the main hypotheses? Guidance from the effect size reported in a more or less related study typically cannot begin to address these questions. We are not arguing that every study should be powered to find small effects, or that only large effects are worth identifying. We are arguing that finding the effect size that another study found is of much value only when working from a theory that makes a point prediction about such an effect size. (Why is that effect size the effect size the proposed work is judged by?) Absent the type of theory-testing that Vaidyanathan and her colleagues advocate, that almost never occurs.

Vaidyanathan et al. (this issue) emphasize the dangers of assuming that new technology will solve our problems, without having to flesh out and test our theories appropriately. Their point can be developed with reference to the growing emphasis on innovation in a grant application or novelty in a data paper, even though most good science is sorting through and testing non-novel ideas and findings. Citing the fact that work is novel simply because it is the first to study population X using task Y and measure Z (especially if Z is relatively new), or because it is the first to report a particular finding, does not represent the spirit of the NIH grant-review criteria of “innovation”. Novelty is of no interest in science, except when we are genuinely surprised – when it is at variance with what we expected; when it is evidence against our theory, explicit or implicit; or when it identifies not only a new but interesting and potentially useful phenomenon or approach. We have to have a theory against which to evaluate the importance of something novel.

A related problem is that grant applications and the introductory sections of data papers frequently do not discuss the motivation for the work. The stated “objectives” or “specific aims” are often in effect to collect data, such as “to examine” or “to assess” some phenomenon. But examining and assessing are the behavior of the scientist, not the motivation of the scientist. What is the burning issue that needs to be addressed or question that must be answered? The need to spell this out is a key feature of the criticism Vaidyanathan et al. (this issue) offer of what they call mere “descriptive correlates”. To expand on this point, Discussion sections all too often review a correlation reported in the Results section, argue that it is suggestive evidence for a causal relationship, and assert that future research is needed to determine whether the finding actually matters, as if the main goal is not to justify future research but to justify the work the authors are trying to get published: put these results out there because they suggest the need for decisive research that this paper does not provide. In the spirit of Vaidyanathan and colleagues, perhaps NIH should consider augmenting or replacing its “innovation” criterion with a “theory-testing” criterion, or a “how important are the mechanisms being tested?”.

Vaidyanathan et al. (this issue) make a compelling case for the need for articulated theory about etiological mechanisms, subjected to risky tests, with less faith placed in data-driven statistics and technology du jour. Their stances on the role of genes in thoroughly mechanistic accounts of psychopathology, on DSM 5 and RDoC, and on the primacy of etiology rather than mere description of empirical relationships are consistently thoughtful. They offer a number of valuable punchlines, such as: “A more technologically sophisticated tool has not necessarily advanced our understanding of mental illness – it may in fact derail it for a time.” On the fundamental importance of tackling etiology: “Convincing nosological systems are those that are based on corroborated etiological theories, preferably with identified causal agents…”. And a caution at the altar of parsimony: “even the best model is inferior to extremely complex model averages (which seem very far from parsimonious).” We have only two minor differences with their contentions, specifically the assertion that using EEG one cannot “identify a single anatomical source” (one can, with great confidence, under some circumstances, although not always of interest to a psychologist) and the characterization of biology as “underlying” mental disorder and personality constructs (for extended critique see Miller, 2010). These very widely held premises do not detract from a paper that is a provocative contribution addressing why, in discovering the etiologies of psychopathology, we have achieved so little with so much effort.