Risky Business: Pathways to Progress in Biologically Informed Studies of Psychopathology

In the target article, Vaidyanathan, Vrieze and Iacono reflect on the lack of advancement in our etiologic understanding of psychopathology. They persuasively argue that theory-guided integrative research, due to its targeted testing of putative mechanisms, should be prioritized. Through a comprehensive narrative, and drawing elegantly on a vast body of endophenotype research, these authors suggest that “risky tests” of causal theories (i.e., a test that narrows the number of causal interpretations available) offer the most potential to inform our understanding of psychopathology. In addition, the authors point out that statistics and technology, when haphazardly applied, can have perilous consequences. They advocate, instead for risky study designs that are guided by a strong theoretical basis, are free of diagnostic boundaries and fueled by knowledge from multiple research domains. We could not agree more with their proposal.

Causal inference: some risks that are worth taking

My way of learning is to heave a wild and unpredictable monkey-wrench into the machinery.

Dashiell Hammett, The Maltese Falcon

Vaidyanathan et al note that “…the riskier the test, the stronger the inference one can draw about the theory being tested.” Such risky study designs may reflect an investigator’s ingenuity in leveraging naturally occurring patterns in the data that result in a quasi-experiment, such as the availability of identical twins discordant for early marijuana exposure. For instance, numerous independent discordant twin studies suggest that the association between early cannabis use and other illicit drugs (e.g., cocaine, heroin, amphetamine), commonly referred to as the gateway phenomenon, is not simply accounted for by a common liability to drug use (Lynskey, Vink, & Boomsma, 2006; Lynskey et al., 2003; Agrawal, Neale, Prescott, & Kendler, 2004; Lessem et al., 2006; Grant et al., 2010). Twins who use cannabis prior to age 17–18 are considerably more likely to also report other illicit drug use relative to their abstaining or late-onset genetically similar counterparts. This observation does not diminish the robust role of common genetic and environmental influences that link cannabis and other drug use nor does it exclude the possibility that neural correlates such as P3 might serve as a component of this common liability (Vanyukov et al., 2012). However, if the basis of this common liability is anything but individual-specific, then it is adequately controlled for in the discordant twin design (Kendler et al., 1993). Something else is clearly going on.

What might this excess risk of illicit drug use in early-onset cannabis users reflect? Even though passive tetrahydrocannabinol (THC) administration in rodents modifies response to heroin, cocaine and nicotine in diverse ways, compelling evidence for a biological mechanism of causal influence (e.g. receptor sensitization) remains tenuous (Cadoni, Pisanu, Solinas, Acquas, & Di Chiara, 2001; Panlilio, Zanettini, Barnes, Solinas, & Goldberg, 2013; Panlilio, Solinas, Matthews, & Goldberg, 2007; Solinas, Panlilio, & Goldberg, 2004). Mendelian randomization might offer a solution (Smith & Ebrahim, 2003); however such experiments rely heavily on a validated genetic marker of vulnerability (e.g. ALDH2 for alcohol consumption in Asians; Irons, McGue, Iacono, & Oetting, 2007), of which there are few. Indeed, we are not aware of any such genetic signals that might be unequivocally linked to early cannabis use. Even if such single nucleotide polymorphisms (SNPs) were identified, tests of Mendelian randomization require that they align with pathways exclusively related to cannabis (and not other substances or confounders), such as its metabolism. Identifying such SNPs may prove to be challenging given that (a) genetic variation in enzymes responsible for the metabolism of THC are not well characterized (Huestis, 2005), and (b) early age at first cannabis use is unlikely to be influenced by genes for metabolism. It is also possible that early exposure to cannabis results in enduring alterations in neural circuitry resulting in a differential response to other psychoactive substances. However, disentangling predisposition from causation in such an experiment requires longitudinal data (e.g., the upcoming NIH-sponsored Adolescent Brain Cognitive Development Study¹). We speculate that the most reasonable causal explanation is the social milieu in which cannabis involvement occurs. Early cannabis use increases youth access to resources that promulgate experimentation with other drugs (Hall & Lynskey, 2005). The growing culture of legalized recreational cannabis use may afford the most practical quasi-experimental test of this hypothesis although a Dutch study of twins discordant for cannabis use did not find decriminalization to have a substantial impact on reducing its association with harder substances (Lynskey et al., 2006). While we haven’t successfully “explained away” the greater odds of other illicit drug use in cannabis users with an early start, discordant pairs of identical twins provide the ideal natural setting to continue to investigate future causal hypotheses (e.g. epigenetic change in gene expression as a result of chronic early cannabis exposure).

Other quasi-experimental risky tests of causation reflect planned or unanticipated environmental occurrences, such as deployment (Nock et al., 2014) or natural disasters (Galea et al., 2007). In these cases, the serendipitous availability of pre-exposure data has been instrumental in establishing putative causal links. The authors offer several such citations in the target article. We are reminded also of the remarkable Great Smoky Mountains Study if only for the fact that the environmental “intervention” resulted in a strong positive impact on subsequent mental health. The Great Smoky Mountains Study is a longitudinal investigation of mental health in children and youth from the highly rural southern Appalachian mountain region of North Carolina, which includes a sizeable Cherokee Nation community (Costello et al., 1996b). Comparisons of these Cherokee Nation children relative to white children from geographically matched neighborhoods as well as from urban settings underscored the important role of poverty in mental health (Costello, Farmer, Angold, Burns, & Erkanli, 1997; Costello et al., 1996a); however, disentangling its role from other socio-regional confounders was next to impossible. Three years subsequent to the baseline, a casino opened on the reservation which led Cherokee Nation families to receive income supplements as well as increased opportunities for employment, including hiring preferences. Families of similarly disadvantaged non-Native American youth did not receive these incentives. Increased educational attainment, reduced criminality and fewer psychiatric diagnoses in children and youth exposed to income supplements was noted, even extending into adulthood (Costello, Compton, Keeler, & Angold, 2003; Akee, Copeland, Keeler, Angold, & Costello, 2010). Such a substantial and long-term monetary intervention of only a controlled subset of the study population is impossible to design, yet it convincingly demonstrated the role of economic enhancements on individual mental health and behavioral outcomes.

Examining the widespread effects of state-level variation in policy also constitutes a risky quasi-experimental research design. For instance, state-level variation in alcohol (e.g. minimum legal drinking age; Plunk, Cavazaos-Rehg, Bierut, & Grucza, 2013) and tobacco (e.g. vending machine restrictions; Grucza et al., 2014a) policy impact substance use (e.g. effects of MLDA on marijuana use; Krauss, Cavazos-Rehg, Agrawal, Bierut, & Grucza, 2015) and related mental health outcomes (e.g. on suicide rates; Grucza et al., 2014b; Grucza et al., 2012). In the absence of self-selection (i.e. evidence that individuals move to states with policies that align with their behavior), lower rates of current smoking, for instance, in women residing in states with vending machine and repackaging restrictions and identification requirements suggest the causal role of legislative control in population-level behavior (Grucza et al., 2013). The strengths of this design are that the environmental exposure is easy to access (although challenging to quantify, e.g. Pacula, Powell, Heaton, & Sevigny, 2014) and the sample size for the test is the U.S. population. That policy is a modifiable target for study makes such studies even more appealing.

Notably, there are powerful risky study designs that can be experimentally contrived, such as, the administration of a psychoactive substance (e.g. marijuana administration; Metrik et al., 2012), the manipulation of human pharmacology (e.g. administration of a pharmacologic agent; Grillon et al., 2011) or the environment (e.g., Trier Social Stress Test; Kirschbaum, Pirke, & Hellhammer, 1993). Broadly described as challenge paradigms, these experiments might be considered as the ultimate (ethically acceptable) risky study design in studies of human behavior as they explicitly test the effects of an experimentally-manipulated variable, typically within the context of a controlled within-subject design.

An illustration of such a risky challenge paradigm can be found in the extensive pharmacologic challenge literature surrounding the role of stress in the etiology of depression and anxiety. Purely associative research has documented that depression, anxiety, and other psychiatric disorders are characterized by dysregulated stress hormone (e.g., cortisol) function within the hypothalamic-pituitary-adrenal (HPA) axis (Faravelli et al., 2012; Stetler & Miller, 2011). Grillon and colleagues examined the impact of placebo-controlled hydrocortisone (i.e., natural or synthetic cortisol used as a medication) administration on startle response (Grillon et al., 2011) Guided by a theoretical model, these investigators separated short duration fear-potentiated startle² from long duration anxiety-potentiated startle³. Fear-potentiated startle has been primarily linked to signaling in the central amygdala, while anxiety-potentiated startle has been associated with signaling in the Bed Nucleus of the Stria Terminalis (BNST), a region within the extended amygdala (Davis, Walker, Miles, & Grillon, 2010; Somerville, Whalen, & Kelley, 2010). Corticosteroid receptor agonism did not affect fear-potentiated startle, but at relatively high doses, increased anxiety-related fear-potentiation, results highly consistent with clinical evidence that individuals with an anxiety disorder display normal fear-potentiated, but heightened anxiety-related startle (Grillon et al., 2008; Grillon et al., 2009; Ray et al., 2009).

There are two obvious limitations of challenge studies. First, laboratory-based pharmacological manipulations are controlled approximations of the normative context in which human psychopathology unfolds and hence, are limited in their ecological validity. Second, sample sizes for these expensive experiments are likely to be small(ish). Problems notwithstanding, pharmacologic challenge studies provide incredible power to address key mechanistic hypotheses typically derived from correlational research. Moreover, because these systems can be similarly manipulated in non-human animal models (e.g., pharmacologic challenge, optogenetics, genetic manipulation), this approach provides unprecedented synergy with non-human animal research allowing more clearly and directly comparable evidence to be integrated across species. The latter brings us to the second prominent recommendation in the target article: a call for a strategy that looks for parallels in findings from multiple domains.

Translational Science: Taking things apart only to put them all together

One’s ideas must be as broad as Nature if they are to interpret Nature.

--Arthur C. Doyle, A Study in Scarlet

The ultimate goal of psychopathological research is to identify processes underlying behavior and illness that surpass mere probabilistic hiccups in individual datasets. While an expectation of reproducibility is reasonable, the advancement of etiologic pathways requires, as the authors put it, a “true synthesis of information.” It is here that “a good scientific theory” can guide the search for an improved mechanistic understanding while encouraging the use of diverse technological and statistical tools. However, such theory-driven approaches are strongly predicated on the construction and acceptance of a priori knowledge and perhaps no field of research underscores the complexities posed by “candidate” hypotheses as psychiatric genetics. The authors of the target article propose that GWAS is motivated by the theory that commonly occurring genetic variation explains an appreciable proportion of heritable variation in psychopathology. Further, they argue that GWAS are risky tests because they occupy the search space encompassed by all candidate gene variants. The rather severe punishment for multiple comparisons (p<5 × 10⁻⁸) is also warranted because the likelihood of chance findings in GWAS is not only high but also quite likely. Correspondingly, the constant skirmishes (e.g. Flint & Munafo, 2013) involving the relative inferiority of candidate gene studies, regardless of the technological innovation of the study design or the scientific rigor of the experiment, have resulted in the marginalization of their impact on psychopathological research. The question then arises: when it comes to gene discovery, if it doesn’t appear in a GWAS, is it not real?

The widespread adoption of GWAS has resulted in some of the most prominent discoveries in psychiatric genetics⁴. In addition to major advances in the identification of single variants (Schizophrenia Working Group of the Psychiatric Genomics Consortium, 2014) and pathways (Network and Pathway Analysis Subgroup of Psychiatric Genomics Consortium, 2015), we now recognize that a substantial proportion of the dark matter of heritability actually resides within the multivariate structure of genomewide data (Yang, Lee, Goddard, & Visscher, 2011), that the polygenic basis to variance in and covariance between psychiatric diagnoses can be systematically quantified (Lee et al., 2013) and that such polygenicity can be teased apart from population substructure (Bulik-Sullivan et al., 2015). However, as Vaidyanathan and colleagues sagely point out, relying on only one source [of information] can be misleading. We illustrate with two examples, now well known in the literature on the genetics of addictions.

The first illustration deals with the now universally recognized role of rs16969968 and other SNPs in the cholinergic nicotinic receptor genes (CHRNA5-CHRNA3-CHRNB4) cluster on chromosome 15 (Bierut, 2011; Ware, van den Bree, & Munafo, 2012). Rs16969968 is a missense mutation that has been identified to be associated, at extraordinarily significant levels (p<10⁻⁷⁰), with cigarettes smoked per day (Liu et al., 2010; Thorgeirsson et al., 2010; Tobacco and Genetics Consortium, 2010), lung cancer (Hung et al., 2008; Thorgeirsson et al., 2008; Amos et al., 2008) and chronic obstructive pulmonary disease (Saccone et al., 2010; Pillai et al., 2009) in multiple large meta-analyses of GWAS. Importantly, the identification of this SNP has spurred multi-disciplinary research into the etiology of a leading cause of preventable death. We now also know that rs16969968 modifies the function of receptors to which nicotine binds (Bierut et al., 2008; Jackson et al., 2010). Mice lacking medial habenular α5 receoptor subunits display increases in motivation to self-administer nicotine (Fowler, Lu, Johnson, Marks, & Kenny, 2011) although a neuroimaging study of nicotine dependent humans suggests that homozygotes for the protective G allele are more likely to show brain activation (e.g. in the hippocampus) in response to smoking-related cues (Janes et al., 2010). The impact of this variant is moderated by age at onset of smoking milestones, by peer (Johnson et al., 2010) and partner (Chen et al., 2014) smoking and parental monitoring (Chen et al., 2009). Importantly, emerging evidence suggests that carriers of the risk-conferring allele (and a related high-risk haplotype) might delay their smoking cessation by two median quit years (Chen et al., 2012). However, these individuals are also ideal targets for cessation pharmacotherapy as its efficacy appears considerably enhanced in those who carry the high-risk haplotype (Chen & Bierut, 2013). In this instance, a single series of GWAS studies revolutionized our understanding of an addictive behavior. Yet, what often goes unmentioned is that the role of rs16969968 in tobacco smoking was first isolated in a hypothesis-driven candidate gene study (Saccone et al., 2007).

For rs16969968, it was simply a matter of statistical power and the good fortune that the associated phenotype (cigarettes per day) was relatively easy to assess in a diversity of studies. However, GWAS was only the debut of this variant and this family of genes. What we have subsequently learned about the manner in which rs16969968 affects the etiology of smoking has largely occurred in independent analytic platforms. One could argue that GWAS, in this case, did not produce a novel genomic target but reassured us about an existing one – we had to take it from there.

In our second example, GWAS have not been as charitable. Commonly occurring single nucleotide polymorphisms in the gene encoding the gamma aminobutyric acid, receptor A, subunit 2 (GABRA2) which have been linked to alcohol dependence, albeit inconsistently (e.g. Irons et al., 2014), across multiple independent samples (Edenberg & Foroud, 2014; Li et al., 2014). GABRA2 was initially selected as a candidate gene because multiple family studies of alcoholism identified an excess of allele-sharing for the region of chromosome 4 which harbors GABRA2 and other GABA-ergic genes (Edenberg et al., 2004). An accumulation of studies also document its association with a variety of externalizing outcomes (e.g. Dick et al., 2009) as well as electrophysiological endophenotypes, such as beta power (Begleiter & Porjesz, 2006; Malone et al., 2014). Even though no GABRA2 SNP under current study results in a protein coding change (rs279858 is in an exon but is synonymous), there is some evidence that they correlate with gene expression (Haughey et al., 2008). There is evidence that variation in this gene is linked to medial frontal response to alcohol aromas (Kareken et al., 2010) and to activation in the insula during monetary reward and loss anticipation (Villafuerte et al., 2011). Rodent research suggests that abolishing the action of the GABRA2 gene results in altered response to ethanol in rodents (Liu et al., 2011; Dixon, Walker, King, & Stephens, 2012) which is corroborated by a human study showing the role of SNPs in this gene in modulating sensitivity to alcohol’s acute effects (Haughey et al., 2008). Despite these independent lines of support for a role of GABRA2 variants in the etiology of alcohol dependence, addictions and other externalizing problems in samples from varied developmental epochs and with differing ascertainment protocols, no GWAS to date has identified these SNPs at genomewide significant levels (Frank et al., 2011; Treutlein et al., 2009; Bierut et al., 2010; Heath et al., 2011; Gelernter et al., 2014; McGue et al., 2013). Did all these studies get it wrong?

If we strictly adhere to the statistical gold standard of GWAS p-values, then the extant GABRA2 signals should be considered false positives. This might also mean diminished enthusiasm regarding future risky study designs related to GABRA2, including the identification of functionally relevant variants in this gene. We might argue that our sample sizes for GWAS of alcohol dependence are small and that when they get large enough, we can unequivocally accept or reject the hypothesis that GABRA2 influences risk for alcohol involvement. However, GABRA2 SNPs were not identified in the only large GWAS meta-analysis of drinks per week (Schumann et al., 2011) nor was it significant in a recent large GWAS of alcohol dependence (Gelernter et al., 2014) which successfully identified rs1229984⁵. We can wait until those impressive big data experiments are completed or, with appropriate circumspection, we can let a good scientific theory guide us towards bettering our understanding of the role of GABRA2 in alcohol dependence right now. Much like Vaidyanathan and colleagues, we concur that approaches like GWAS or whole brain analyses (versus analyses involving regions of interest, or ROI) afford the benefit of agnosticism which is ideal for thorough statistical confidence but they devalue, and even penalize, accumulated knowledge about the etiology of psychopathology. We can and have learned amazing lessons from GWAS but quite likely, we’ve missed quite a few opportunities because of our reliance on a single statistical threshold.

Statistics and Technology: Just say when…or maybe not

The target article by Vaidyanathan, Vrieze and Iacono outlines a scientific outlook with the potential to unfetter progress in psychopathological research. We have elaborated on the significance of two key suggestions made by these authors: the importance of risky study designs and the value of theoretical insight when conducting biologically informed research. We are certain that readers will identify other aspects of the target article that will inspire their research objectives.

We close, however, with one potential viewpoint where we diverge (if only modestly) from the authors’ position. The authors caution against “statisticism” and “technomyopia.” We conceptually agree with their contention that rogue attempts at statistical modeling and technological gloss, in the absence of a theoretical mooring, are apt to generate noise. The problem with this assertion, one that we confront in our own work as well, is: in the absence of the luxury of hindsight, how do we adjudicate what constitutes an unwarranted implementation of statistics or technology? The case of GWAS is a good one. In the afterglow of the unsurpassed progress made by the Psychiatric Genomics Consortium (PGC; (Sullivan, Daly, & O’Donovan, 2012)) and other large consortia (e.g. ENIGMA; Thompson et al., 2014), we are convinced that GWAS represents a judicious use of technology. However, suspicions regarding the contributions of common variation to psychopathology in the past decade have profoundly impacted funding allocations for GWAS (Sullivan, 2012). Therefore, in most cases, the benefit or futility of a technology (or statistical approach) may play out in a future that may not be worth predicating current research effort on.

Instead of figuring out how to (or who should) arbitrate when and how technological and statistical power should be wielded, why not use multiple lines of investigation to substantiate our hypothesis? Technology and statistics are tools, inasmuch as a guiding theory is just that, a theory. What distinguishes a guiding principle from an opinion is a statistically valid experiment; while not everything that counts can be counted, that which can, should be counted properly. It also doesn’t hurt to add a little technological savvy to the process – a clever experiment might fail to reject the null hypothesis just as well as one that is ordinary but we may have more fun conducting the former. Our point is this: one person’s risky test is another’s observational study (e.g. GWAS – a risky test?); and every observational study is the progenitor of a risky study design. Research should be evaluated for its originality, scientific merit and experimental rigor, be it motivated by a guiding theory or the consequence of a sexy statistical model. Perhaps not everything that can be counted counts, but that’s probably what peer-review is for.