Computational Psychiatry

Abstract

Psychiatric disorders such as autism and schizophrenia arise from abnormalities in brain systems that underlie cognitive, emotional and social functions. The brain is enormously complex and its abundant feedback loops on multiple scales preclude intuitive explication of circuit functions. In close interplay with experiments, theory and computational modeling are essential for understanding how, precisely, neural circuits generate flexible behaviors and their impairments give rise to psychiatric symptoms. This Perspective highlights recent progress in applying computational neuroscience to the study of mental disorders. We outline basic approaches, including identification of core deficits that cut across disease categories, biologically-realistic modeling bridging cellular and synaptic mechanisms with behavior, model-aided diagnosis. The need for new research strategies in psychiatry is urgent. Computational psychiatry potentially provides powerful tools for elucidating pathophysiology that may inform both diagnosis and treatment. To achieve this promise will require investment in cross-disciplinary training and research in this nascent field.

Introduction

In 1988, a computational neuroscience “manifesto” (Sejnowski et al. 1988) mentioned three reasons for the emergence of this new research field: advances in neuroscience had generated a large body of neurophysiologic data, new computers possessed sufficient power to conduct neural model simulations, and simplified brain models were introduced that provided insights into complex neural circuit functions. Since then, dramatic advances made on all three fronts fundamentally changed the computational neuroscience landscape (Abbott 2008). Notably, computational neuroscience initially focused on the early stages of sensory processing (Sejnowski et al. 1988), because studies of the neural bases of higher cognitive functions were beyond empirical neuroscience of that era. Indeed, only in recent years, has the confluence of single-unit physiology, human functional brain imaging, and advances in computational modeling made significant strides in tackling executive functions (such as working memory and decision-making) that underlie cognitively controlled flexible behavior. These higher functions critically depend on the prefrontal cortex (PFC) (Fuster 2008, Miller and Cohen 2001, Wang 2013, Szczepanski and Knight 2014). Because impairments of the PFC and related circuits are implicated in major psychiatric disorders, such as schizophrenia and autism (Goldman-Rakic 1994, Insel 2010,Courchesne et al. 2011, Anticevic et al. 2013), the newly acquired insights and computational models offer an opportunity to elucidate how cellular and circuit level pathologies give rise to cognitive deficits observed in mental illness, advances in this direction could inform studies of psychiatric diagnosis, pathophysiology and treatment.

Therefore, the time is ripe for Computational Psychiatry to emerge as a field at the interface between basic and clinical neuroscience (Montague et al. 2012, Friston et al. 2014). In this Perspective, we review recent work demonstrating that computational psychiatry introduces novel approaches and tools to investigate neural circuit mechanisms underlying the cognitive and behavioral features of neuropsychiatric disorders. First, we will spell out the rationale of a computational approach to Psychiatry, i.e., “why Computational Psychiatry? What theories and models are relevant to this field?” Second, we will discuss how theories and models have been applied to the investigation of behavioral impairments in terms of transdiagnostic endophenotypes. Third, we will summarize recent work that advocates for a model-aided framework of diagnosis and treatment. The fourth part will be devoted to biophysically-based neural circuit modeling that we argue represents the optimal approach for cross-level understanding from cellular processes to collective and emergent circuit dynamics and ultimately to behavior. Fifth and finally, we will end with practical recommendations related to the training and funding needed to foster this nascent field.

Why Computational Psychiatry?

It is widely acknowledged that current psychiatric diagnostic schema and the treatments for psychiatric disorders lack a firm biological foundation. The complexity of the brain presents unique challenges to the development of highly specific mechanistic hypotheses to guide research in psychiatry. Advances in genetics, molecular and cellular neurosciences are providing, at long last, clues to the etiology of human cognitive, emotional, and behavioral problems. For example, candidate-gene studies have revealed gene variations (such as DISC1 (Brandon et al. 2009)) associated with psychiatric disorders. However, many in the field feel that attempts to seek single genes underlying complex psychiatric phenotypes have been largely disappointing, and that efforts to link genes to more basic cognitive and behavioral functions and functional impairments could be more promising. The progress in these areas has yet to provide a firm basis for a diagnostic system or a single pharmacotherapy for common psychiatric disorders (Krystal and State 2014).

A major hindrance in our capacity to develop novel pharmacotherapies for psychiatric disorders is the extremely superficial nature of our understanding of how circuits represent behavior. In this regard, synaptic and systems physiology are producing remarkable advances in our specific understanding of the functional properties of microcircuits and the beginnings of connecting these insights into behavioral processes including basic visual perception (Parker and Newsome 1998), fear conditioning and extinction (Johansen et al. 2011), and mental representations in working memory (Arnsten et al. 2010). There are even examples where aspects of the neural representation of distinct fear memories can be ascribed to the functional integrity of a few distinct sets of cells in the amygdala (Josselyn 2010). Yet, perhaps as a consequence of the limitations of our animal models combined with the limited spatial and temporal resolution of current neuroimaging technologies (MRI, MEG, PET), there is not a single symptom of a single psychiatric disorder for which we fully understand its physiologic basis at a molecular, cellular, and microcircuit level. In other words, we have only a somewhat vague idea of how the brain generates the cognitive, emotional, and behavioral problems that lead people to seek treatment by psychiatrists and other mental health clinicians.

As a consequence of our limited understanding of how circuits represent information, there are a plethora of attempts to explain circuit dysfunction in psychiatric disorders in superficial ways, giving rise to an equally large number of relatively risky potential pharmacologic strategies to address the unmet need for more effective treatments. The implications of this knowledge gap are profound for the field of psychiatry and for society. For example, psychiatric diagnoses have categorical qualities as exemplified by the Diagnostic and Statistical Manual for Psychiatric Disorders 5 (DSM-5). Although this new version of DSM takes into consideration the recent explosions in the genetics of disorders, such as autism and schizophrenia (Krystal and State 2014), it is widely criticized for lack of a solid biological foundation based on either etiology or pathophysiology. Categorizing patients by symptom checklists results in enormous clinical heterogeneity within diagnostic categories, surprisingly poor inter-rater reliability for many common psychiatric diagnoses (Freedman et al. 2013), and very likely, poorer clinical outcomes.

An alternative schema has emerged from the recognition that behavioral impairments are traits that may be shared across psychiatric disorders (Krueger 1999). The shift from a categorical diagnostic focus to a dimensional transdiagnostic approach emerged in the form of the Research Domain Criteria (RDoC, http://www.nimh.nih.gov/research-priorities/rdoc/index.shtml) (Insel et al. 2010, Insel 2014). The RDoC program aims at identifying core cognitive, emotional and social dysfunctions, then elucidating their brain mechanisms bridging different levels (from molecules, cells, circuits to functions). Yet, the next step in this process is to determine whether the circuits are dysfunctional in the same way across disorders or whether, when characterized in increasingly accurate molecular and physiological ways, categorical features of psychiatric diagnoses reemerge. Further, diagnoses may have both categorical and dimensional features. For example, schizophrenia appears to be a more severe form of circuit dysfunction than bipolar disorder with respect to the thalamo-cortical functional connectivity (Anticevic et al. 2013), but a completely distinct type of disorder than bipolar disorder with respect to the variance or “noise” level of cortical activity (Yang et al. 2014). Neither DSM or RDoC in its current form provides guidance as to how to integrate the dimensional and categorical features of psychiatric pathophysiology. A second consequence is the lack of precision with which one can predict whether a particular treatment mechanism will work for psychiatric disorders. It is not just that biomarkers of illness are lacking, but rather the biomarkers that we have are not sufficiently mechanistically precise as to specify a particular treatment. Further, even when aspects of molecular pathology are characterized, the impact on micro-and macro-circuit functions and the paths to correct that circuit dysfunction are not clear. As a result, in the case of schizophrenia, it is not clear that GABA signaling deficits (Lewis et al. 2005, Lewis and Gonzalez-Burgos 2006) should be treated by GABA_A receptor agonists nor deficits in NMDA receptor signaling should be treated with drugs that increase the stimulation of the glycine co-agonist site of the NMDA receptor (Buchanan et al. 2011, Goff 2014).

The gap between genetic, molecular, and cellular studies, on the one hand, and systems and behavioral neuroscience studies, on the other, currently cannot be bridged purely through experimentation. Take, again, the example of the prefrontal cortex (PFC). Its crucial role in a wide range of executive functions (Fuster 2008, Miller and Cohen 2001, Wang 2013) begs the question: what are the key properties that enable the PFC to subserve cognitive processes, in contrast to primary sensory or motor systems? This question is difficult to address by laboratory experiments alone, partly because PFC circuitry is endowed with powerful positive and negative feedback loops and the behavior of any such dynamical system is not predictable by intuition alone. While physiological studies in animals and humans yield data on the correlation of particular measurements to specific cognitive operations, theory and modeling are usually needed, together with experimentation, to investigate the “follow up” questions: what circuit mechanisms give rise to the observed neuronal and other brain signals? What are the computational algorithms and generalizable principles that are reflected in the observed biological signals and sufficient to explain behavior?

Computational modeling offers a suitable approach to quantitatively explore the properties of complex systems across levels of investigation. Therefore, by incorporating computational neuroscience modeling within translational neuroscience research programs, it may be possible to develop more specific hypotheses related to circuit dysfunction in model systems and psychiatric disorders. There are many forms of computational models, the present article covers two kinds. Models of Mathematical Psychology or algorithmic models from Computer Science are enormously useful for quantifying behavioral data and relating their fitted parameters to neural computations (Maia and Frank 2011, Montague et al. 2012). On the other hand, biophysically-informed computational modeling, that are constrained by the biophysical properties of identified synaptic signaling mechanisms and other properties of microcircuits, has proven to be an effective approach to understanding the neurobiology underlying cortical functions and psychiatric disorders (Wang 2006, Anticevic et al. 2013).

Biologically-based neural circuit models

What is biologically-based neural circuit modeling? Simply put, it is a computational framework that is constrained by neurobiology and designed to achieve a cross-level understanding of brain functions in terms of neural dynamics, computation, and biological mechanisms (Figure 1). One may question whether such models are too complex to be useful in cognitive science or psychiatry (Carandini 2012, Montague et al. 2012). Three points are worth noting on this regard. First, biologically-based modeling is a broad term that embraces a diversity of models with varying degrees of complexity. A model does not necessarily improve when more biological details are included. There is always a tradeoff between incorporating important details in order for the model to be suitable (given a scientific question) on one hand and simplicity and generalizability on the other hand. It is also tremendously useful to be able to go back and forth between models differing in their levels of abstraction, for instance between a spiking network model and its reduced “mean-field” firing-rate model for population-level dynamics. Second, neuronal modeling is most appropriate for those functions for which we have some knowledge about the underlying neural processes, such as dopamine neural signaling of reward-prediction error, persistent activity subserving the internal representation of working memory and neural integrators in perceptual decision-making. By contrast, modeling at the neuronal level would seem premature for other behavioral phenomena such as hallucination or feeling of depression, in the absence of neurophysiological characterization. Finally, to the extent that biophysically-based neural circuit modeling begins by incorporating the simplest and most fundamental features of synaptic connectivity, it is arguably the simplest possible framework that permits us to elucidate the inter-relationship between biological mechanism, neural dynamics and computations, and circuit functional output (Figure 1A).

Figure 1.

(A) Mechanistic understanding of brain functions must relate structure (molecules, cells and network connectivity) and dynamics with behavior. Brain measures probe spatiotemporal neural activity patterns that are correlated with specific aspects of behavior. Theory and modeling provide a powerful tool to elucidate how such a pattern is produced by its biological substrate, on one hand, and give rise to computations necessary to account for brain function, on the other hand. (B) Biologically-based neural circuit modeling is calibrated by physiology of single neurons and synapses, and constrained by quantitative network connectivity data. This approach is arguably necessary for the 3-way understanding between function, neural dynamics and computation, biological mechanism.

In a spiking network model, single neurons are often described by either the leaky integrate-and-fire model or the Hodgkin-Huxley model. These models are calibrated by physiological measurements, such as the membrane time constant and the input-output function (the spike firing rate as a function of the synaptic input current), which can be different for excitatory pyramidal cells and inhibitory interneurons. Furthermore, it is worth emphasizing that in biophysically-based models, synapses must be modeled accurately. Unlike connectionist models in which coupling between neurons is typically an instantaneous function of firing activity, synapses have their own rise-times and decay time constants, and they exhibit summation properties. Synaptic dynamics are crucial factors in determining the integration time of a neural circuit and the stability of a strongly recurrent network (Wang 1999). Finally, networks endowed with a biologically plausible architecture need to be constructed based on quantitative anatomy (Douglas and Martin 2004). For example, a commonly assumed circuit organization is local excitation between neurons of similar selectivity combined with a more global inhibition. Dynamic balance between synaptic excitation and inhibition is another feature of cortical microcircuits that has been increasingly recognized experimentally and incorporated in cortical network models (http://www.scholarpedia.org/article/Balance_of_excitation_and_inhibition).

Consider decision-making, the process of reaching a particular choice among several alternative options, such as rendering a judgment out of multiple possibilities given incomplete information or choosing one of actions expected to yield different outcomes (Glimcher 2003, Gold and Shadlen 2007, Wang 2008, Glimcher and Fehr 2013). Broadly speaking, there are two types of computational models of decision-making: behavioral models and neural circuit models. In Behavioral Psychology, decision-making is commonly modeled by the drift diffusion model (DDM) (Ratcliff 1978, Smith and Ratcliff 2004). In this model, an activity variable X represents the difference between the respective amounts of accumulated information about the two alternatives, say X_A and X_B, X = X_A − X_B. The dynamics of X is given by the drift diffusion equation, dX/dt= μ + w(t), where μ is the drift rate, w(t) represents noise. The drift rate μ represents the bias (net difference in the evidence) in favor of one of the two choices (and is zero if there is no net bias). For instance, in a random-dot motion direction discrimination task, μ is proportional to the strength of motion signal. This system is a perfect integrator of the input. The integration process is terminated and the decision time is read out, whenever X(t) reaches a positive threshold θ (choice A) or a negative threshold −θ (choice B). If the drift rate μ is positive, then choice A is correct, whereas choice B is an error. Therefore, this type of models is commonly referred to as ramping-to-threshold model, with the average ramping slope given by μ.

A biophysically-based neural circuit model has been proposed for decision-making (Wang 2002). This model reproduces not only behavioral observations but also single neural activity associated with decision-making observed in a monkey experiment (Roitman and Shadlen 2002). Moreover, it suggests a specific biological basis for temporal accumulation of evidence in decision-making. The drift diffusion model is an ideal perfect integrator (with an infinite time constant), whereas neurons and synapses are leaky with short time constants of tens of milliseconds. The neural circuit model suggests that a long integration time can be realized in a decision network through recurrent excitation. Reverberating excitation represents a salient characteristic of cortical local circuits (Douglas et al. 1995, Douglas and Martin 2004). When this positive feedback is sufficiently strong, recurrent excitation in interplay with synaptic inhibition can create multiple stable states (“attractors”). Such models have been initially proposed for working memory. The same models, provided that excitatory reverberation is slow (i.e. mediated by the NMDA receptors), has been shown to be capable of decision-making computations (Wang 2002, Machens et al. 2005, Miller and Wang 2006, Wong and Wang 2006, Soltani and Wang 2006, Deco et al. 2007, Wang 2008, Furman and Wang 2008, Deco et al. 2009, Engel and Wang 2011, Hunt et al. 2012). Interestingly, physiological studies in behaving non-human primates often reported neural activity correlated with decision making in cortical areas such as the prefrontal cortex or the parietal cortex, that also exhibit mnemonic persistent activity during working memory. Hence, this model and supporting experimental data suggest a common, “cognitive-type” circuit mechanism for decision-making and working memory in the brain (Wang 2013).

Behavioral modeling is often powerful in describing computations that solve a problem normatively or algorithmically. On the other hand, neural circuit models may be more suited for enabling us to investigate the underlying neural mechanisms and potentially pharmacologic or genetic manipulations of the circuits. Importantly, neural circuit models are not merely implementations of abstract mathematical models. For instance, the two types of models of perceptual decision-making have distinct predictions at the behavioral level (Wang 2008). These approaches are usually developed independently, but we are witnessing some convergence of the two in recent years. For example, spiking network models have been shown to have the capability of fitting quantitatively with behavioral performance (accuracy and reaction time) data (Lo et al. 2009), whereas such data fitting and model comparisons are commonly done with more abstract models due to their lower computational cost. Spiking network models can also be reduced to population rate models (Wong and Wang 2006), that have features of abstract connectionist models. On the other hand, connectionist neural network models have increasingly taken biological information (with identified brain structures, receptors, etc) into account (O’Reilly and Frank 2006). Thus, to bridge gaps in the current knowledge base and to facilitate research, there are advantages to move back and forth across several models that vary in their degree of abstraction, biological realism, their level of analysis (circuits, computational operations, behaviors).

Endophenotypes across brain disorder categories

Inasmuch as features of the pathophysiology of psychiatric disorders are shared across diagnostic boundaries (Krueger 1999), a promising research direction is to search for trans-diagnostic endophenotypes, i.e., quantitative heritable traits that are intermediate between risk genotypes and the psychiatric disorder syndrome itself (Figure 2A, Gottesman and Gould 2003). While it has yet to be demonstrated that endophenotypes have a more simple genetics than psychiatric diagnoses, there remains a hope that endophenotypes may be more precisely defined, measured, and related to the underlying biology and to animal models. For instance, impulsivity and compulsivity are behavioral endophenotypes that cut across a range of diagnostic categories including obsessive-compulsive disorders, substance dependence, attention-deficit/hyperactivity disorder. Neither impulsivity nor compulsivity may be unitary constructs, but they may derive from a set of psychological processes which themselves are candidate endophenotypes (Figure 2B, Robbins et al. 2012). Thus, one could show impulsive choice behavior because of an aversion to delayed gratification, or impulsive response due to motor disinihibiton or timing impairment. While this dimensional approach has not supplanted the prevailing psychiatric diagnostic schema, it has powerfully stimulated psychiatry research.

Figure 2.

(A) Gene regions, genes, and putative endophenotypes implicated in a biological systems approach to schizophrenia research. The dynamic developmental interplay among genetic, environmental, and epigenetic factors that produce cumulative liability to developing schizophrenia. Endophenotypes as schizophrenia discriminators involve sensory motor gating, oculomotor function, working memory, and glial cell abnormalities. Many more gene loci, genes, and candidate endophenotypes remain to be discovered (represented by question marks). The figure is not to scale. (B) The impulsivity and compulsivity constructs. The diagram describes possible psychological component mechanisms underlying the two constructs. It would appear that these different measures likely do not inter-correlate well, which would argue against a unitary construct for either impulsivity or compulsivity, but this issue is still actively being researched. Both impulsivity and compulsivity involve motor/response disinhibition, but at different stages of the response process. (A) was reproduced from Gottesman and Gould (2003), (B) from Robbins et al. (2012), with permission.

It is a major challenge to accurately and reliably identify endophenotypes. To make progress, it is beneficiary to complement consideration of symptoms (how people feel) with attention to what people do (choices and actions). By using behavioral paradigms that are designed to probe a specific cognitive function or functional domain, one can quantify the abnormalities of a particular function that are shared by multiple mental disorders. Those carefully designed tasks should be doable by both human subjects and nonhuman animals, thereby enabling more productive translational research (Carter et al. 2008, Wang 2013, Insel 2014). Theories can be developed and applied to both normal subjects and patients, providing insights into the core of a brain dysfunction.

Consider the case of disturbances in decision-making. Many people, who meet current diagnostic criteria for a number of neuropsychiatric disorders, repeatedly make bad choices in the social, vocational, and recreational domains that compromise the quality of their lives. There is increasing evidence that specific impairments in decision-making may represent cognitive endophenotypes across diagnostic boundaries (Robbins et al. 2012, Montague et al. 2012). A number of studies have dealt with the valuation process in reward-based decision-making. The computations that enable one to learn to evaluate alternative options through experience are fundamental for adaptive choice behavior, i.e., to make a choice, assess its outcome, and to use this experience to guide the next choice. Reinforcement learning (RL) theory (Rescorla and Wagner 1972, Sutton and Barto 1998, Rangel et al. 2008) offers a framework for this adaptive process and impairments associated with psychiatric conditions (Montague et al. 2012, Maia and Frank 2011, Lee 2013). This field, which lies at the interface behavior and neurobiological mechanisms, was galvanized by the discovery that phasic activity of dopamine neurons in the ventral tegmental area signals reward prediction error (RPE) (Montague et al. 1996, Schultz et al. 1997). Specifically, dopamine phasic firing has been shown to confirm with RPE according to temporal-difference RL (TDRL) (Sutton and Barto 1998, Dayan and Abbott 2001). TDRL computes the reward expectation in terms of all anticipated reward events in the future, and learns to predict reward by driving RPE to zero. For the sake of simplicity, here we describe a simplified notion of RPE, δ_t = r_t − V_t, where r_t is the actual reward and V_t is the expected reward, at time t). The idea is that the mismatch between the actual reward and the expected reward generates an “error signal” that informs learning. RL is hypothesized to be driven by αδ_t, with the rate α controlling the speed of learning. Therefore, there is a solid foundation for bridging reward-related learning with a specific underlying brain circuit (the dopamine system). Empirical evidence for impaired RL has been documented for Parkinson’s disease, schizophrenia, Tourette’s syndrome, attention deficit disorder, drug addiction, depression (Maia and Frank 2011, Lee 2013, Huys et al. 2013), demonstrating powerfully the importance of function-based, transdiagnostic, approach in psychiatry.

For instance, addiction can be viewed as RL gone awry. Indeed, a pioneering application of RL to psychiatry (Redish 2004, Redish et al. 2007) was inspired by TDRL. It was proposed that addiction access the same RL system as in the normal brain, but drug-induced positive prediction errors could produce unbounded increases in the value of drug receipt. A merit of such quantitative models is that they are precise enough to be falsifiable by new experiments, a hallmark of scientific inquiry. Redish’s model predicts that a behavioral trait called blocking does not occur when drugs are used as unconditional reinforcers. Blocking refers to the observation that after a subject learns to associate a stimulus A with a reward, later pairing A with another stimulus B should not lead to learning to associate B with the reward. If, however, drugs (as stimuli A and B) lead to unlimited value increase, blocking should not be observed. Behavioral experiments using cocaine as unconditional stimulus showed that this is not the case, i.e. blocking does occur (Panlilio et al. 2007). One possible interpretation of this result is that blocking is not due to the specific form TDRL of RL. Indeed, blocking is accounted for in an alternative model of addiction that assumes the expected reward V_t to be computed by a weighted average over past reward events (Dezfouli et al. 2009). Another possibility is that RL involves multiple competing systems (Redish et al. 2007).

The RL approach has also been applied to depression. Huys et al. (2013) set out to test the hypothesis that depression is associated with an altered sensitivity to reward; specifically, the RPE becomes δ_t = ρr_t − V _t, where the parameter ρ represents reward sensitivity. Meta-analysis of experiments with about 50 healthy subjects and 50 subjects with major depression disorder revealed has been carried out by fitting behavioral data with a RL model. It was found that compared to the control group, the patient group shows a significantly reduced reward sensitivity (a smaller value of ρ), but no change in the learning rate α, consistent with the anhedonia and lack of motivation found in patients with depression. Similar findings were also reported by Strauss et al. (2011). This work illustrates how computational modeling enables us to dissect distinct aspects (reward sensitivity but not learning rate) of a maladaptive behavior.

The RL theory is currently been extended beyond single-factor considerations. In particular, it has been recognized that RL involves two separate neural systems (Balleine and Dickinson 1998, Daw et al. 2005, 2011, Kahneman 2011, Dolan and Dayan 2013). One of these systems subserves habits and related behaviors. It is referred to as “model-free” because these behaviors are elicited in an automatized way by cues. The second, model-based, system is endowed with an internal representation of the causal structure of the environment and underlies goaloriented behaviors. The model-free and model-based systems must be balanced. A dual-system learning model (Daw et al. 2011) has been combined with human brain imaging to examine specific ways an imbalance of these two systems might lead to maladaptive choice behavior in mental illness. Using this framework, it was found that repeated exposure to addictive drugs shifts behavior from model-based to model-free emphasis (Kurth-Nelson and Redish 2011, Lucantonio et al. 2012). Likewise, data fitting by the dual-system model revealed that subjects diagnosed with obsessive-compulsive disorder display a bias towards model-free habit acquisition (Voon et al. 2014). The central control mechanisms governing the balance maintenance and shifts between model-based and model-free systems represent an area of intense ongoing research (Simon and Daw 2011).

Whereas the model-free system relies on RPE, the model-based system presumably depends on a more abstract “state prediction error” which might implicate lateral prefrontal cortex, giving rise to “dual system” RL models (Glascher et al. 2010). RL approaches have advanced translational neuroscience research on such phenomena as delusions that have been previously extremely challenging to study from this perspective. The focus on prediction error, a mismatch between expectation and experience, has inspired neurobiological studies of psychosis (Corlett et al. 2010). Delusions are false beliefs about the world that persist tenaciously despite repeated encounters with contradicting evidence. Corlett and his colleagues (Corlett et al. 2007) found that violations of causal associations activate the right lateral prefrontal cortex (rPFC) during fMRI, a putative prediction error signal. However, deficits in this fMRI prediction error signal among subjects with first episode psychosis strongly correlated with the severity of delusions across subjects (Corlett et al. 2007). Thus, false beliefs may be generated through compromised prediction error and sustained as aberrant learning transitions from being represented by model-based to model-free systems (Corlett et al. 2010).

RL has also been extended to hierarchically organized behaviors (Botvinick et al. 2009). These studies focused on RL illustrate well how theory and computational modeling, in conjunction with experimentation, can help dissect distinct component processes (such as reward sensitivity, learning rate, balance between model-free and model-based systems, et al.), each may be abnormal in multiple mental disorders but in different ways. This opens up the possibility that each cognitive endophenotype (such as impulsivity) could be defined in terms of a specific combination of quantitative impairments of these component processes. If so, future progress in this direction could yield a promising new framework to guide translational neuroscience studies of neuropsychiatric disorders.

Big data and model-aided diagnosis

Typically, the process of building from a behavioral experiment to a computational model follows several steps: (1) a cognitive task is strategically designed to probe a particular function (e.g. reward-related learning in decision-making), (2) an appropriate computational model (e.g. reinforcement learning) is chosen to simulate the process (e.g. valuation) under consideration, (3) model-fitting of data yields estimation of model parameters (e.g. reward sensitivity and learning rate). Many of these studies compare people deemed to be free of a psychiatric diagnosis to people who have been recruited specifically for the presence of a specific psychiatric diagnosis (e.g. according to DSM or international classification of diseases (ICD) criteria). Significant differences between the healthy group and patient group in some model parameters (e.g. reward sensitivity but not learning rate) provide the basis for characterizing the presumed “abnormality” in the patient group. However, computational psychiatry is not limited to existing diagnostic schema. Its focus on relating mechanisms to cognitive operations and behavioral processes promotes a transdiagnostic perspective. For instance, a similar bias toward model-free versus model-based learning has been found in disorders involving both natural (binge eating) and artificial (methamphetamine) rewards, as well as obsessive-compulsive disorder (Voon et al. 2014).

Recently, Frank and collaborators (Wiecki et al. 2014) proposed to extend this approach from subject groups to individuals. This requires a fourth step, i.e., to use sophisticated statistical analysis algorithms to investigate whether model parameter values extracted from individual subjects are clustered into distinct groups (Figure 3A). This step is crucial for this paradigm to potentially serve as a clinical tool, since diagnosis must obviously be done for single individuals. A similar approach has been advocated by Stephan and his colleagues (Figure 3B) (Brodersen et al. 2014). These authors proposed a cross-disciplinary approach that combines behavior, brain measures (fMRI) and computation (dynamical causal modeling, DCM (Friston et al. 2003, Stephan et al. 2007)). In a working memory study of schizophrenic patients they focused on DCM based estimates of effective connectivity between visual, parietal and prefrontal cortex, since these three cortical areas were critically involved in their visual working memory task. An unsupervised clustering procedure operating on the individual connectivity patterns yielded three distinct patient subgroups (Figure 3C): (a) those with greater fronto-parietal connectivity, (b) those with weaker fronto-parietal connectivity, and (c) those with greater visuo-frontal connectivity. The authors further pushed the approach by including two more steps (Figure 3B): (5) assessment of whether clusters of subjects obtained by model-fitting are correlated with different severity of behavioral impairment (indeed they found that subjects in the three clusters display a different degree of negative symptom severity (Figure 2D)), and (6) interpretation of the results from step (5) that attributes the behavioral deficit (negative symptom) to a possible underlying brain substrate (visual-parietal-prefrontal circuitry connectivity), generating new hypotheses to be tested in future research.

Figure 3.

(A) Illustration of the 4 levels of computational psychiatry. Clinical and nonclinical populations are tested on a battery of cognitive tasks. Computational models can relate raw task performance (e.g. RT and accuracy) to psychological and/or neurocognitive processes. These models can be estimated via various methods. Finally, based on resulting computational multidimensional profile, training using learning algorithms can either uncover groups and subgroups in clinical and healthy populations, or relate model parameters to clinical symptom severity. (B) Conceptual overview of model-aided clustering of fMRI data. First, separately for each subject, BOLD time series are extracted from a number of regions of interest. Second, subject-specific time series are used to estimate the parameters of a model. Third, subjects are embedded in a score space in which each dimension represents a specific model parameter. This space implies a similarity metric under which any two subjects can be compared. Fourth, a clustering algorithm is used to identify salient substructures in the data. Fifth, the resulting clusters are validated against known external (clinical) variables. Once validated, a clustering solution can, sixth, be interpreted mechanistically in the context of the underlying model. (C–D) Model-based clustering of fMRI data from schizophrenic patients in a working memory task. (C) An unsupervised clustering analysis of the patient group only, using Gaussian mixture models operating on dynamical causal model (DCM) parameter estimates, yield the average posterior parameter estimates (in terms of maximum a posteriori estimates) for each coupling and input parameter in the model. This is displayed graphically by the thickness of the respective arrows. (D) The three subgroups, which are defined on the basis of connection strengths, also differ in terms of negative clinical symptoms as operationalized by the negative symptoms (NS) subscale of the PANSS score. (A) was reproduced from Wiecki et al. (2014), (B–D) from (Brodersen et al. 2014), with permission.

This line of work raises the question of whether it might be possible to use brain imaging data (or models of such data) rather than symptoms as the substrate for diagnostic classification schema. A related line of thinking is to view psychiatric illness from the perspective of brain connectome (Rubinov and Bullmore 2013), according to which the analysis of functional connectivity patterns inferred from brain imaging offers a window to pathoconnectomics associated with mental disorders. It would be interesting to know the impact of attempting to, on a very large scale, develop model parameters that cluster patients in new ways. Would this approach yield a classification schema different from DSM? Would this classification schema be replicable and generalizable? Would it suggest new directions for research and treatment? This type of strategy might address a conundrum in psychiatry, i.e., the absence of biomarkers. It may be impossible to develop meaningful illness biomarkers within a diagnostic framework that is not based in biology. However, if the diagnostic framework were, itself, built around an imaging biomarker, then it would seem highly likely that this biomarker would have predictive power with regards to diagnosis and treatment.

A number of factors will determine the success of this framework: very large samples of subjects, efficient and statistically reliable analysis methods, and judicious choices of computational models. With the advance of big data science, and computational modeling, a radical modern paradigm shift may be on the horizon.

Biophysically-based neural circuit modeling: understanding across levels

In contrast to more abstract models, biophysically realistic neural circuit modeling has the potential to be rigorously calibrated by quantitative neurophysiology and anatomy. Ultimately, this is necessary to elucidate deficits at the molecular, cellular and circuit levels that underlie cognitive and behavioral disorders in mental illness.

Among hierarchically inter-related cognitive dysfunctions associated with schizophrenia (Millan et al. 2012), perhaps the best studied is working memory (Park and Holzman 1992, Lee and Park 2005, Lewis and Gonzalez-Burgos 2006, Barch and Ceaser 2012). Working memory, the brain’s ability to encode and sustain the neural representation of information in the absence of direct sensory stimulation and to manipulate this information in the service of future action, is a core cognitive function that depends on the PFC (Fuster 2008, Goldman-Rakic 1995, D’Esposito 2007, Baddeley 2012). Fortunately, working memory has been particularly amenable to biophysically-based neural circuit modeling, because of the richness of experimental data at multiple levels of study.

A well known working memory paradigm is the delayed oculomotor response task, in which a subject is required to remember a visual cue (a directional angle) across a delay period in order to perform a memory guided saccadic eye movement (Funahashi et al. 1989, Constantinidis and Wang 2004). A biologically-based network model of spiking neurons has been developed for this spatial working memory experiment (Fig. 4A) (Compte et al. 2000, Renart et al. 2003, Wang et al. 2004, Carter and Wang 2007, Wei et al. 2012, Kilpatrick et al. 2013, Hansel and Mato 2013, Pereira and Wang 2014). Fig. 4B shows a model simulation of the delayed oculomotor task. Initially, the network is in a resting state in which all cells fire spontaneously at low rates. A transient input drives a subpopulation of cells to fire at high rates. As a result they send recruited excitation to each other via horizontal connections. This internal excitation is large enough to sustain elevated activity, so that the firing pattern persists after the stimulus is withdrawn. Synaptic inhibition ensures that the activity does not spread to the rest of the network, and persistent activity has a localized, bell shape (“bump attractor”). At the end of a mnemonic delay period the cue information can be retrieved by reading out the peak location of the persistent activity pattern; and the network is reset back to the resting state. This type of spatial working memory network is endowed with a continuous family of bump attractors, each encoding a specific potential location.

Figure 4.

(A–B) Spiking network model of working memory. (A) Model architecture. Excitatory pyramidal cells are labeled by their preferred cues (0° to 360°). Pyramidal cells of similar preferred cues are connected through local excitatory-to-excitatory connections. Inhibitory interneurons receive inputs from excitatory cells and send feedback inhibition by broad projections. (B) A stimulus is encoded and actively maintained by a self-sustained network persistent activity pattern (a “bump attractor”) in a simulation of the delayed oculomotor experiment. C: cue period D: delay period, R: response period. Pyramidal neurons are labeled along the y-axis according to their preferred cues. The x axis represents time. A dot in the rastergram indicates a spike of a neuron whose preferred location is at y, at time x. An elevated and localized neural activity is triggered by a transient cue stimulus and persists during the delay period. (C) The effects of iontophoretic NMDA blockade on working memory activity in a computational model of working memory. Under control conditions, a stimulus cue selectively activates a group of neurons, leading to persistent activity sustained by NMDAR-dependent recurrent excitation. NMDA conductance is reduced from control to 90%, 80%, and 70% (to bottom) of a reference level in a few pyramidal neurons in the network model. Stimulus-selective persistent activity gradually decreases with more NMDAR blockade and eventually disappears in these affected cells. (D) An example of an individual dorsolateral PFC cell recorded from behaving monkey in a delayed oculomotor response task. Upper panels: control condition, lower panels: after iontophoresis of Ro 25-6981 (25 nA), a blocker of NR2B-containing NMDA receptors. The rasters and histograms show firing patterns for the neuron’s preferred direction and the nonpreferred direction (opposite to the preferred direction). Iontophoresis of Ro 25-6981 markedly reduced mnemonic delay period firing to baseline. (B) was adapted from Compte et al. (2000), (C–D) from Wang et al. (2013), with permission.

In this model, a mnemonic persistent activity pattern is sustained internally by strong recurrent excitation, which the model predicts to be slow and dependent on the NMDA receptor mediated synaptic transmission at local synapses (Wang 1999, 2001) (Figure 4C). In a recent experiment with monkeys performing a working memory task (Wang et al. 2013), iontophoresis of drugs that blocked the NMDA receptors suppressed delay-period persistent activity of PFC (Figure 4D), in support of an important role of the NMDA receptors in PFC processes. Another monkey experiment showed that ketamine (an NMDA receptor antagonist) reduces task selectivity of PFC neurons in parallel with behavioral impairment (Skoblenick and Everling 2012). These findings are directly relevant to psychiatry. Indeed, it has been hypothesized that NMDA hypofunction underlies working memory deficits in schizophrenia (Coyle et al. 2003, Moghaddam and Krystal 2012), and sub-anesthetic dose of ketamine produces working memory impairment in healthy human subjects, similar to that seen in schizophrenia (Krystal et al. 1994). The finding that NMDA receptors are critical for mnemonic persistent activity and its selectivity offers a possible mechanistic explanation as to why NMDA signaling pathway is essential for working memory function.

Like Yin and Yang in Ancient Chinese Philosophy, the dynamic balance between synaptic excitation and inhibition within local and distributed networks is a fundamental property of cortical function. This balance is important for normal functions within a biophysically-based PFC neural circuit model because it defines many emergent properties of the network including: dynamic network stability (because if unchecked by inhibition, strong recurrent excitation would lead to runaway positive feedback), fast coherent oscillations (generated by the interplay between fast AMPA receptor mediated excitation and slower GABA_A receptor mediated inhibition), stimulus-selectivity (synaptic inhibition is critical for neural tuning), and resistance to distractors (reduced responsiveness to distracting stimuli by neurons not involved in memory storage) (Compte et al. 2000, Brunel and Wang 2001, Wang 2013).

These results have functional implications for the observed pathology of inhibitory circuits associated with schizophrenia (Lewis et al. 2005, Lewis et al. 2012). In particular, enhanced distractibility represents a common behavioral deficit in schizophrenic patients (Goldman-Rakic 1987, Mesulam 2000, Luck and Gold 2008). A recent computational study examined how a reduced inhibition might lead to PFC’s deficient ability to filter out distracting stimuli during working memory (Murray et al. 2014). Disinhibition induced a broadening of the neural representation for the memorandum maintained in working memory through persistent activity (Figure 5A). Importantly, this feature of the circuit was a function of the overall balance between excitation and inhibition (Figure 5B). Neural broadening, in turn, induced specific behavioral deficits, making working memory more vulnerable to intervening distractors. In the model, distractibility depends on the similarity between the distractor and the mnemonic representation, and therefore broadening the mnemonic representations increases the range of distractors that can disrupt behavior. The authors tested this model prediction by analyzing behavior from healthy humans administered ketamine, a pharmacological model of schizophrenia, during a spatial delayed match-to-sample task. Matching the model prediction, ketamine increased the rate of errors specifically for distractors that would overlap with a broadened mnemonic representation (Figure 5C). Just as the biophysical basis of the model allows instantiation of potential pathologies, it can also readily incorporate pharmacological treatments to compensate for these deficits. In particular, in this model it was demonstrated as proof-of-principle that glutamatergic or GABAergic manipulations could restore excitation-inhibition balance, reversing the broadened mnemonic representations and corresponding distractibility induced by disinhibition (Figure 5D). An open question is concerned with the brain mechanisms for deciding which information should be considered task-relevant versus distracting and how this may or may not be related to reward value processing of potentially relevant or distracting stimuli. Impairments of this decision process could be relatively independent from those of working memory circuit’s ability to resist distractors as described above, which would suggest an orthogonality between these deficits. Future research is needed to assess whether this is indeed the case.

Figure 5.

Computational modeling of excitation-inhibition (E/I) balance in working memory circuits. (A) A spatial working-memory model can generate a bump-shaped stimulus-selective persistent activity pattern following stimulus withdrawal. Disinhibition, mediated by NMDAR hypofunction on interneurons, broadens working-memory representations at the neural level. (B) The parameter space of NMDAR hypofunction highlights the importance of E/I balance for working memory function. If the E/I ratio is elevated as in disinhibition, the width of the representation increases. In contrast, if the E/I ratio is reduced too much through weakened recurrent excitation between pyramidal cells, the circuit cannot support memory-related persistent activity (upper left corner). (C) Broadening of working-memory representations was tested using behavioral data from human subjects performing a spatial working-memory task combined with ketamine infusion, a pharmacological model of schizophrenia. Consistent with broadening, ketamine induced errors specifically for near distractor probes (left), as predicted by the model (right). (D) Compensations can restore E/I balance and ameliorate behavioral deficits in the model. We paired the disinhibition mechanism with either reduced excitation (purple) or increased inhibition (green), following proposed pharmacological treatments. Adapted with permission from (Murray et al. 2014).

In the model, the network’s ability to filter out distractors is impaired by a reduced excitation in inhibitory neurons. The main insight is that predominant behavorial disturbance due to modest disinhibition may not be so much the inability of memory storage per se as the difficulty of ignoring behaviorally irrelevant inputs during memory maintenance. The observation that ketamine in human subjects leads to impaired resistance against near distractors, as predicted by the model, suggests that disinhibition involves NMDARs. Intuitively, this could be caused by a reduced NMDAR mediated excitation in inhibitory neurons. In support of this view, there is evidence that, in rodents, acute ketamine administration led to a decreased activity of putative fast-spiking (FS) interneurons, and increased activity of putative pyramidal cells (Homayoun and Moghaddam 2007). Moreover, since FS inhibitory neurons are critically involved in the generation of fast γ oscillations (Buzsaki and Wang 2012, Wang 2010), a reduced excitation of those neurons could explain abnormal γ synchrony observed in schizophrenic patients (Spencer et al. 2004, Lisman et al. 2008).

However, in fast-spiking interneurons of the mice frontal cortex, NMDAR mediated excitation is small and insensitive to NMDAR blocker AP5 (Rotaru et al. 2011). In adult rats, the majority of fast-spiking interneurons are devoid of NMDA receptors, whereas NMDAR dependent synaptic excitation is more significant in other subclasses of regular-spiking and low-threshold spiking inhibitory cells (Wang and Gao 2009). The latter mediate dendritic inhibition, thereby gating synaptic inputs onto pyramidal cells. Further, the dendrite-targeting interneurons function in an input-specific manner, enabling pyramidal neurons to be selectively activated by task-relevant inputs. This has been incorporated in an extended working memory microcircuit model endowed with three subtypes of inhibitory neurons: (a) PV-expressing soma-targeting interneurons that control pyramidal firing output, (b) interneurons that express calbindin or somatostatin and gate dendritic inputs to pyramidal cells, (c) interneurons that express calrintinin or VIP and preferentially target dendrite-targeting interneurons (thereby providing a new disinhibition mechanism) (Wang et al. 2004, Wang 2013). It was found that dendritic inhibition controls the network’s ability to resist irrelevant distractors more effectively than peri-somatic inhibition that controls the spiking output of pyramidal neurons. Taken together, one plausible scenario consistent with currently available evidence is that disinhibition induced by ketamine results from a reduction of NMDAR dependent excitation of dendrite-targeting interneurons. This prediction can be tested using cell-type specific genetic tools (Kepecs and Fishell 2014, Higley 2014) in future animal experiments.

What happens when the excitation-inhibition balance is tilted in a way that synaptic excitation becomes excessively strong? Model simulations showed that one consequence of such an imbalance could lead to behavioral inflexibility: attractor states encoding memory items become so robust that it becomes difficult to switch off from one memory attractor state either to rest (memory erasure) or another memory state (Rolls et al. 2008, Durstewitz and Seamans 2008, Gruber et al. 2010). This idea is interesting especially in the light of the fact that working memory is not limited to sensory stimuli but also more abstract information such as behavioral task sets or rules (Miller and Cohen 2001, Wallis et al. 2001, Sakai 2008, Buckley et al. 2009, Lapich et al. 2010, Sigala et al. 2010), and attractor network models have been extended to internal representation of behavioral rule or context in flexible behavior (Rigotti et al. 2010, 2013). Thus, behavioral inflexibility may be reflected in the difficulty to make a transition from a behavioral context to another one, which is a hallmark of abnormal cognition in schizophrenia.

This framework is also useful for analyzing abnormal neuromodulation in mental illness. The dopamine system represents an example par excellence. It is well known that working memory performance exhibits an inverted U-shaped dependence on dopamine modulation: too little dopamine, you loss working memory; too much dopamine, you are inflexible with switching on and off in a working memory system. Dopamine modulation acts on targets such as NMDA receptor mediated excitatory synaptic excitation and GABA mediated inhibitory synaptic inhibition (Brunel and Wang 2001, Seamans et al. 2001, Durstewitz et al. 2000), or the gain of single-neuron input-output relationship (Cohen and Servant-Schreiber 1992). Computational modeling showed that an inverted-U shape of dopamine modulation can be readily explained if dopamine modulation has a differential sensitivity to the NMDA conductance and GABA conductance (Brunel and Wang 2001). Furthermore, interestingly, the network’s ability to ignore distractors is sensitive to modulation by dopamine of recurrent excitation and inhibition. Therefore, even a mild impairment of dopaminergic signaling in the prefrontal cortex could be very detrimental to robust working memory maintenance in spite of ongoing sensory flow.

These studies on working memory demonstrate how biophysically-based modeling in interplay with experimentation can play a powerful role in making discoveries and producing new hypotheses about the brain mechanisms of core cognitive processes implicated in psychiatric disorders.

Looking forward: building a new cross-disciplinary field

The economic cost of mental illness represents an enormous burden on the society Wittchen et al. (2011, Olesen et al. (2012, Vos et al. (2012). The critical nature of our knowledge gap for the clinical neuroscience fields, including neurology, neurosurgery, psychiatry, and psychology, is well known. In the United States, National Institute of Health initiatives including the Human Connectome Project (http://www.humanconnectomeproject.org) and the BRAIN Initiative (http://www.nih.gov/science/brain/) are designed to advance current approaches and to develop new technologies to characterize brain circuit function. Parallel initiatives are underway in Europe and Asia.

In this Perspective, we marshaled findings from recent work on reinforcement learning and working memory to argue for a Computational Psychiatry approach to brain disorders. This perspective emphasizes an integration of experimentation, data analysis and theory in concerted efforts to understand neural circuits involved in mental illness. Although we have focused on local circuit mechanisms, computational psychiatry must also be developed for large brain systems. A notable line of research in this regard is concerned with the interplay between cortex and basal ganglia, which is important for both working memory and decision-making (O’Reilly and Frank 2006, Lo and Wang 2006, Ding and Gold 2013). In fact, behavioral evidence from a cleverly designed experiment suggests that impaired RL in schizophrenia is attributable, largely, to working memory deficits rather than valuation process (Michael Frank, personal communication). Another interplay involves cortex and thamalus (Vukadinovic 2011, Anticevic et al. 2013). More broadly, new approaches applied to the study of the connectivity properties of large-scale brain systems are exciting developments (Sporns 2009, Bullmore and Sporns 2009, Markov et al. 2013) with important implications for psychiatric disorders (Anticevic et al. 2013, Rubinov and Bullmore 2013, Yang et al. 2014).

Unprecedented ongoing progress in neuroscience offers extraordinary opportunities as well as challenges. First, progress in genomics, massive neuroimaging and other advances are creating enormous datasets that, in turn, require new mathematical/statistical tools. Second, there is an increasing recognition that, so far, mechanistic preclinical studies have been almost exclusively focused on local circuits, but we need to develop large-scale brain circuit models in order to investigate how the PFC controls and interacts with many other brain regions in a highly interconnected large system. Third, major mental disorders like schizophrenia, autism qnd ADHD are neurodevelopmental diseases (Moore et al. 2006, Belujon and Grace 2008, Insel 2010, Fair et al 2012). Thus it is critical that computational models incorporate developmental changes in synaptic and circuit function in disease-related models. For instance, the human neural representation of working memory assessed with fMRI changes during adolescence (Satterthwaite et al. 2013). Similarly, synaptic mechanisms evolve during adolescence. In rodents, for example, NMDA receptors are abundant on PV interneurons early in life, but they are present more sparsely in adults (Belforte et al. 2010). In these circuits, reducing NMDA receptor expression early in life, but not in adulthood impairs cognitive function in adulthood. There is a dearth of computational modeling dedicated to understanding critical periods in neurodevelopment and the impact of even “transient” developmental disruption on circuit development and cognitive function in adulthood. Progress along these lines will require sophisticated neural circuit modeling in conjunction with genetic, physiological and imaging experimentation. Fourth and finally, can one quantitatively capture specific features of the normal and dysfunctional flow of thought associated with mental illness? A recent work took the view that language could be used “as a privileged measuring lens into thought”, and showed that quantitative analysis of speech could yield accurate sorting of schizophrenia versus mania with high sensitivity and specificity (Mota et al. 2012). Language is a human cognitive ability implicated in mental disorders, thus elucidation of brain’s language circuit represents another neuroscientific theme relevant to Psychiatry.

It is our belief that these challenges cannot be overcome without theory and computational modeling. To advance the field, we need new infrastructure, resources and training of cross-disciplinary young talents who are well versed both in mathematical modeling and experimentation. In particular, it would be important to develop training programs whereby graduate students and postdoctoral fellows trained in the physical and mathematical sciences could more easily be introduced psychiatry develop without the input of physician-scientists. Third, government funding agencies and non-profit organizations and foundations should offer new programs to promote highly cross-disciplinary education and research in computational psychiatry. Through these concerted efforts, we are optimistic that computational psychiatry could play an indispensable role in addressing the great challenges of mental health in the twenty-first century.