Neuroimaging of Cognition

Abstract

Neuroimaging, particularly that based upon functional magnetic resonance (fMRI), has become a dominant tool in cognitive neuroscience. This review provides a personal and selective perspective on its past, present, and future. Two trends currently characterize the field that broadly reflect a pursuit of “where”- and “how”-type questions. The latter addresses basic mechanisms related to the expression of task-induced neural activity and is likely to be an increasingly important theme in the future. This trend entails an enhanced symbiosis among investigators pursuing similar questions in fields such as computational and theoretical neuroscience as well as through the detailed analysis of microcircuitry.

A Beginning in Neuroimaging

In the late 1970s, I commenced training as a psychiatric resident at a large mental institution in North London’s Friern Barnet Hospital, uncertain how my future would unfold. This crumbling Victorian institution, which gave rise to the cockney colloquialism for a madhouse, “Colney Hatch” was famously, and sympathetically, depicted in Richard Hunter’s book “Psychiatry for the Poor” (Hunter and MacAlpine, 1973). The very same author, a maverick psychiatrist, used to conduct a weekly postmortem brain dissection demonstration on his deceased patients. At the time, within mainstream psychiatry, the mere idea of studying the human brain as a means to unravel the mysterious nature of psychiatric illness was viewed as arcane and treated with derision. Psychiatry was then a discipline paralyzed by a pervasive intellectual agnosia when challenged as to the likely causes of severe mental disorder.

I found Hunter’s weekly brain dissection demonstrations fascinating and frustrating. Although the patients coming to postmortem had suffered life-long severe mental illness, it was rare for us to detect any macroscopic pathology. Indeed, I cannot recall a single instance, despite the proselytizing zeal of our demonstrator, where a convincing clinicopathological correlation could be established. As part of the weekly ritual, and no doubt to reinforce our attendance, we were served tea and chocolate biscuits, which provided the context for idle conversation among my fellow psychiatric residents. We often, subversively given the context, speculated that the only conceivable avenue for progress was not through brain dissection, however fine grained, but by dint of some future technological innovation. What we often imagined was a dissecting device for the living, one that would enable the physiological function of the brain to be revealed in its entire splendor. Less than 30 years later, I need only to remind myself of this time period to fully appreciate the extraordinary advances that have ensued in the interim, developments that have delivered the sophisticated technology of functional brain imaging.

By 1984, I had completed my first stint as a researcher, thanks to a generous training fellowship from the Wellcome Trust. At this time, there was growing excitement about a technology that seemed to presage a conversion of our postmortem dissection table fantasies into reality. A number of centers around the world were beginning to use positron emission tomography (PET) to measure regional cerebral metabolism using radiolabeled glucose or regional cerebral blood flow (rCBF) using radiolabeled oxygen. The scientific outputs from these centers were primarily resting-state investigations, where the emphasis was on physiological quantification of blood flow and metabolism. Yet, for those of us who took an interest, we could immediately intuit its future possibilities. Within the space of a few years, a number of these centers were extending resting-state applications of PET, capitalizing on the potential of perfusion techniques to measure task-induced brain activity. In hindsight, these techniques appear crude and unsophisticated, but at the time they were of enormous importance in realizing a dream within neuroscience, namely the prospect of a physiological window into the human mind.

I am certain that the excitement and expectation generated by these developments was the principal motivating factor for the many young scientists who committed to the field and were happy to be labeled by their colleagues, often mockingly, as “imagers.” The intellectual promise offered by what I perceived as a revolutionary means to study the brain was certainly sufficient motivation for me to return to research once I completed my psychiatric training. In 1989, I arrived at the MRC Cyclotron Unit at the Hammersmith Hospital in London where I and my close colleagues over the next 20 years, Karl Friston and Chris Frith, joined a team led by Richard Frackowiak. We committed, as did a number of other pioneering groups worldwide, most notably the group of Marcus Raichle in St Louis, to an ambitious program to use noninvasive imaging to map the functional architecture of the living human brain. What follows in this review is a highly selective perspective on how this field has evolved, its current and future directions. As a consequence, and because of a remit that includes a word restriction, the review makes no claim to comprehensiveness.

A Revolution in Studying the Living Human Brain

To measure brain activity associated with discrete states of mind is the holy grail of cognitive neuroscience. In the late 1980s and early 1990s, there were two significant developments, resting on nascent progress in perfusion imaging, that proved pivotal in realizing this dream. The first was the development of sophisticated data analysis tools, subsequently becoming a standard in the field, which provided whole brain imaging analysis in the form of statistical parametric maps (Fox et al., 1988; Friston et al., 1991). Second, the early 1990s saw the surprising and rapid emergence of a new approach to brain mapping using MRI based upon the blood-oxygen-level-dependent (BOLD) technique (Ogawa et al., 1990). This latter approach had significant benefits over PET, including greater spatial resolution and the absence of the potential hazards associated with radioactive tracers. The significance of this development derives from the fact that neuroscience now had a technology that was noninvasive and that afforded multiple repeated measures of brain activity across various task manipulations. The end result was fMRI, a technique that afforded much greater experimental flexibility.

Apart from obvious methodological advantages associated with fMRI, it had, in my view, an equally important cultural consequence. Access to PET as a technique, entailing by necessity the use of ionizing radiation, was vested in the hands of an elite. Their privileged position was predicated on either having nuclear medicine expertise, a medical license, or both. The fact that fMRI involved no inherent biohazard meant that it could now be deployed within less restricted environments, for example, within psychology departments of universities. In essence, fMRI democratized access to a powerful technology for investigating the living human brain, allowing a broad cross-section of academic disciplines to pursue new agendas. This, in my view, accounts for a virtual exponential rise in published neuroimaging output that, over the past 15 years, has been largely led by questions related to human cognition. Most people would agree that democracy is a good thing; equally, most people would agree that all things taken in excess should carry a health warning.

A Selective History of Imaging Human Cognition

The first applications of activation-based neuroimaging in the late 1980s involved the use of PET and utilized simple subtraction techniques based upon presenting alternating blocks of stimuli. In essence, this approach involved measuring brain activity in a condition of interest in one block and then subtracting activity associated with a carefully specified control condition acquired in a second block. This procedure was predicated on the idea of pure insertion of sequential cognitive processes and allowed, in principle, localization of discrete cognitive functions (Friston et al., 1996). Early examples of this methodology included identifying brain regions that process single words (Petersen et al., 1988; Posner et al., 1988) and the human homolog of V4 (Lueck et al., 1989) and V5 (Zeki et al., 1991), areas specialized for visual color and motion processing, respectively. These so-called “block designs” also characterize early fMRI studies. Where PET measurements necessarily required long data collection sequences, such temporal inflexibility was overcome using fMRI.

In the first instance, more flexible thinking on the part of experimental design provided new, more rigorous approaches that involved the implementation of parametric (Grafton et al., 1992) and factorial experimental designs (Friston et al., 1992). These provided a higher degree of experimental control and inference. Such developments coincided with a new class of questions, such as “how does the presence or absence of selective attention influence brain activity associated with processing some other factor?” (O’Craven et al., 1997) In addition, this increasing sophistication in factorial methodologies provided the platform for in vivo pharmacological challenges in conjunction with cognitive activation paradigms, enabling a characterization of how neuromodulatory influences impact discrete cognitive processes in health and disease (Dolan et al., 1995).

At the end of the 1990s, experimenters began to harness fMRI to its full potential. This was initiated by the implementation of event-related fMRI, a procedure akin to measuring evoked responses in electrophysiology, and obviated the shortcomings of the less flexible blocked designs. Researchers were now able to capitalize on known physiological properties of neurons, such as their susceptibility to adaptation with repetition of an input, and, using this knowledge, to enhance both spatial and functional sensitivity. This latter approach, akin to repetition suppression, allowed detection of representations that might be instantiated within topographically overlapping neuronal networks. This new approach was applied to both low-level and high-level stimulus attributes (Tootell et al., 1998: Grill-Spector and Malach, 2001; Winston, et al., 2004).

The 1990s was a decade that witnessed an explosion in applications of neuroimaging that generated data on virtually every conceivable aspect of human cognition, including sensori-motor learning (Karni etal., 1998), attention(Kastneretal., 1998), short-(Courtney et al., 1998; Jonides et al., 1993; Owen et al., 1996b) and long-term memory (Fletcher et al., 1995; Schacter and Buckner, 1998; Shallice et al., 1994; Wagner et al., 1998), perception (Haxby et al., 1991, 1996), language (Howard et al., 1992; Petersen et al., 1990; Price et al., 1992, 1997), and executive functions (Baker et al., 1996; Grasby et al., 1993; Owen et al., 1 996a). While many findings recapitulated what would be predicted from lesion based models, it also turned out there were many surprising findings. For example, in the case of episodic memory, it turned out that performance of relatively simple tasks engaged regions of the brain that would not have been predicted from lesion models (Petrides et al., 1995; Tulving et al., 1994a, 1994b). This reflected, in part, the fact that attribution of function, based upon a lesion-deficit model, rested on an assumption that the consequences of a lesion solely reflected disruption of function in the damaged region. However, sophisticated perspectives on the impact of a lesion account for the fact that cognition arises not just out of functional differentiation but also out of functional integration. Consequently, the impact of a localized lesion is likely to reflect both local and distributed effects, evidenced by more widespread activations seen with neuroimaging than was predicted by a lesion-deficit model.

One far-reaching impact of neuroimaging has been the extent to which investigators were now empowered to tackle issues and questions that were not reflected in classical psychological parsing of the mind, as might be found in standard textbooks of psychology. In this regard, the impact of neuroimaging has arguably been greatest in relation to topics that were either ignored in neuropsychology or were heretofore difficult to tackle experimentally. Two pertinent examples involve the growth of studies that address the biological underpinnings of consciousness and emotion, respectively.

In studies of consciousness, neuroimaging provided the means to address disparate questions inconceivable without this new technology. These include how the exercise of “free will” engages discrete circuits (Frith and Dolan, 1996), the fate of unseen but psychologically effective stimuli (Dehaene et al., 2001; Dehaene et al., 1998), the correlates of subjective perception (Lumer et al., 1998; Lumer and Rees, 1999), imagined future states (Sharot et al., 2007), as well as the level of residual neuronal function seen in patients with varying impairments of consciousness (Owen, 2008), to name a few. In the field of emotion, functional neuroimaging allowed a characterization of how the brain responds to external emotional stimuli (Morris et al., 1996; Whalen et al., 1998), the expression of emotional learning and extinction(Bucheletal., 1998; LaBaretal., 1998), and how emotion impacts on perception (Noesselt et al., 2005; Vuilleumier et al., 2004) and memory (Kensinger and Schacter, 2006). Even more importantly, the subjective nature of emotional measures were overcome with direct investigation, where studies characterized the neural basis of distinct feeling states (Damasio et al., 2000) and enabled a description of how interoceptive states, for example, changes in peripheral autonomic status, are mapped in the brain (Critchley et al., 2004). Interestingly, the representation of interoceptive states in anterior insular cortex was subsequently shown to also provide a substrate that enables a person to represent the subjective feeling states of others as, for example, expressed in empathy for pain (Singer et al., 2004).

Less than 20 years since its inception, the field of neuroimaging, using fMRI, has reached a high level of maturity and methodological sophistication. Neuroimaging now attracts interest not only from cognitive neuroscientists but also from a wide array of fields that lay outside those that might be thought to be concerned about the brain. Consequently, we have witnessed the emergence of a strong symbiosis between functional neuoroimaging and a range of other disciplines, in many instances constituting unlikely bedfellows, including genetics (Drabant et al., 2006; Hariri et al., 2002), economics (Camerer, 2003; de Quervain et al., 2004; King-Casas et al., 2005), ethics (Hsu et al., 2008), and aesthetics (Winston et al., 2007), to name a few.

Current Perspectives within Neuroimaging

A repeatedly asked question over the past 20 years is what brain imaging has brought to neuroscience that was not already known. First, functional imaging realized what was never previously possible: namely, a characterization of the functional anatomy of the intact brain without the confound of pathology and the likely consequential plastic reorganization in response to disease or developmental abnormalities. Second, functional neuroimaging highlighted that even simple tasks engaged more widespread areas of the brain than would have been assumed from the lesion-deficit approach. This has led to a richer conceptualization of how brain function underpins cognition not only in terms of functional differentiation (localization) but also in terms of functional integration (distributed function). This latter characterization has motivated a new class of questions and methodological approaches that address how distributed brain regions interact during performance of a psychological task, as characterized in terms of functional connectivity (Fletcher et al., 1996) (which assess a significant dependence or mutual information between regional activities in different parts of the brain) and effective connectivity (the causal influence one regions exerts over another) (Buchel and Friston, 2001; Friston, 1994; McIntosh and Gonzalez-Lima, 1994). Furthermore, these developments have now furnished investigators with the tools to ask questions related to how brain regions comprising a distributed network interact within that network after some experimental manipulation: for example, the presence or absence of attention or the evolution of some form of learning (Plailly et al., 2008).

Neuroimaging has now consolidated its position as the major platform for a systems level understanding of the human brain. Currently within the field, there appears to be two broad conceptual trends. On the one hand, there is a “Kuhnian” culture of normative and largely descriptive imaging neuroscience using standard and well-established approaches based upon classical experimental manipulations and data analysis. This constitutes the majority of the field and is a testament to its scientific maturity. This contrasts with emerging approaches that are more embedded in theoretical and computational neuroscience and largely concerned with a broader and more mechanistic set of questions that speak to fundamental questions regarding how the brain works. Here, neuroimaging is seeing a striking convergence with expertise from other fields, such as engineering, dynamic programming, and computational neuroscience.

One recent development that is more allied with the former descriptive approach has been the use of multivariate pattern classification procedures. An example is the use of support vector machines (SVMs) to establish statistical dependence between distributed responses in a circumscribed part of the brain and some experimental variable. These approaches have excited much media attention and have been in the vanguard of claims related to mindreading or predicting future intentions. In simple terms, these procedures involve training a classifier with brain activity obtained during an experimental manipulation: for example, being presented with a house or a face. Then, in a subsequent phase of the experiment, conducted in the same subjects, the trained classifier can provide a reliable prediction of the sensory input (was it a house or a face?) (Kamitani and Tong, 2005). From a scientific perspective, these approaches have been claimed to be more sensitive than equivalent univariate analyses (Haynes and Rees, 2006). What emerges in most multivariate classification applications is a probabilistic mapping between activity in a subset of brain voxels and some aspect of the sensorium or behavior. Criticisms of these approaches include a suggestion that they represent a return to neophrenology and an absence of clarity regarding what new neurobiological questions are specifically amenable to these classifiers. Since these classification techniques are extremely sensitive to subject and paradigm specifics, the precise nature of what these signals represent is at best difficult to pinpoint. Newer multivariate approaches go some way to addressing these concerns, for example, the use of decoding based upon quantitative receptive-field models that have allowed people to decode natural images from an observer’s brain activity (Kay et al., 2008).

One exciting, and less contentious, development that reflects a convergence with theoretical neuroscience has been the implementation of experimental paradigms motivated by computational rather than psychological models. This approach owes much to the impact of the late David Marr, who argued that a sine qua non for understanding the human brain, or any system within the brain, involves specifying a computational goal (Marr, 1982). Such a perspective has two immediate consequences in terms of empirical research. First, it speaks to defining the algorithmic nature of the transformations between input and output and, second, specifying how this is implemented in the hardware of the brain (Marr, 1982). The application of this approach has involved harvesting the outputs from a computational model that provides a formal description of a transformation between a sensory input and behavioral output. For neuroimaging, the key variables are those proposed to mediate this transformation, an example being a prediction error signal underpinning reward-based or punishment-based learning. These variables are then correlated against fMRI data to determine brain regions manifesting a response profile consistent with the model variables. The key advantage of these model-based approaches derives from the fact that they address how a particular cognitive process is implemented in a specific brain area as opposed to identifying the location of its instantiation. An early example has been the implementation of temporal difference (TD) learning models, where the striking finding was that a TD-like prediction error signal was expressed in the ventral striatum during reward learning (McClure et al., 2003; O’Doherty et al., 2003). Model-based approaches also enable specific inferences regarding underlying neuromodulatory influences. For example, the likely dopaminergic nature of a reward prediction error signal seen in reward-based neuroimaging experiments was demonstrated in an investigation that used a computational learning model, Q-learning, in conjunction with an in vivo pharmacological manipulation of dopamine function (Pessiglione et al., 2006). This general approach has opened up a very rich field of investigation, particularly in the context of human decision making, that has seen a characterization of neuronal responses associated with exploratory and exploitative choice behavior (Daw et al., 2006), reward predictability and uncertainty (Berns et al., 2001; Critchley et al., 2001), intertemporal choice (McClure et al., 2004), and subjective utility (Kable and Glimcher, 2007).

In line with these developments, neuroimaging is also witnessing a more mechanistic emphasis that unambiguously reflects a shift within the field from “where-”to “how-”type questions. This emphasis parallels the development of generative modeling of brain responses using, for example, Dynamic Causal Modeling (DCM) (David et al., 2006; Stephan et al., 2008; Friston et al., 2003). In DCM, the data (be it fMRI, EEG, or MEG) is viewed as being caused by a perturbation of hidden neuronal states by some experimental input. These perturbations produce neuronal dynamics through neuronal interactions that are passed through biophysically motivated forward models to form the observed responses (for example, the BOLD response in fMRI). Inversion of these models furnishes conditional densities (e.g., conditional mean and precision) on the parameters of the underlying neuronal and hemodynamic models and their marginal likelihood for model comparison. The identification or inversion of generative models, given some data, enables conditional inferences about the parameters of these models and, critically, comparison of different generative models (Penny et al., 2004). The term “model comparison” may be erroneously understood as an artificial delineation of parameter space but, in fact, simply embodies hypothesis testing where these competing hypotheses can be framed in terms of competing generative models of the same data.

This aforementioned approach contrasts with standard techniques that rely solely on establishing statistical dependencies or probabilistic mappings between behavioral and physiological data or between physiological data acquired from different parts of the brain, which provide no machinery for model comparison beyond the existence of that mapping. Recently, attempts have been made to marry the different modeling approaches described above and create models that jointly represent computational and physiological mechanisms of neuronal systems. For example, predictive coding models of perception can be formulated in terms of hierarchically structured dynamic causal models (Friston, 2005). Alternatively, computational models of learning and decision-making can be embedded into dynamic causal models; this allows one to investigate how computational processes are reflected by time-dependent changes in specific connection strengths or neuronal state variables. An example is a recent study that integrated a Rescorla-Wagner learning model into a dynamic causal model, demonstrating that prediction errors drive the consolidation of connection strengths during associative learning (den Ouden et al., 2008).

Functional neuroimaging has been heralded as a technology that will impact upon our understanding of nervous system diseases. A sanguine view is that its impact has been, at best, modest. In neurology, for example, it has shed light on mechanisms in recovery of function following stroke (Ward et al., 2004, 2006). Classificatory approaches, using support vector machines (SVMs), show considerable promise in providing a classification of likely neurodegenerative process, based upon structural brain appearance, that may surpass the accuracy of neuroradiological experts (Kloppel et al., 2008). Paradoxically, it is in the problematic field of psychiatry that a so-called “translation” potential holds the greatest prospect of realization. There is a growing theoretical and empirical sophistication within cognitive neuroscience related to understanding emotional learning and extinction, motivation, and reward. The likelihood that this sophistication can provide both a more principled classification of psychiatric disorders and a high level specification of aberrant cognitive processes should not be underestimated. In the case of schizophrenia, the ability to characterize neuronal interactions using approaches such as DCM, including how neuromodulatory influences alter these interactions (Corlett et al., 2007; Honey et al., 2008; Murray et al., 2008), means that theoretical models of this disease, based upon the idea that the disorder reflects a failure of integration, can now be tested empirically.

Macro- and Microscopic Mapping of the Brain

Current general theories of the brain invoke the idea that a core function is to infer the causes of sensory input. Almost universally, such theories call upon hierarchical network models of how sensory data are generated. The relevance of systems neuroscience arises out of the fact that hierarchical optimisation in the brain is distributed, entailing recurrent self-organized interactions among different levels of sensory and motor hierarchies. At the systems level, brain imaging using fMRI (as well as EEG and MEG) provides for a precise delineation of these functional architectures. As described, one can invert biologically realistic models with a large number of state variables and biophysical parameters, using state-of-the-art variational Bayesian techniques, with the inherent ability to quantify the uncertainty about any experimentally induced changes (Friston et al., 2007). Inversion of these models allows one to make inferences about neuronal processes that cannot be observed directly with available noninvasive techniques, for example, sensitization of glutamatergic postsynaptic responses or changes in neuronal spike frequency adaptation.

Ongoing validation studies in rodents have demonstrated the potential of this approach (Moran et al., 2008) and highlight a more general imperative to connect micro- and macroneurobiological scales by marrying and cross-validating techniques in animal and human neuroimaging. The increasing availability of intracranial recordings from patients provides a further bridge to validate noninvasive models and test the resolution of our predictions. In other words, one can optimize the level of modeling and assess, in a quantitative fashion, whether a particular imaging modality is sufficient to answer questions framed at a particular level of neurobiological model. However, future progress here is almost certainly going to benefit from much closer interactions between imaging neuroscience and scientists pursuing a microscopic characterization of underlying circuits. This knowledge is necessary to provide empirical constraints on microcircuitry and relative connection strengths that can then be transcribed directly into current models of EEG, MEG, or fMRI data, in essence informing macroscopic characterizations of brain function with microscopic observations of brain structure.

Conclusions and a Personal Coda

The dominant position of functional neuroimaging in systems neuroscience is unlikely to change in the immediate future. This reflects not only the absence of any emerging competition to existing technologies but also the necessity for system level descriptions of cognitive processing, no matter how one conceives of how the brain works. However, an increasing discourse between imaging neuroscience and experts from disciplines such as computational and theoretical neuroscience, as well as those working at the level of microcircuitry, seems inevitable. The largely untapped potential of functional neuroimaging to provide a richer characterization of aberrant processes in psychiatric disorders is likely to be of increasing importance. The more far-fetched claims made in relation to neuorimaging, evoking an Orwellian nightmare, are unlikely to threaten our human attachment to privacy. The reality is that deterministic predictions on behavior need to contend with the fact that even simple neuronal systems, and their expression in behavior, embody indeterminacy. Indeed, this indeterminancy is introduced by the brain itself, where it is suggested that it provides for a variance that optimizes long-term survival (Carpenter, 2004).

It is now nearly 20 years since I agreed to a career in imaging neuroscience, a period that coincides with the publication history of Neuron, and I find myself caught in an ironic twist of fate. Iam sitting in awaiting room reflecting on having committed, at extremely short notice, to writing this opinion piece while preparing myself for an MRI scan on an increasingly painful right shoulder. I keep a rumbling anxiety in the background by an old trick of distraction, in this case realized by aimlessly flicking through an assortment of well-fingered magazines. Almost inevitably, I stumble across a magazine article that is a feature on brain imaging, entitled “The brain hacker,” with an equally attention grabbing subtitle, “they can read your mind.” The opening paragraph is indeed ominous and enough to make any mortal shudder at the prospect of submitting to any procedure suffixed with MRI, even ostensibly on a relatively unsophisticated piece of shoulder anatomy. Who can relax when reading: “Machines that read your thoughts have long been the stuff of science fiction. But now brain-scanning techniques are being used to see if you prefer Coke to Pepsi. Are secrets a thing of the past?” Reading on, I learn that the application of brain imaging is all-embracing when it comes to the human condition, and it can now unravel political preferences (neuropolitics), the likely choice of goods to purchase (neuromarketing), unmask covert racism, and distinguish truth from lies and even true love from mere lust. I momentarily remind myself, “It’s only your shoulder.” As I ease into the engulfing claustrophobia that is the bore of an MRI magnet, I reassure myself, without undue effort, that my personal secrets are safe and that the reality of brain imaging is more prosaic than the ubiquitous hype of newspapers and magazines.