Profile of David Heeger

“Interdisciplinary” has become a buzzword in science in recent years. According to David Heeger, a professor of psychology and neural science at New York University (NYU), “You don’t do interdisciplinary science by putting a physicist and a biologist in the same office. Nor do you make interdisciplinary science happen by taking a grant proposal and assigning it for review to people from a different field who know nothing about it. You have to be willing to learn the other field’s language…techniques…and literature.” Heeger, who was elected to the National Academy of Sciences in 2013, understands interdisciplinary science more than most. For decades, Heeger’s research has straddled the boundaries between psychology, neuroscience, and computer science, with lasting impacts on all three fields.

David Heeger. Image courtesy of the National Academy of Sciences.

Early Opportunities

David Heeger was born into science. His father, Alan Heeger, is a physicist whose discovery of conductive polymers earned him the Nobel Prize for Chemistry in 2000. In addition, his brother, Peter, is a transplant immunologist at the Icahn School of Medicine at Mount Sinai in New York. Heeger believes that his father’s work at the interface between physics and chemistry influenced his own interest in working across scientific disciplines. He was thrilled when his father won the Nobel Prize, recalling that “[my father] called and woke me up at 6:00 that morning, and I woke up my family screaming!”

As an undergraduate, Heeger attended the University of Pennsylvania, mainly because his father was on the faculty at the time, and tuition was free. It was during his senior year at Penn that he was first exposed to the field of vision science. At the time, there was a thriving vision research group at Penn, which included faculty from engineering, psychology, and the medical school. Heeger would sit in on the group’s interdisciplinary seminar series, learning how these seemingly disparate fields intersected in studying vision, how brain function could be understood in terms of information and signal processing, and how ideas from perceptual psychology could be used to develop computer programs for image and video analysis. The interdisciplinary nature of vision research formed the basis of Heeger’s subsequent research interests. While still an undergraduate, Heeger participated in his first research project, a collaboration between computer scientist Ruzena Bajcsy and psychologist Jacob Nachmias.

After graduating with a bachelor’s degree in mathematics in 1983, Heeger decided to continue studying with Bajcsy, earning his master’s degree in 1985 and his PhD in 1987, both in computer science. As a thesis advisor, Bajcsy was generous, encouraging Heeger to publish his dissertation work as sole author and providing him with valuable professional contacts, including Alexander Pentland, who coadvised Heeger on some of his dissertation work, and Edward Adelson, who would later become Heeger’s postdoctoral advisor at the renowned Massachusetts Institute of Technology (MIT) Media Lab. For his dissertation research, Heeger developed algorithms for extracting velocity from an image sequence or video, inspired by the then-current understanding of how the brain accomplishes the same task. One of the articles describing this research was awarded the first biennial David Marr Prize from the International Conference on Computer Vision in 1987 (1).

The Normalization Model

As a postdoctoral fellow at MIT, Heeger continued his work on image processing. He also began working on computational neuroscience—understanding brain function in terms of information processing. This work would eventually lead to the development of Heeger’s “normalization model” of neural computation.

To illustrate how the normalization model works, imagine a neuron in the brain’s primary visual cortex that responds to images with a vertical edge but not to images with a horizontal edge. An image containing both horizontal and vertical edges will produce a weaker response in this neuron than one with a vertical edge alone, a phenomenon known as cross-orientation suppression. According to the normalization model, this happens because a second neuron sensitive to horizontal edges responds at the same time, and the output of each neuron is divided by the combined response of both. This division, analogous to normalizing a vector in mathematics, acts as a type of automatic gain control, preventing the combined signal from getting too large and overloading the neural circuit.

Heeger’s original paper on the normalization model (2), published in 1992, has been cited more than 1,400 times. In subsequent work, Heeger and his collaborators demonstrated quantitative agreement between experimental data and the predictions of the normalization model. Meanwhile, other researchers have found that the model can explain neuronal function in a wide variety of neural systems and species, from decision-making in primates to olfaction in fruit flies. Of the model, Heeger says, “I continue to be surprised at the impact that it’s had, and how far this one simple idea can go.”

Stanford and fMRI

In 1990, after finishing his postdoctoral fellowship, Heeger joined the vision research group at the Ames Research Center of the National Aeronautics and Space Administration (NASA) in California, where he had become acquainted with a group of researchers during a vision science conference. At the time, this group focused on understanding aspects of vision that applied to aircraft pilots, as part of NASA’s aeronautics missions. Heeger spent more than a year at NASA before taking a faculty job at nearby Stanford University, where he would spend the next decade.

It was around the time Heeger moved to Stanford that the first articles were published on using magnetic resonance imaging (MRI) to measure brain activity. At the same time, the Stanford Medical School opened an MRI research center. Heeger recalls that the director of the new center e-mailed the Stanford faculty asking if there was anything the center could do for them. Heeger’s colleague Brian Wandell replied, asking if they could conduct fMRI experiments. “The next thing I know,” says Heeger, “I’m in Brian’s garage on a weekend, sawing a bunch of two-by-fours to build a projection screen, so that we can show visual images to people while they’re in the MRI scanner.” For Heeger, fMRI research provided yet another opportunity to combine perceptual psychology and neuroscience with image processing and computer vision, because image-processing algorithms had to be developed to analyze the data produced by the fMRI measurements. Heeger’s first fMRI study (3), published in 1996, has been cited nearly 2,000 times.

Heeger emphasizes that his use of fMRI is different from most others’. Most neuroimaging studies focus on determining which parts of the brain are involved in a particular task. However, the seat of visual processing in the brain is already known. Instead, Heeger measures how the activity in previously identified brain regions changes during different tasks and in response to different stimuli. For example, Heeger and his colleagues discovered waves of activity that travel across the surface of the brain coincident with the perception of a visual illusion called binocular rivalry and measured the speed of those waves (4). Heeger and his team have used fMRI to test theories of how the brain visually perceives color, motion, pattern, and texture, and have contributed to research on dyslexia and autism.

Attention, Processing Timescales, and Cortical Function

In 2002, Heeger left Stanford and moved to NYU. “I think it was a good time in my career to make a move,” Heeger says. NYU was home to a large interdisciplinary group of researchers in psychology, neuroscience, and computer science, some of whom Heeger had known and collaborated with since he was a graduate student. Heeger relished the opportunity to join this group. While at NYU, Heeger incorporated normalization into a computational model that describes how changes in visual attention modulate the responses of neurons in the visual cortex (5). This model could reconcile a variety of seemingly conflicting empirical results.

Heeger continues to explore new ways to study the brain. One of his recent projects involves using Hollywood movies as stimuli to drive simultaneous activity in multiple brain areas in a highly controlled manner. This idea came from a postdoctoral associate in Heeger’s laboratory at the time, Uri Hasson, who showed that movies elicit similar activity patterns in the brains of different people. Or as Heeger puts it, “Hollywood is better at taking control over your brain than any cognitive psychologist has ever been.” By breaking films into segments of varying length and scrambling the order of the segments, Hasson, Heeger, and their collaborators identified a hierarchy of brain areas that process information at different timescales (6).

Heeger’s Inaugural Article (7) exemplifies the interdisciplinary nature of his research interests, incorporating aspects of signal processing, neuroscience, perceptual psychology, computer vision, and artificial intelligence. In the article, Heeger proposes a new computational model with far-reaching goals: to explain how the brain integrates sensory input with prior knowledge or expectations. As Heeger explains, “I can instruct you to imagine Woody Allen, or Bill Clinton’s face, or what your living room looks like. And when you do that, there’s activity in the visual part of your brain that looks pretty much like you were actually looking at the picture. This means that visual perception and visual imagery are linked, accomplished by the same circuits and processing in the brain.” Furthermore, the response of neurons in the visual cortex to an image is always delayed by 50–200 milliseconds. “So how can you catch a baseball if the perceptual processing is a quarter of a second delayed, and then it takes another quarter of a second to program the movement?” Heeger asks. “You’ll reach out to catch it after it’s already passed by.” In Heeger’s model, as sensory information is processed through layers of neurons, each layer provides feedback to the previous layer based on prior expectations. Thus, the brain can use the same type of computations to perceive external images and generate images from memory by varying the weight given to prior knowledge. Incorporating prior knowledge also allows prediction.

Heeger sees the Inaugural Article as an attempt to develop a unified theory of cortical function—an empirically testable theoretical framework for guiding both neuroscience research and the design of machine learning algorithms with artificial neural networks. Like the normalization model, Heeger describes the theory as a pretty simple idea that does some interesting things. “We’ll see how far it goes,” he says.