Connecting Art and the Brain: An Artist's Perspective on Visual Indeterminacy

Abstract

In this article I will discuss the intersection between art and neuroscience from the perspective of a practicing artist. I have collaborated on several scientific studies into the effects of art on the brain and behavior, looking in particular at the phenomenon of “visual indeterminacy.” This is a perceptual state in which subjects fail to recognize objects from visual cues. I will look at the background to this phenomenon, and show how various artists have exploited its effect through the history of art. My own attempts to create indeterminate images will be discussed, including some of the technical problems I faced in trying to manipulate the viewer's perceptual state through paintings. Visual indeterminacy is not widely studied in neuroscience, although references to it can be found in the literature on visual agnosia and object recognition. I will briefly review some of this work and show how my attempts to understand the science behind visual indeterminacy led me to collaborate with psychophysicists and neuroscientists. After reviewing this work, I will discuss the conclusions I have drawn from its findings and consider the problem of how best to integrate neuroscientific methods with artistic knowledge to create truly interdisciplinary approach.

Introduction

The last decade or so has seen substantial interdisciplinary activity between the arts and sciences, with many scientists applying knowledge and methods from their own areas in order to gain new insights into how art is made and appreciated (Journal of Consciousness Studies, 1999, 2000, 2004; Zeki, 1999; Livingstone, 2002; Solso, 2003; Martindale, 2006). Occasionally scientists have worked closely with art historians to share ideas and approaches (Freedberg and Gallese, 2007; Onians, 2008). One of the factors motivating this new collaborative spirit is the realization that artists have made certain discoveries about the way the human brain works that are only now being uncovered by scientists. According to Zeki (1999, p. 2): “…most painters are also neurologists.” Cavanagh (2005), another eminent vision researcher, talked of “the artist as neuroscientist.” Given that for many centuries artists have been intensively studying the way the world is perceived it is perhaps not surprising they have come to understand certain features of the way, for example, we sense objects, color, form, or depth. Through their investigations artists have left a permanent record of their findings in all the countless works of art in museums and galleries around the world. The task of unpacking all this deposited artistic knowledge and reconciling it with our current scientific understanding of perception and cognition is vast. Which is why the recent collaborative activity is to be welcomed, despite the great inter-cultural and methodological challenges it poses. The challenges will be addressed later in this paper. First, though, I will discuss my own experience as an artist collaborating with scientists in the study of art. As will be seen, the process of working closely with scientists has raised a number of issues of the kind I believe many collaborators face when trying to work across two quite distinct traditions. As a result of this work I remain convinced not only that artists and scientists can work together effectively to create new, mutually relevant, knowledge but that it is very important they do.

The paper will start by introducing the topic that was the subject of the collaborative investigations, which I term “visual indeterminacy.” This is an area of great significance in the history of art and, as I hope to show, is potentially of considerable neuroscientific interest also. In setting out the background, I will describe how I became aware of the phenomenon of visual indeterminacy, how other artists have explored its effects, and how I have tried in my own work to produce images that are indeterminate. I will then describe some of the neuroscience that relates to the phenomenon, and the collaborative studies I have undertaken with neuroscientists and psychophysicists to investigate responses to such images. Finally, I will consider some of the implications of this work for art–science interdisciplinarity in general.

The Perceptual Phenomenon of Visual Indeterminacy

Visual indeterminacy is a perceptual phenomenon that occurs when a viewer is presented with a seemingly meaningful visual stimulus that denies easy or immediate identification (Pepperell, 2006). I first became aware of it as an undergraduate art student watching the silent German Expressionist film The Cabinet of Dr Caligari (Weine, 1920), which is known for its non-naturalistic sets and highly contrasting monochromatic lighting. About three quarters of the way through watching the film something remarkable happened: the image suddenly became unrecognizable. Although I could clearly see the screen was full of shapes (there was no problem with my vision as far as I was aware) they did not form a meaningful scene, and I was left struggling to identify the forms before me. Figure 1 shows two stills from the movie, the first at the moment of “non-recognition” and the second from a few frames on, when a figure leans up from a desk, and I was once again able to make out what was depicted.

Figure 1.

Two stills from The Cabinet of Dr Caligari showing (left) the moment of non-recognition and (right) the moment of recognition some 5 s later.

This 5-s sequence had a big impact on me, with repercussions that continue to this day. Unlike normal visual perception where the world is full of objects we readily recognize, in this short lapse of time my usual conceptual grip on the world failed. I remember the experience as marked by a mild form of anxiety and bewilderment combined with an active struggle to make sense of what I was seeing. I've since realized that such experiences are not uncommon. Indeed, I've spoken to many others who report similar momentary lapses of recognition. I have certainly been aware of it in my own perception many times since.

One of the most vivid televisual memories of my childhood was a segment in an early evening quiz show called Ask the Family, broadcast in the UK in the 1970s, which pitted two families against each other in a test of general knowledge and observation. The section of the show in question involved an everyday object being presented in close up or from an unusual angle. As the camera pulled out to reveal the object in full the families raced to identify it as quickly as possible. Part of the reason, I suspect, this piece of television trivia is remembered so readily by those who saw it is because it was one of the rare occasions in popular culture where an image was deliberately presented in such as way as to be unrecognizable. When faced with such images we seem to be compelled to determine their meaning, so paying a different kind of attention to them than we would with easily recognizable views of the same thing. These days we might even enroll the help of the online community to resolve visual conundrums of this kind. The image in Figure 2 was an image posted on a university bulletin board by a confused IT manager who wanted help in identifying what his Christmas-themed biscuit represented.

Figure 2.

Image of an “indeterminate biscuit” taken from a University web site, which included the caption: “What on Earth is this???? Found in a package of Cadbury's Festive Friends chocolate biscuits in the office this afternoon. What on earth is it supposed to be? Santa? Reindeer? Snowman????? Do tell if you know!” Image reprinted with permission.

Visual indeterminacy can be defined, then, as the perceptual experience occurring in response to an image that suggests the presence of objects but denies easy or immediate recognition. Anecdotal evidence suggests that being confronted with such images arouses a need to determine what is depicted, so that additional attention is given in order to resolve the conundrum.

A Brief Survey of Visual Indeterminacy in Art History

The allure of indeterminate images has not escaped the attention of artists, who have frequently exploited their capacity to perplex audiences. In the first major study of unrecognizable images in art, Gamboni (2002) tracked the use of indeterminacy in works stretching deep into art history showing the way artists deliberately included elements or passages within paintings that confounded the viewers’ capacity to identify what they saw. A famous example is found in a work made in the late eighteenth century by Joseph Wright of Derby titled Experiment on a Bird in an Air Pump (National Gallery London, 1768). The painting depicts a scientific demonstration of the effect of oxygen deprivation on a bird, and is generally rendered with immaculate clarity. Yet there is a strange object floating in a backlit jar prominently positioned in the foreground of the scene. Ever since the painting was first exhibited people have wondered what this object is. It has been variously labeled a bird's carcass, a skull, and a pickled organ, but there is still no universally agreed interpretation (Schupbach, 1989, pp. 346–347). It has generally been assumed that the artist deliberately fashioned the object in this way in order to add an extra dimension of interest for audiences.

One artist who did more than most to exploit the artistic possibilities of visual indeterminacy was J. M. W. Turner, the English painter associated with the Romantic movement of the early nineteenth century and famous for his atmospheric landscapes and seascapes. In spite of his now titanic reputation, in his lifetime Turner was often vilified for producing what were seen as unreadable and indistinct works, which many critics thought flouted good taste and artistic probity (see Figure 3). It is surprising that the images Turner exhibited publicly and which were complained about most vociferously, such as the landscapes of the early 1800s, appear to us to now as quite clear and distinct. Of one such painting another artist commented: “…so much was left to be imagined that it was like looking into a coal fire, or upon an Old Wall, where from many varying and undefined forms the fancy was to be employed in conceiving things” (Gage, 1975, p. 450).

Figure 3.

A nineteenth century caricature satirizing J. M. W. Turner's painting methods, which were regarded by many of his contemporaries as producing indistinct or unrecognizable scenes.

Had critics seen some of the works Turner did not show in public, such as the highly indeterminate Interior of a Great House: The Drawing Room, East Cowes Castle of around 1830, now in the Tate collection in London, they would very likely have been bewildered. Historians are still unclear about the subject or the motive for the painting, and indeed even when inspected closely it is impossible to make out all but a fraction of the objects depicted. It has been described as being in a “state of dissolution” (Butlin and Joll, 1984, p. 282) being executed with a vigor and freedom hardly seen in Turner's contemporaries.

It is interesting to speculate what was going on in Turner's mind that led him to create such works, and in the minds of the public who struggled to read even his more recognizable pieces. We do know, however, that his paintings had an impact on the 30-year-old Claude Monet when he visited London in 1870 to avoid conscription into the Franco-Prussian War, where he saw Turner's work for the first time (House, 1986, p. 113). Monet soon echoed Turner's atmospheric images in a painting made in 1872, Impression, Sunrise, the title of which, when exhibited in 1874, gave the Impressionist movement its name. This rather sketchy rendering of Le Havre harbour in fog particularly incensed one contemporary critic, who ridiculed its imprecision and scornfully asked: “What does that canvas depict?” (Leroy, 1874).

In fact, as far as Monet was concerned the function of his painting was not to obscure but to faithfully depict the appearance of the world, in other words what he saw rather than what he knew to be out there in front of him. A painting like one of the many views he made of Rouen Cathedral in the 1890s is not so much a depiction of the cathedral's walls themselves than the light reflected by those walls (House, 1986, p. 221). It is up to us as viewers, according to the theories of vision popular among artists in Monet's day, to read into those patterns of light the form of the cathedral from which they were derived using our own conceptual resources. This is what Gombrich (1960, p. 161) referred to in Art and Illusion as the “beholder's share” of the pictorial bargain, the contribution viewers supply to the meaning of the image from their own imaginations. Gombrich (1960, p. 250) noted that Turner's great champion, the art critic, and theorist John Ruskin, urged artists to paint only what arrives at the “childish” or “innocent eye,” that is, the eye as a recorder of “flat stains of color, merely as such, without consciousness of what they signify – as a blind man would see them if suddenly gifted with sight.” Monet subscribed to such a view himself, remarking that he “…wished he had been born blind and then suddenly gained his sight so that he could have begun to paint in this way without knowing what the objects were that he saw before him.” (Nochlin, 1966, pp. 35–36).

As a young man in 1895 the Russian artist later to be credited with introducing abstraction to European art, Wassily Kandinsky, saw one of Monet's series of paintings depicting sunlit haystacks in a Moscow gallery. Unable to recognize what the painting was of, he later recounted:

And suddenly for the first time I saw a picture. That it was a haystack (or rather, a grain stack), the catalog informed me. I did not recognize it … And I noticed with surprise and confusion that the picture not only gripped me, but impressed itself ineradicably upon my memory. Painting took on a fairy-tale power and splendor. And, albeit unconsciously, objects were discredited as an essential element within the picture.

(Parsons and Gale, 1992, p. 255).

A similar experience is recounted in a passage from Kandinsky's Reminiscences when he returned to his studio at dusk and was astonished to see “an indescribably beautiful picture, pervaded by an inner glow” standing against the wall (Lindsay and Vergo, 1982, pp. 369–370). In it he could discern “only forms and colors” and no comprehensible objects. It was in fact one of his own rather impressionistic paintings turned on its side, the subject of which he had failed to recognize. Kandinsky realized the potential of objectless images to evoke a remarkable perceptual response. He subsequently spent many years refining a visual language through which this insight could be expressed.

Among contemporary artists, Gerhard Richter is somewhat unusual in that he works in a number of quite distinct styles. He is particularly recognized for both his photo-like images, precisely rendered, and his generally larger abstract works, which he frequently produces by an almost chance-like act of scraping, leaving the final effect to the unpredictable interaction between paint and tools.

But rather than being seen as either realistic, in the conventional sense, or abstract, in the sense of non-representational, Richter's work can be better understood as “indeterminate” in the way so far described here. What the artist is trying to produce is a sense of uncertainty, lack of fixedness, which draws the viewer in to try and resolve what they are seeing. Richter himself is very explicit about this, saying: “Pictures which are interpretable, and which contain a meaning, are bad pictures.” A good picture, on the other hand, “…demonstrates the endless multiplicity of aspects, it takes away our certainty, because it deprives a thing of its meaning and its name. It shows us the thing in all the manifold significance and infinite variety that preclude the emergence of any single meaning or view.” (Elger and Obrist, 2009, pp. 32–33). And in this exchange with the art critic Robert Storr he offers an insight into his own theory of indeterminate perception:

GR: I try to avoid something in the painting resembling a table or other things. It is terrible if it does because then all you can see is that object.

RS: So you allow for aspects or suggestions of images in the abstract work but not actual pictures?

GR: Not actual pictures. I just wanted to reemphasize my claim that we are not able to see in any other way. We only find paintings interesting because we always search for something that looks familiar to us. I see something and in my head I compare it and try to find out what it relates to. And usually we do find those similarities and name them: table, blanket, and so on. When we do not find anything, we are frustrated and that keeps us excited and interested until we have to turn away because we are bored. That's how abstract painting works…

RS: I am just saying that you use paintings as a way of making it difficult for people to read the image.

GR: Yes, that's right.

(Storr, 2003, pp. 178–179).

What is evident from this brief survey of visual indeterminacy in art is that artists who make hard to decipher images are doing so not just to be wilfully obscure or to confound their audiences. They are also acting rather like vision scientists by exploring how certain kinds of images engage the visual system and how we make sense of the world. Moreover, by heightening our visual awareness, so certain artists believe, indeterminate images in their various forms can produce interesting, even revelatory, esthetic experiences.

Creating Indeterminate Images

Like the artists cited here, my initial interest in the phenomenon of visual indeterminacy was artistic. I became absorbed by the challenge of creating images, both still and moving, that could induce the same state of visual uncertainty in others that I had undergone myself when watching the Cabinet of Dr Caligari sequence. I tried many methods of achieving this using film, video, collage, fractal image generation, and digital image manipulation. In each case I was trying to produce a picture of sufficient complexity to strongly suggest the presence of some object or scene yet at the same deny easy or immediate identification. Figure 4 shows an early paper collage examples of these experiments. I soon found the problem of “trapping” the human visual system in this way much harder than I had first anticipated. As I now realize is well known to vision scientists, the human visual system is extraordinarily effective at rapidly identifying objects in perception (Thorpe et al., 1996; Rousselet et al., 2002). Even given the scantest of clues – such as two dots and a curve – we can interpret things, like faces, almost instantaneously. Alternatively, if the information in an image is too noisy or distorted we simply categorize it as a “meaningless” abstract texture, and make no attempt to discern objects in it (see Figure 5).

Figure 4.

Uncertainty 4, paper on card, 29 cm × 15 cm, 1992. An early attempt to create an indeterminate image using paper collage.

Figure 5.

The image on the left is a noisy texture that does not suggest any objects and so is effectively treated as abstract. The simple arrangement of two dots and a curve on the right show how readily we are able to recognize objects, even from the scantiest of clues. The problem of creating indeterminate images is how to avoid both these kinds of interpretation.

The challenge in making artworks that are truly indeterminate, then, was to achieve a fine balance between recognizability and abstraction in order to excite the inquisitiveness of the viewer's visual system while frustrating its capacity for recognition at the same time. After many years of experimentation I gradually developed a method of drawing, and then painting, which seemed to produce this effect quite reliably. I discovered that by using a classical pictorial architecture, of the kind frequently found in European paintings made between the 1500s and early 1900s, I could create an image that incited strong expectations of recognizable objects and scenes. (This classical period was the epoch in figurative art that many people associate with recognizable depiction of forms, in contrast to later Modernism where artists turned increasingly to distortion and abstraction.) By using this overall pictorial structure but omitting, or otherwise manipulating those features of the image that would be readily recognized I was able to achieve a consistently indeterminate image. Some examples of these paintings are shown in Figures 6, 7, and 8.

Figure 6.

Paralysis, Oil on panel, 27 cm × 33 cm, 2006. Private collection.

Figure 7.

Impulse, Oil on canvas, 80 cm × 70 cm, 2006. Collection of the University of Exeter.

Figure 8.

The Flight, Oil on paper, 30 cm × 40 cm, 2007. Private collection.

In the early stages of making this work, the process of deciding what made a certain image successfully indeterminate in the terms described above was largely a matter of my personal judgment. I had to rely on my own reading of the image I was producing, and gage whether or not the forms in it were sufficiently evocative of objects or scenes, or whether they were too abstract or textural to incite the curiosity of the viewer. Increasingly I sought the opinions of others by showing the paintings in galleries or the studio and asking viewers to describe the processes occurring in their own minds as they studied the works. After doing this many times I found people tended to report they were having similar kinds of experiences. Their initial response was to think they were seeing a classical painting depicting a familiar theme, such as landscape, figure, or still life. But wherever they looked to find objects that would corroborate this initial response they failed to do so. They would fixate on an area in which they thought they saw a human limb or a piece of cloth, but would then realize that this was a false start, and would look for some other salient feature to pin their interpretation on. Many reported they were looking at certain forms within the images and sifting through the possible interpretations in their mind, testing various options in order to successfully name what it was they were looking at. Most people reported this experience in positive terms, as interesting, or visually exciting, although some did tell me the images were “disturbing” or made them feel anxious.

This process of testing the indeterminacy effect of paintings on viewers was very useful as a way of confirming or refuting my own judgments about the way the images would be read. Those paintings I felt were more effective also tended to be the same ones other people would report as having the strongest effect on them. But although useful in guiding my judgment, these viewer surveys were not carried out in any scientifically valid way. They were simply verbal reports elicited under a variety of conditions and recorded rather haphazardly. Having had a longstanding interest in the science of perception and visual consciousness I wondered if scientific methods could be usefully applied to study the effect I was investigating in a more systematic way. I also became increasingly interested in what science might have to say about the phenomenon of visual indeterminacy, and what effects the process of looking at indeterminate images might be having on the vision systems and brains of those looking at them.

Scientific Background to Visual Indeterminacy

As I started to look for scientific literature relating to visual indeterminacy it became clear this was a relatively lightly investigated area of perception compared, for example, to the related phenomenon of ambiguous or reversible images. Ambiguous images, such as the Necker cube, the Duck–Rabbit illusion, or the Boring vase, are distinguished by having alternating interpretations (the image is perceived either as a duck or a rabbit) each of which is quite determinate (Kleinschmidt et al., 1998; Meng and Tong, 2004). Also well known are the issues around perceptual organization and so-called “hidden figures,” exemplified in R. C. James’ famous photograph of a Dalmatian dog in a dappled environment (Gregory, 1970; Palmer, 1999; Ramachandran and Hirstein, 1999). These, and other similar “puzzle pictures,” direct the viewer to search for objects that are concealed in some way within the structure of the image, and once found then not easily lost. An example is the image of a cow first presented by Dallenbach (1951), a version of which is reproduced in Figure 9.

Figure 9.

This is an image of a cow, although most people are unable to see it at first glance, or even after prolonged study. Once seen, however it is very difficult to see the image as it appeared prior to the point of recognition. From American Journal of Psychology. Copyright 1951 by the Board of the University of Illinois. Used with permission of the author and the University of Illinois Press.

When I first saw this photograph I remember having a good deal of difficulty in finding the cow, although once I did it was very hard to see it as anything else. The experience I had prior to the point of recognition was similar, as I recall, to that occurring during the Cabinet of Dr Caligari sequence many years before. Both were marked by a sense of struggle in which various alternative interpretations were tried out until the flash of recognition occurred. My interest in such images was less in the moment of recognition than the preceding process of object search, and what kinds of perceptual processes might be taking place during this time.

The perceptual state of visual indeterminacy occurring prior to the moment of recognition bears similarities to the rare neurological disorder of associative visual agnosia. A notable case study of this condition, presented by Humphreys and Riddoch (1987), concerned a patient, John, who had suffered a stroke resulting in a bilateral lesion in the region supplied by the posterior cerebral artery. Much of John's capacity to see was spared, but his ability to recognize what he saw was greatly impaired. When shown a series of line drawings of everyday objects he was able to identify only a small proportion, and relied on “working out” what was depicted from specific clues within in the image, such as the curliness of a pig's tail, rather than by seeing the object “as a whole” (p. 60). In arriving at their diagnosis of associative visual agnosia the authors ruled out other possible factors that could have contributed to John's inability to recognize everyday objects, including any residual deficit in his stored knowledge or visual sensation. He showed no difficulty in recognizing objects by other means, such as touch, or describing them in detail from memory and was able to make quite accurate copies of drawings, albeit slowly. Figure 10 is a drawing made by John (on the right) copied from the picture of the owl (on the left). The authors note that John could quite accurately copy line drawings of objects, “…even when he had no idea what the object was” (p. 69). To demonstrate this they point out that John faithfully reproduced the gap on the right side of the original owl's head, not seeing it as an omission in the original drawing.

Figure 10.

The drawing on the right is the copy made by John, the patient with visual agnosia, of the drawing on the left. The fact that John could make this copy showed that his capacity to “see” was in tact, although he had no idea what it was he was copying. Reproduced by permission (©1987 Oxford University Press).

For Humphreys and Riddoch, John's case suggests the normal processes of object recognition involve a number of operations that occur, to some extent, independently of each other but which can be broadly grouped into two layers (pp. 101–102). The first set of processes organize the perceptual “input” data according to position and orientation, and bind multiple visual elements into wholes. A subsequent set of processes then match that input data to associations about function and meaning. The authors conclude: “In general terms, (John's) case supports the view that “perceptual” and “recognition” processes are separable…” (p. 104).

In her extensive study of visual agnosia Farah (2004) makes the same broad distinction between perceptual input and the conceptual associations involved in visual object recognition. While stressing the non-serial, multidirectional processes in vision, she summaries: “Visual form agnosia validates the distinction implicit in the labels “early” and “intermediate” vision, on the one hand, and “high-level,” “object” vision on the other, by showing the first set of processes can continue to function when the second set is all but obliterated. It shows us a kind of richly elaborated but formless visual “stuff,” from which “things” can be derived” (p. 156). The phrase “richly elaborated but formless visual stuff” accurately describes the appearance of indeterminate images prior to the point of recognition. It is precisely the inability to match this “visual stuff” to one's stored memories and associations that seems to characterize the visually indeterminate state. For most of us this can occur occasionally, but for the unfortunate sufferers of visual agnosia it is a permanent condition.

My hunch was that during the period where viewers are searching for meaning among the pictorial clues something is occurring in their cognitive processing which is different from that occurring during normal recognition. This seemed to be supported by some scientific studies looking at brain responses to unrecognizable versus recognizable images. An experiment by Supp et al. (2005), for example, used EEG techniques to examine the changes in cortical networks within the time-window of the event-related potential component N400. They showed subjects sequences of recognizable and unrecognizable gray scale pictures, the latter matching the former as closely as possible in terms of size, complexity, and structure. The results showed a marked increase in cooperation in certain parts of the brain and a greater degree of overall coherence between different regions during the viewing of unrecognizable pictures as compared with recognizable ones. This, they concluded, reflected the greater demands made on the viewer's perceptual and cognitive resources and consequent “unease” involved in the task of semantically matching the undecipherable stimuli: “…the greater number of coherence increases for meaningless object processing suggests enhanced recruitment of more distributed left and right areas during unsuccessful memory search” (p. 1143). This finding seemed to corroborate my own sense of “unease” when confronted with an unrecognizable image, and the sense of mental struggle involved in trying to resolve the conundrum.

In another study that compared brain responses to recognizable and unrecognizable images, Rainer et al. (2004) measured neural activity in the V4 area when exposing the monkeys to images that were increasingly degraded, from clear to abstract noise. The monkeys learned to recognize familiar images that were degraded compared to novel ones that were treated in the same way. The researchers found that monkeys exposed to indeterminate images showed significantly increased neural activity in both primary and higher cortical areas of the brain than when faced with familiar or recognizable stimuli. From this Rainer and his team drew the conclusion that not only are particular loci in the brain recruited in response to indeterminate stimuli, but that the attempt to decipher such stimuli leads to enhanced overall coordination in brain activity: “This suggests that V4 plays a key role in resolving indeterminate visual inputs by coordinated interaction between bottom-up and top-down processing streams” (p. 275).

It was while presenting a lecture on indeterminate art at the Max Planck Institute for Biological Cybernetics at Tübingen, at the invitation of Gregor Rainer, that I proposed a possible study in which the effects of looking at indeterminate paintings would be compared to paintings that looked similar but contained recognizable objects. To demonstrate this I showed a painting of my own next to a detail of Michelangelo's Sistine Chapel ceiling (Figure 11).

Figure 11.

On the left is the painting Succulus (Oil on canvas, 123 cm × 123 cm, 2005) and on the right a detail from Michelangelo's Sistine Chapel ceiling. Both images have a similar visual structure and coloring, yet one is full of recognizable objects while the other is not.

My painting had a similar visual structure and colors as the Michelangelo, but omitted any clearly discernable objects, such as people or clothing. Working on the basis of the intuitive hunch noted above, I proposed that comparing the brain activity of subjects exposed to these similar images might reveal some useful information about the processes involved in object recognition. It happened that in the audience were two scientists who offered, in different ways, to carry out tests using my paintings as stimuli, which led to several collaborative studies being undertaken on art and visual indeterminacy.

Scientific Experiments on Visual Indeterminacy

Working with the neuroscientist Alumit Ishai, and her team at the Department of Neuroradiology in the University of Zürich, I created a set of stimuli that included a selection of my own paintings, all of them in the indeterminate classical style described above, and the same number of paintings made by other artists, which had a similar visual appearance but were full of recognizable objects. These included works by artists such as Turner, Tintoretto, Rubens, Michelangelo, and Fuseli among others. Some samples can be seen online at: http://www.robertpepperell.com/Stimuli/Stimuli.html. These stimuli were divided equally into monochrome and color sets, and then presented in a number of behavioral experiments. Subjects with no specialist art training were shown the stimuli in random order and asked to perform a number of tasks, including deciding whether each image contained familiar objects (a measure of object recognition) and how powerfully the images affected them (a measure of esthetic response). Scientific details of the experiments and the results can be found in the published paper (Ishai et al., 2007) but I'd like to reflect here on some of the findings that I found surprising and interesting from an artistic point of view.

One of the unexpected results concerned the extent to which subjects reported seeing familiar objects in my indeterminate paintings. Given that I had striven so hard to remove any trace of recognizable objects, leaving only strong suggestions, it was interesting to discover that people were claiming to see things they recognized on average 24% of the time. (It was less surprising to me that the effect was stronger with the color images compared with the monochrome as I had always found it easier to create the effect of visual indeterminacy when making monochrome paintings; it is noticeable how readily a pinkish hue will suggest flesh or a bluish hue sky.) As one might expect, subjects reported seeing familiar objects in the other artists’ work almost 100% of the time. It was also notable that the subjects gave almost identical scores for esthetic affect across all the paintings in the study, regardless of how recognizable they were. What this seems to indicate is that, in rating their esthetic response, the subjects were less influenced by the literal meaning of the images they saw than the immediate visual impact of the shapes, colors, and composition. This is despite the fact that previous studies have shown a tendency for non-art trained audiences to prefer pictures they can recognize more than abstract ones (Healey and Enns, 2002; Vartanian and Goel, 2004).

In art historical terms the distinction between the meaning of an art work and its physical appearance has been understood in terms of “content” and “form,” and this distinction has given rise to prolonged and often impassioned debate among theorists of art and esthetics as to which aspect is the more significant in determining the effect of an artwork and, indeed, whether the two aspects can really be distinguished at all (Bell, 1914). What this study suggested to me as an artist is that the distinction does have some validity given that the “formal” properties of the artworks seem to be a more significant factor in their degree of esthetic appreciation than “meaning” factors, at least over the short (4 s) viewing period used in the trials. The study also showed that subjects were significantly slower to make judgments about indeterminate paintings than they were about recognizable ones, whether they saw objects in them or not, which might suggest that the attempt to find objects in the indeterminate images requires a different kind or greater degree of underlying cognitive processing than when perceiving recognizable images. This seemed to corroborate the implications of the studies cited above, where viewing indeterminate images can lead to differential activity in certain areas of the brain. Crucially, though, there was a significant correlation between the length of time taken to determine whether or not images contained objects and rating of esthetic effect, such that the longer it took to make a decision, the more powerful the image was thought to be.

It is worth elaborating on the fact that the rating of esthetic effect employed in the study was slightly unusual. Rather than using a rating of “ugly to beautiful,” on the basis that esthetic experience is synonymous with the appreciation of beauty, we used one of “powerful affect” on a scale of 1–4, with 4 being the most powerful. The reason was that, as even a cursory glance at art history will show, the esthetic impact of a work of art is not necessarily linked to how beautiful, pleasurable, harmonious, or pleasant it is. Some of the most impressive art works can be quite ugly, disturbing, distorted, or dissonant. One thinks of Goya's Saturn Devouring his Son (1823, Prado, Spain), Picasso's Mother and Child (1907, Musée Picasso, Paris), or the Chapman brother's Hell (1999–2000, Saatchi Collection, London). The use of the term “powerful” arguably more accurately captures the range of emotions felt by an audience in response to a work of art and is therefore more objective as a measure of affect than the more limited category of beauty alone. The fact that from this study it appears increased recognition latencies are associated with an increase in the “powerfulness” rating of the image indicates that the amount of struggle or effort needed to comprehend an image has some positive relationship to its esthetic value. This was also something I had intuitively suspected, based on my own esthetic experiences of indeterminate artworks and the fact that such images are so often revered in the canon of art.

Although firm deductions cannot be drawn from this single study, it seems that the experiments described above, coupled with the subjective reports I gathered from viewers when making the works, give good reason to believe that something is happening in the case of viewing indeterminate art works that is not happening with immediately recognizable ones. I know my own experiences of seeing indeterminate images, whether art works or not, to be moments of great vividness and highly focused attention, where the habitual operations of recognition are fleetingly suspended as the mind struggles to resolve the components of the image into something meaningful. This is by no means a straightforwardly pleasurable experience; it can sometimes be quite frustrating or disorienting, and not immediately rewarding. In esthetic terms, however, I regard the experience as being of great value, since for a few moments I am acutely aware of the visual form of the scene before me in a way I am not when the image is semantically determined. This, then, might be thought of as part of the heightened mode of perceptual experience associated with indeterminate images, and may help to explain why artists over the centuries have been so frequently drawn to making them.

A follow-up study used neuroimaging techniques to look at the activity in subjects’ brains while viewing the same stimuli plus another set of artworks that were entirely abstract, that is, with no suggestion of objects at all (Fairhall and Ishai, 2008). Also included were scrambled images that were essentially visual noise. The study was design to test the prediction that abstract, indeterminate, and recognizable images would produce a “posterior-to-anterior gradient of activation along the ventral visual pathway, with stronger response to abstract compositions in inferior occipital gyrus; stronger response to indeterminate paintings in intermediate regions in posterior fusiform gyrus; and stronger response to representational paintings in anterior fusiform gyrus” (p. 925). Using a similar object recognition task as employed in the previous study, the behavioral data recorded in the scanner revealed an even stronger propensity for subjects to report seeing familiar objects in my indeterminate paintings (now 36% of the time). They even reported seeing familiar objects in the abstract paintings 18% of the time, even though these had been chosen specifically for their lack of object-suggestive content. As expected, there were almost no reports of objects being seen in the scrambled images.

In neuroscientific terms, the results were able to partially confirm the hypothesis, and again I only want to comment here on some of the interesting implications the study had for me as an artist. It was gratifying for me to know, for example, that based on the data in this study at least there is a detectable “indeterminacy effect” produced in subjects when looking at my paintings. By comparing the level of activation between the scrambled paintings and my own there was a significant differentiation in certain brain areas (precuneus and medial frontal gyrus), which were described in the paper as the “neural correlates of object indeterminacy.” Once again, subjects took longer to decide on the question of whether the images contained familiar objects or not in the indeterminate and abstract paintings compared to the recognizable ones, which suggests a more effortful process is going on when judging ambiguous or suggestive imagery. But I was slightly surprised that the effect of seeing indeterminate images on recordable brain activation was less pronounced than I had expected. In my naivety about the way the brain works, and what the scanning process is able to detect, I had anticipated a far stronger degree of differential activation during the exposure to indeterminate images as compared to recognizable ones than was found.

But the study did confirm one of my other intuitively held beliefs about the way we perceive the visual world. Part of my anxiety, or unease, during the moment of indeterminate perception in the Cabinet of Dr Caligari sequence arose from the sense of compulsion I felt to make sense of what was in front of me. I have felt the same many times since when unexpectedly confronted with an indeterminate scene. The fact, confirmed in this study, that subjects reported seeing objects in images that did not contain them, even more so than in the one cited previously, is evidence of the involuntary impulse we have to turn the rich and complex visual data around us into meaningful things. As the paper concluded: “Our findings indicate that this seemingly effortless process (of recognition) occurs not only with familiar objects, but also with indeterminate stimuli that do not contain real objects. It therefore seems that the primate brain is a compulsory object viewer, namely that it automatically segments indeterminate visual input into coherent images.” (p. 929) This helps to explain why indeterminate images can be so compelling.

Separate studies conducted by Wallraven et al. (2007a,b) at the Max Planck Institute used the same indeterminate paintings employed in the previous studies, but this time subjected them to a range of psychophysical tests using eye-tracking and categorization tasks. The purpose was to look at the ways subjects would react to indeterminate stimuli, and also to see if there were any empirical grounds for verifying my own intentions in making my art. Having worked so long to make successfully indeterminate paintings on the basis of intuition, guesswork, and the informally acquired reports of others, it was again fascinating for me as an artist to see what more rigorous and objective measures might reveal about, quite literally, how people looked at the work.

In one set of behavioral experiments, my indeterminate paintings and the visually similar representational paintings were submitted to a range of tests looking at subjects’ responses to variations in size and orientation. They also undertook a categorization task where participants were asked to classify the images into one of seven genres, which were “Biblical scenes,” “Landscapes with person,” “Landscape without a person,” ”Portrait,” ”Still life,” “Battle scene,” and “None of the above.” In another experiment, participants were shown the sequence of indeterminate images and, in addition to the categorization task above, were also asked to identify whether or not the images contained people, during both of which their eye-tracking movements were recorded. Again the scientific details can be consulted in the relevant papers, but two outcomes were of particular interest to me as the originator of the indeterminate stimuli.

First, the analysis of the data produced by the experiments seemed to verify my intentions in making the indeterminate images. Wallraven et al. (2007a) proposed generally that visual information can be ordered along two parameter dimensions, namely “abstract/representational” and “unique/ambiguous.” Images that score highly on the “unique” parameter are very distinct in meaning, whether they are abstract (as in the case of certain symbols or icons) or representational (as in the case of photographs or photorealistic paintings). Images that are rated as being more “ambiguous” may be almost entirely non-representational (as in the case of certain abstract art) or have multiple meanings (as in the case of certain optical illusions or surrealist artworks). These two parameter dimensions also function on two distinct layers which normally operate together in visual perception: the perceptual layer, which broadly speaking is the same “bottom-up” or “lower-level” set of processes involved in organizing visual data described above in the section on visual agnosia, and the conceptual layer, which consists in the “higher-level” or “top-down” information retrieved from stored memories and associations (Figure 12).

Figure 12.

Schematic representation of the pictorial “interpretation space” (Wallraven et al. 2007a) representing the “abstract/representational” and “unique/ambiguous” parameter dimensions existing of both perceptual and conceptual layers of vision. Indeterminate images occupy the area between representational and abstract and between unique and ambiguous parameters, but tending toward the ambiguous end of the axis.

As someone interested in the way we perceive images, this “map” of what the authors called the visual “interpretation space” is extremely useful as a way of organizing the different variables that can influence the way an image is read. On could imagine, for example, using it to visualize the whole history of recognizability visual art, and thereby track the shifting patterns of taste across the centuries. When this same parameterization was applied to my own paintings by the team it was expected that in order to fulfill my ambition to make works that were neither fully recognizable nor fully abstract they would need to be assessed as being located roughly in the center–right of the graph, that is, avoiding the extremes of each parameter, but tending toward ambiguity. Once the genre categorization tasks had been carried out the results showed that the images were indeed distributed around a “region of indeterminacy” at the center of the graph, with a bias toward to ambiguous end of the axis. For the experimenters, this data offered a perceptual validation of the artistic program behind the work. It was also interesting to note that, as with the previous studies cited, subjects reported a relatively high percentage (37%) of my images as being “representational,” despite the lack of any clear objects being depicted.

The other interesting finding from these experiments from my perspective concerned the differences in eye movements the participants displayed when engaged in the person finding task as compared with the genre categorization task. What surprised me was the extent to which the fixation maps varied between the two tasks, even though subjects were looking at the same images. As has been known since the time of the early eye-tracking experiments by Yarbus (1967) how one looks at a image is critically dependent on what is being looked for. While this is clearly well known to scientists it is not, as far as I am aware, something generally known to artists. Yet is clearly a fundamental aspect of the way we apprehend the world, which presumably reflects the way expectation and meaning are mediated by the visual system in general and the brain in particular. It is interesting to consider, therefore, how directive clues in art, such as titles or hanging context, might affect the way audiences look at works of art. It also alerted me to how important the use of titles might be in my own work in leading the “eye” of the viewer as they try to interpret the image.

The question of how titles affect the interpretations of paintings stimulated the final art–science collaborative study I wish to mention (Wiesmann and Ishai, 2010). The study used a selection of Cubist paintings made by the artists Pablo Picasso, Georges Braque, and Juan Gris in the period before First World War. Cubist paintings of this period are characterized as being highly indeterminate in so far as they are directly observed depictions of everyday objects – tables, fruit, newspapers, glasses, etc., – but represented in a fragmented and “exploded” manner that makes immediate identification very difficult. Art experts, and others familiar with the genre, are able to “read” these Cubist scenes and find within them the various forms and objects from which they are constructed. But those without this expertise tend to see only patterns, lines, and textures rather than distinct objects of any kind (Golding, 1988).

One part of the study looked at the extent to which descriptive titles presented alongside Cubist paintings affected the viewer's capacity to identify objects in the scene. Crucially, however, half the subjects undergoing the task of detecting familiar objects received a short training session before the trial in which they were instructed on how to “read” Cubist paintings and find objects in them. The study gathered both behavioral and fMRI data, and again the scientific methods and results are available in the published paper. Samples of the stimuli can be seen at: http://www.robertpepperell.com/Cubism/index.html. What was surprising from my own perspective was the extent to which the “trained” subjects differed from the control group in terms of the number of objects recognized. Despite the fact that the subjects were not art experts and received only a relatively brief training sessions (30 min) they were significantly better than the control group in recognizing familiar objects. The study also found that the role of the descriptive titles, which effectively declared what the paintings depicted, has little effect on the control group but a marked effect in helping the trained group to find more familiar objects. To me, as both an artist and art teacher, these results were somewhat counterintuitive inasmuch as: (a) I would have expected the process of learning to read Cubist paintings to be something only acquired over many hours of study rather than the brief period of training undergone by these subjects, and (b) that meaningful titles would have had some positive effect on helping those with no training to find familiar objects more often than when looking at the same image only accompanied by the word “Untitled,” as was the case here. The study also showed enhanced activation in the parahippocampal cortex of the trained subjects, the amplitude of which increased as a function of the number of objects recognized. This suggested that the subjects had used broader contextual associations to identify the objects in the paintings rather than the cognitive resources normally linked more specifically to object recognition. It is also tempting to wonder whether subjects thus trained in recognizing objects in Cubist paintings are also then better at other object recognition tasks, and indeed whether learning to understand Cubist art can actually improve cognitive performance in other areas; it would certainly be good news for art lovers if that were the case.

Art, Science, and the Brain

The various investigations I have undertaken with neuroscientists and psychophysicists have proved illuminating and rewarding from my artistic perspective. I initially set out to discover what science might be able to tell me about the specific issue of visual indeterminacy, and how people respond to my paintings. In doing so I have gained an enormous amount of insight into the way the visual system operates, how the brain functions, and indeed how science itself operates when investigating these phenomena. I have become aware of the great potential of the scientific method to elucidate processes that artists often work with intuitively but rarely grasp in any systematic way. But I have also seen at first hand the limitations of the scientific method when studying the experience of art, and have been reminded of the very different cultures that exist between art and science that make meaningful collaboration a sometimes demanding process. In the final section of this paper I want to briefly reflect on these issues and how future joint research between artists and scientists might benefit from these experiences.

In the first place, it is important to acknowledge the inherent limitations of the scientific method when investigating the way we perceive art – at least as they appear from an artist's view. It often goes unremarked, for example, that most if not all lab-based studies of audience responses to art will use reproductions instead of real works of art. Reproductions are not always of the highest quality, and cannot be shown in a way that properly reflects the physical properties of the work itself. When preparing the images for the Cubism study, for instance, it was necessary to conform all the images to the same scale and format due to the demands of the experimental procedure. This meant a lot of cropping and resizing, which resulted in the loss of size discrimination between large and small paintings. And there is the broader question of how valid it is to measure the effects artworks on the basis of reproductions at all. Some empirical esthetics studies have shown significant differences in the judged hedonic or pleasure value of original artworks compared to reproductions (Locher et al., 2001). Certainly any serious scholar of art would make a point of examining the real work before arriving at any definitive evaluation of its esthetic impact. Many qualities inherent in a work of art simply do not covert into photographic media, including scale, degree of surface gloss, texture of brushwork, or the way that certain colors can change depending on the angle of viewing (as is the case, for example, in many paintings by the abstract artist Ad Reinhardt). All these are crucial esthetic properties that artists work hard to control, and their absence or impoverishment in conventional photographic reproductions restricts what many lab-based studies can tell us about the experience of looking at them. Then, of course, there are all the well-known problems associated with subjects being placed in fMRI scanners, with the distracting noise and discomfort they create (Cooke et al., 2007). Something similar, but less intrusive, is true of eye-tracking devices that require the head to be locked in a stable position – something that clearly would not happen in a natural gallery setting. While these limitations do not, in my view, diminish the value of such studies they should perhaps be more frequently acknowledged when discussing the implications of the results.

Another issue that those wishing to study the effects of art on the brain might want to consider is the risk of what might be called “neuro-determinism,” that is, the expectation that esthetic experience can be fully accounted for in terms of brain-centered processes. The neurobiologist and pioneer of the neuroesthetic approach to art–science integration, Zeki (1999, p. 217), said in his seminal book on the subject: “My aim in writing this book has been really to convey my feeling that esthetic theories will only become intelligible and profound once based on the workings of the brain…” While he has been careful elsewhere to insist he is studying the neural correlates of experiences like beauty and not necessarily the causes (Kawabata and Zeki, 2004), there is a understandable temptation to assume that some of our most uniquely human experiences, such as art appreciation, might be explicable purely in terms of certain kinds of brain activity. It is worth noting in this context a growing tendency within philosophy and psychology toward “externalist” models of perception and cognition. These models, to varying degrees, deny or resist the idea that the brain is the sole location of mental properties such as beliefs, memories, and even the mind itself (Noë, 2005, 2009; Clark, 2008; Hurley, 2008; Velmans, 2008). Allied to this is the fact that many artists and art theorists, when discussing the matter, seem to intuitively support the idea that mental properties and esthetic experiences extend beyond the head and into the world (Pepperell, 2011). The purpose of raising this issue here is to point out that certain basic assumptions about how esthetic experiences might be constituted can differ fundamentally between those making the art and those studying its biological effects. In order to achieve a fuller understanding of what the brain contributes to esthetic experience as a whole we will need to reconcile these divergent approaches.

This leads to the final point, which concerns the need to recognize how great the disciplinary gulf still is between art and science, despite all the work done in recent times to bridge it. I have been attending science conferences now for over 10 years, and working closely with scientists on and off for about five. In that time I have rarely found members of the scientific community to be anything other than generous with their time and ideas, politely inquisitive about my proposals, and forgiving of my own naivety about their specialisms. Even so, I am also constantly reminded of how different the basic conceptual categories can be between the arts and sciences, a cultural divide of the kind famously identified by Snow (1993) in the middle of the last century and still largely in force today. The difference is in part, I believe, born from the need for scientists to be explicit, analytical, and logical in their working and reporting processes. Quite often for artists the opposite is the case, their training and traditions having implanted in them a proclivity toward vagueness, synthesis, and irrationality. The Cubist painter Braque (1971) was fond of saying: “Art is meant to disturb; science reassures.”

Finding common ground between two such distinct traditions is not always straightforward. It was somewhat sobering for me to discover that the constraints on the experimental equipment used in the collaborative fMRI study cited above required the subjects to express their esthetic appreciation for the artworks on a scale between 1 and 4. For those schooled in the infinite subtleties of artistic expression the idea that the merits of a great Turner or Rubens painting could be judged on such a crude scale and in as brief a moment as 3 or 4 s would border on the absurd. Yet if we are to make any progress at all in understanding art using the empirical methods of scientific enquiry these are exactly the kinds of procedures we will have to adopt, at least until more sensitive techniques of investigation become available. Just as I have had to modify my expectations about what empirical techniques are able to measure so I have been fortunate to find scientific collaborators willing to adjust their disciplinary spectacles in order to appreciate the relatively chaotic point of view of an artist. The result has been, from my point of view, a deeper understanding of what science can tell us about art, and what art can tell us about science.

Art–science collaborations work best when each discipline is enriched through the process, rather than one being parasitic on the other. There is always a risk that the compromises necessary to make progress are made at the expense of the essential values and outlooks of both approaches, which can only result in bad art and bad science. The challenge is how best to reconcile these distinct traditions without sacrificing the integrity of either. Only by meeting this challenge will we be able to create a truly interdisciplinary approach to the study of problems as complex as the way we make and appreciate art.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.