Emotional palette: a computational mapping of aesthetic experiences evoked by visual art

Abstract

Despite the evolutionary history and cultural significance of visual art, the structure of aesthetic experiences it evokes has only attracted recent scientific attention. What kinds of experience does visual art evoke? Guided by Semantic Space Theory, we identify the concepts that most precisely describe people’s aesthetic experiences using new computational techniques. Participants viewed 1457 artworks sampled from diverse cultural and historical traditions and reported on the emotions they felt and their perceived artwork qualities. Results show that aesthetic experiences are high-dimensional, comprising 25 categories of feeling states. Extending well beyond hedonism and broad evaluative judgments (e.g., pleasant/unpleasant), aesthetic experiences involve emotions of daily social living (e.g., “sad”, “joy”), the imagination (e.g., “psychedelic”, “mysterious”), profundity (e.g., “disgust”, “awe”), and perceptual qualities attributed to the artwork (e.g., “whimsical”, “disorienting”). Aesthetic emotions and perceptual qualities jointly predict viewers’ liking of the artworks, indicating that we conceptualize aesthetic experiences in terms of the emotions we feel but also the qualities we perceive in the artwork. Aesthetic experiences are often mixed and lie along continuous gradients between categories rather than within discrete clusters. Our collection of artworks is visualized within an interactive map (https://barradeau.com/2021/emotions-map/), revealing the high-dimensional space of aesthetic experiences associated with visual art.

Introduction

Central to human culture and evolution is our capacity to create art. Recent archeological evidence suggests that humans have been creating visual art—cultural artifacts that appeal primarily to the visual sense—for at least 40,000 years¹. Human capacities for music and dance are likely older². Why did humans become such an aesthetic species? One answer is that the arts represent emotion and become central to how members of a culture share in the experience and expression of emotions that enable collective living³. Through the creation of art, cultures store conceptualizations and practices related to emotion, and through the experience of art, individuals come to a shared understanding of emotions and their place within culture⁴. Brushstrokes, colors, lines, symbols, and themes are used by artists to elicit rich aesthetic experiences in viewers.

Early philosophical accounts and recent theoretical models posit that visual art can broadly impact our neurophysiology, feelings, judgments, and social behavior, recognizing that emotional and cognitive processes play a crucial role in these experiences^5–7. Although empirical research has begun to explore the emotions we feel and the qualities we perceive in response to visual art, a comprehensive taxonomy of the specific aesthetic experiences associated with visual art remains elusive. What is the structure of aesthetic experiences associated with visual art? Do people’s emotional responses differ from the feeling qualities they perceive in the artwork itself? Do aesthetic experiences have clear boundaries, or are they blended to create intermediate feeling states? To what extent do the emotions people feel in response to visual art correspond to the emotions of daily living? Answers to these questions are critical to understanding the processes by which visual art engenders aesthetic experiences.

Aesthetic experiences: The cultural archive hypothesis

In her influential books^4,8, philosopher Suzanne Langer advances the thesis that the arts’ “purpose is to objectify feeling”. Each artwork, she details, is a unique kind of representation of emotion. In making visual art, members of a culture archive beliefs about what Langer called “life’s patterns”: concerns or themes of social living that are central to the human experience, like the nature of suffering, loss, unfairness, infidelity, the rise and fall in power, and life and death⁹.

Advances in the understanding of cultural evolution converge with this thinking. Culture can be thought of as an ever-evolving repository of shared knowledge, practices, and experiences that enables individuals to adapt to the challenges and opportunities in the natural, human-designed, and social environment¹⁰. Acts of imaginative culture, like the visual arts, evoke aesthetic experiences of emotion, enabling individuals to become sophisticated practitioners of emotions, and how they structure judgment, action, and social interactions^3,11,12.

In keeping with this thesis, with forms of representation in the arts, members of a culture portray emotionally rich interactions, like those involving courage or jealousy or power, in stories, legends, myths, and visual artworks¹³. Members of a culture use visual techniques in paintings, figurines, and carvings to portray or represent profound interactions like childbirth, sexual relations, power dynamics (e.g., enslavement), and combat, and dramatize bodily expressions of emotion in dance, dramatic performance, and masks^14–16. These cultural representations serve as memorable ways of eliciting shared emotional and cognitive experiences, inviting individuals into the understanding and practice of the patterns of a culture¹⁷.

The cultural archive hypothesis has been extended to select empirical studies of visual art. In an early study, art students were assigned to produce nonrepresentational line drawings conveying one specific expressive meaning, such as femininity, energy, depression, illness, anger, joy, or tranquility¹⁸. Another group of students then rated the extent to which the drawings conveyed the intended expressive content. Results showed agreement among judges, with 30–50% of raters reliably judging the meaning expressed by the art students in their nonrepresentational drawings. A more recent study showed that viewers spontaneously reported feeling similar emotion patterns, such as sadness and confusion, reported by the artists when making a piece of art, regardless of whether these had been intended¹⁹. Visual art seems to communicate emotions, which viewers routinely receive and understand.

These studies suggest how the arts might allow members of a culture to develop a shared understanding of the emotions, ideas, and perceptions that constitute the moral stances of their culture²⁰. Through such emotional enculturation, people learn how to engage in vital interactions, such as negotiating status, redressing injustice, or tending to someone who is suffering, and embody their roles and identities within a culture’s pattern of relationships. The arts archive human emotion and the social patterns that unfold in their experience. Yet, which aesthetic experiences artworks elicit, and how they do so in viewers, is not well understood.

Past efforts to understand aesthetic experiences in response to visual art

The cultural archive hypothesis centers upon the questions we address in the present investigation: How do emotions archived in artworks elicit experiences in viewers? What is the nature of these aesthetic experiences? Past studies have begun to offer answers to these questions. One common approach has been to focus on broad evaluations of artworks, such as the individual’s felt pleasure/displeasure or mere liking/disliking²¹. Within this tradition, the animating idea is that hedonism, or broad feelings of pleasure, is core to aesthetic experience. Relevant empirical studies have captured the global evaluation of the work but not experiences of specific emotions, like anger, love, and compassion, or specific qualities attributed to the artwork, like peaceful, elegant, and warm.

Another tradition has focused on a small set of emotions, such as “joy”, “sadness”, “anger”, “fear”, and “interest”, providing key insights on experiences of basic emotions^22–24. However, the space of emotional experience is now known to be much richer (see²⁵ for a review). Advanced computational techniques applied to the study of facial–bodily expression, vocal bursts, prosody, the feelings evoked by music and video, and brain patterning triggered by emotional videos provide convergent evidence for at least eighteen distinct states found across all modalities and media, in addition to modality- or media-specific states, such as pride and shame in the face/body and the feeling of dreaminess in response to music^14,26–31.

Past approaches to measuring aesthetic experiences in response to visual art have also focused on discrete or modal emotions with clear boundaries between categories. However, much of human emotional life is complex, involving states of mixed feeling, such as “nostalgia” and “craving”, and of a nuanced nature, such as “entrancement”, “horror”, and “grief”²⁷. Although it is possible to experience pure emotions, we are likely to feel a mix of aesthetic experiences in response to a single artwork, such as a blend of awe, love, and reverence when viewing religious renaissance artworks, or variants of sadness in response to different subjects depicted in different artworks. Studying the full richness, blends, and gradients of aesthetic experiences requires sampling a wide range of experiences, a central focus of the present research. Studying a diverse set of artworks further enables this effort to more fully characterize the semantic space of aesthetic experiences in response to visual art. A concern of past studies is the reliance upon a limited number of artworks presented to viewers—5 to 75 pieces of art—which does not capture the great variety of cultural styles, eras, and regions that constitute the long history of visual art. Recent studies that examine a large and widely varying set of stimuli consistently show that categories of emotions traditionally seen as distinct are connected by gradients of blended experiences and expressions. For example, pure expressions of awe, surprise, and fear are interconnected by gradients of composite facial, bodily, and vocal displays that consistently convey intermediate meanings^26,27,29,31.

Two studies of visual art that have sampled a wider range of feelings—upward of 75 feeling states—have relied on factor analysis^32,33, which, for reasons we detail in the Results section, potentially mask the high dimensionality, or richer variety and blends, of aesthetic experiences. As a result, studies that used this analytical technique have usually identified no more than seven factors describing people’s aesthetic experiences, including pleasing emotions (e.g., joy, relaxation), epistemic emotions (e.g., interest, surprise), negative emotions (e.g., boredom, confusion), and prototypical aesthetic emotions (e.g., awe, being moved).

Studies have also largely ignored mental states associated with the imagination, states like mysterious or dreamy, a critical oversight given that some of the oldest visual artworks represent much more than ordinary life situations, such as a male figurine with a lion head or cave art scenes of hunters with bird and reptile features^1,34. A central function of art is to portray alternatives to reality, what is hypothetical, beyond the senses, and even not possible or true³⁵. Here, we consider the experiential aesthetic space beyond the feelings commonly elicited by everyday life events³⁶.

Finally, while artworks elicit emotions that directly affect the viewer, they are also objects for perception and reflection, fostering cognitively mediated aesthetic experiences focused on artwork qualities, such as “disorienting”, “provocative”, or “mysterious”. Previous theories highlight the need to take both an artwork-oriented approach (modeling effects of the qualities of a piece of art) and a viewer-oriented approach (modeling the felt emotions) to understand experiences with visual art³⁷. For example, recent empirical work using semantic network analysis shows that the language people use to describe their aesthetic experiences includes both artwork-descriptive features, such as beautiful and controversial, and the cognitive-affective impacts on the viewer, like enraptured and challenged^38,39. People often blend felt emotions and perceived qualities in describing art, where emotions may derive from perceived qualities (e.g., a moving landscape makes us feel moved) or diverge from them (e.g., a tragic scene makes us feel love). It remains unknown whether aesthetic qualities perceived in the artwork constitute a core aspect of the aesthetic experience, or overlap with emotional responses, as previous studies have conflated their measurement or studied them as separate processes.

In sum, most previous studies of aesthetic experience have primarily been concept driven. Namely, scientists have tended to present participants with a limited set of artworks, captured experience with a limited array of items, such as pleasantness/unpleasantness or a small number of emotion terms, have not systematically examined perceived artwork qualities, and used outdated statistical techniques. The empirical results are often low dimensional, for instance, that visual art elicits a narrow range of emotions, that are typically clustered in a few categories, including pleasure, negative emotions, epistemic emotions, and prototypical aesthetic emotions, potentially under-representing the breadth and blending of aesthetic experiences in response to visual art. The present study aspires to fill these lacunae by taking a computational approach to aesthetic experience enabled by Semantic Space Theory.

Understanding aesthetic experiences: a semantic space theory approach

Guided by Semantic Space Theory, here we take a data-driven approach to mapping aesthetic experience²⁵, departing significantly from the concept-driven methods of past studies. Semantic Space theory offers a computational approach that uses wide-ranging naturalistic stimuli and open-ended statistical techniques to capture systematic variation in emotion-related behavior across modalities (face, body, voice, neurophysiology, language²⁵). New quantitative approaches to partitioning variance enable the examination of three properties of semantic space of experience: dimensionality, distribution, and conceptualization. The dimensionality concerns the categorical structure of the semantic space or the number of distinct experiences, which determines the complexity of the semantic space, ranging from a few coherent categories (low dimensional) to many different categories (high dimensional). The distribution examines whether the boundaries between the identified dimensions are sharp or overlapping, for example, determining if emotional experiences can be categorized in clusters with clear boundaries (e.g., anger and disgust) or exist along a continuum that allows for blended experiences (e.g., an intermediate state between anger and disgust). Conceptualization concerns the nature of the concepts that most directly account for variation in how people represent their subjective experiences, for instance, whether emotion concepts, like “awe”, “fear” and “serenity”, or aesthetic qualities, like “mystical”, “intricate” and “striking”, drive the representation of aesthetic experience.

In the present investigation, first, we build upon previous studies to survey a wide array of experiences, which enables mapping the complexity of aesthetic experiences: Participants reported on their experiences in terms of 103 items, comprising several dozen states in the expanding science of emotion²⁸, variants and blends of emotions, states related to the imagination, and perceptual attributes of visual art. Second, experiences are evoked with a diverse set of stimuli, without assuming a one-to-one mapping between stimuli and specific experiences. Past studies have presented participants with up to 75 pieces of visual art. Here, we study 1457 artworks that cover a broad range of styles, eras, geographic regions, and cultures⁴⁰. Drawing on such variation is critical to mapping the structure of aesthetic experiences. Third, new statistical techniques reveal the patterns of relations between experiences. These techniques depart from univariate approaches, such as correlations, t-tests, and factor analytic and principal components analyses (PCA), and model the latent structure of large-scale, multidimensional data, while preserving dimensions of meaning that are nuanced yet reliable^25,27. We extend Semantic Space Theory to visual art, testing the following hypotheses:

Hypothesis 1: Dimensionality

The semantic space of emotion-related behavior appears to be rich, including multiple states across modalities in addition to modality-specific states, as evidenced by research on facial–bodily expression, vocal bursts, prosody, and brain patterning triggered by emotional stimuli^14,26–31. We, therefore, expect aesthetic experiences in response to visual art to be high- rather than low-dimensional, eliciting a much wider array of distinct categories of experiences than typically thought.

Hypothesis 2: Conceptualization

Felt emotions and perceived qualities are widely theorized to represent different foci of our engagement with visual art—one viewer-focused and directly affective, the other artwork-focused and cognitively mediated. Given theoretical and empirical accounts of their distinctions^37,38, we expect felt emotions and perceived qualities to be independent and integral aspects of aesthetic experiences as evidenced by their commensurate prediction of overall aesthetic appreciation of artworks.

Hypothesis 3: Distribution

Although there may be pure aesthetic experiences of clearly defined states, much of emotional response is found to be systematically blended rather than discrete. Recent studies examining a large and varied set of stimuli consistently demonstrate that traditionally distinct emotional categories are connected by gradients of blended experiences^26,27,29,31. Therefore, we expect categories of aesthetic experiences to lie along continuous gradients of experience rather than within discrete clusters.

Methods

To test these three hypotheses, we created a diverse stimulus set of visual artworks, which were rated in terms of emotions felt in response to the artworks (hereafter aesthetic emotions) and perceived artwork qualities (hereafter aesthetic qualities). We also assessed aesthetic appreciation of the artworks (liking) as an overall evaluative judgment. The experimental procedures were approved by the Institutional Review Board at the University of California, Berkeley, and all methods were performed in accordance with the relevant guidelines and regulations.

Participants

Judgments of the visual artworks were obtained using Amazon Mechanical Turk from a total of 1173 participants (540 male, 568 female, 65 other/unreported, M_age = 34.5, SD_age = 13.3). Participants were registered within the U.S. and self-reported their country of origin. We excluded data from participants whose registration location and country of origin did not match. All participants gave their informed consent and received $4.50 for their participation.

Artwork collection strategy

We assembled a diverse collection of 1457 visual artworks (paintings, reliefs, and drawings) selected from the Google Arts & Culture online platform. Departing from past studies’ concentration upon a small collection of artwork samples and informed by the crucial dependence of multidimensional analytic results on sample size⁴¹, we collected widely varying stimuli that were representative of different cultural-historical regions, art movements, and the feelings conveyed in the artworks. Our selection strategy was guided by a historical taxonomy of art⁴⁰ and aimed to cover all major art movements from the Upper Paleolithic to the Contemporary Era, all cultural-historical regions from prehistory to the modern world, and several countries across all continents. Figure 1 provides an overview of the art movements and artworks sampled from each region, and Supplementary Fig. 1 illustrates the coverage of 77 countries across continents (see also Supplementary Methods 1). Furthermore, the visual artwork samples were selected based on their likelihood to convey a wide range of aesthetic experiences found in studies of visual art^{22–24,32,33,42}. Although different eras, regions, and countries could not be equally represented due to the limited availability of earlier artworks, the current collection is the richest and most inclusive stimulus set of visual artwork samples ever studied with regard to aesthetic experiences.

Figure. 1.

Cultural-historical eras represented in the artwork collection. Regions correspond to cultural-historical units rather than present-day countries. Time ranges from the Upper Paleolithic to the Contemporary Era. Red lines represent the period covered by significant art movements per region. Blue lines represent the period covered by our artwork collection per region. Numbers within brackets represent the number of artworks in our collection per region. BCE = Before Common Era. CE = Common Era.

Aesthetic experience surveys

Participants rated each artwork in terms of aesthetic emotions, aesthetic qualities, and overall aesthetic liking. The aesthetic experiences were derived from an extensive review of past studies of aesthetic responses to visual art, consumer products, and natural and built environments. We first created a comprehensive list of aesthetic experiences previously studied, removed duplicates and synonyms, and added aesthetic experiences of interest in emotion science (see Supplementary Table 1 for a full list of the source articles of the aesthetic experience surveys).

Each participant viewed 28 randomly selected artworks for at least 8 seconds, before responding to the aesthetic experience survey questions. Participants were asked to choose the aesthetic emotions evoked by each artwork sample from 45 categories in response to the instructions “Select all emotions this artwork makes you feel”. The emotion categories were: Admiration, Adoration, Amusement, Anger, Anxiety, Awe, Boredom, Calmness, Chills, Compassion, Confusion, Connectedness, Contemplation, Craving, Cringe, Curiosity, Desire, Disgust, Distaste, Dread, Ecstasy, Empathic Pain, Entrancement, Excitement, Fascination, Fear, Feeling Moved, Gratitude, Grief, Harmony, Hope, Horror, Inspiration, Joy, Love, Nostalgia, Pride, Sadness, Serenity, Shame, Shudder, Surprise, Wonder, Tears, and Triumph.

Participants then rated the artwork in terms of aesthetic qualities from 58 categories in response to the instructions “Select all descriptions that apply to this artwork”. The categories of aesthetic qualities were: Absorbing, Absurd, Beautiful, Brooding, Cerebral, Chaotic, Charming, Cosmic, Crisp, Dark, Deviant, Disorienting, Disturbing, Dreamy, Dreary, Dynamic, Eerie, Elegant, Empowering, Expansive, Familiar, Far out, Funky, Glamorous, Grotesque, Humorous, Imposing, Impressive, Intimate, Intricate, Irreverent, Ironic, Lively, Majestic, Mystical, Mysterious, Ominous, Orderly, Ornate, Palpable, Peaceful, Perverse, Powerful, Provocative, Psychedelic, Rhythmic, Sensual, Spiritual, Strange, Striking, Sublime, Tense, Tragic, Unsettling, Vibrant, Violent, Whimsical, and Warm.

Finally, participants rated their overall aesthetic liking of each artwork in response to the question “On a 1–9 scale, how much do you like this artwork?” (1= “dislike a lot” to 9 = “like a lot”). Supplementary Fig. 2 illustrates the surveys used to collect categorical judgments of aesthetic emotions and qualities, and ratings of aesthetic liking.

Based on past estimates of reliability of observer judgment²⁹, for each visual artwork sample, we collected an average of 22.4 responses from separate participants. Participants made forced-choice categorical judgments of the 103 emotions and qualities, leading to a total of 254,519 categories selected. They also provided ratings of aesthetic liking, leading to a total of 32,604 ratings. Data were analyzed using custom code in Matlab.

Results

Dimensionality: visual art evokes a high-dimensional space of aesthetic experience (hypothesis 1)

Hypothesis 1 predicted that visual art would elicit a wide array of distinct dimensions of experience. To compute the number of distinct dimensions of aesthetic experience, previous research has applied traditional dimensionality reduction methods, such as PCA or factor analysis³³. We depart from this analytical approach and instead use a principal preserved component analysis (PPCA), because traditional dimensionality reduction methods are ill-equipped to interrogate the semantic space of subjective experience for two reasons. First, they cannot identify whether an individual category (e.g., ‘fear’) is reliably distinguished from every other category (e.g., ‘horror’) because they test the number of significant factors based mostly on correlations or covariances between judgments, without considering the reliability of reports of individual items. Second, they do not explicitly separate signal from noise variance, because they assume that high variance components contain signal, whereas low variance components contain noise—an assumption that is not always valid⁴³. In contrast, PPCA extracts linear combinations of attributes (here, judgments of aesthetic emotions and qualities), that maximally covary across two sets of data that measure the same attributes (here, randomly split judgments for each artwork). Thus, PPCA allows uncovering latent dimensions of experience that have high agreement across raters (see Supplementary Methods 2–4 and Supplementary Movie 1 for details on PPCA).

We used a generalized version of PPCA to determine the number of dimensions (eigenvalue decomposition of the across-subject covariance matrix). We kept the dimensions that were both significant (p < 0.01) and interpretable in our data, which was defined as > 1/3 of the variance being between artworks vs. between subjects. This yielded 25 dimensions that reliably differentiate the experiences evoked by a broad range of visual artworks, thereby providing support that the structure of aesthetic experiences is high-dimensional (see Fig. 2 for a list of the 25 dimensions). These results depart from assumptions that visual art simply evokes core experiences of pleasure/displeasure or a limited array of basic emotions^{21,22,24,32,33,42}.

Figure. 2.

Twenty-five dimensions capturing dimensionality. Red indicates a positive association and blue a negative association.

Some of these dimensions have been consistently reported by previous studies of visual art, such as “sadness/tragic”, “violent”, “tense/anxiety”, “chills/dread”, “distaste/disgust”, “brooding/dreary”, “boredom”, “strange/confusion”, “striking”, “amusement”, “wonder/awe”, “admiration/absorbing”, and “calmness/serenity”^{22–24,32,33,42}. However, several of the dimensions we found have not been documented in studies of visual art, including “chaotic/disorienting”, “mystical/mysterious”, “psychedelic/cosmic”, “lively/vibrant”, “sensual/desirous”, “spiritual”, “dreamy/whimsical”, “love/adoration”, “intimate/connectedness”, “nostalgia”, “intricate/ornate”, and “elegant”. Thus, aesthetic experiences elicited by visual art are widely diverse, often nuanced, and of mixed nature.

Conceptualization: aesthetic emotions and aesthetic qualities separately predict aesthetic liking (hypothesis 2)

Hypothesis 2 predicted that aesthetic emotions and aesthetic qualities will both predict aesthetic liking of artworks (will be non-redundant). To explore the conceptualization of emotional experiences in response to visual art, we compared reports of 45 aesthetic emotions, 58 aesthetic qualities, and overall aesthetic liking.

First, we tested whether the 45 aesthetic emotions and 58 aesthetic qualities were non-redundant; could one set of judgments be entirely explained by the other? Using nonlinear regression (k-nearest neighbors) we predicted the aesthetic quality judgments from the aesthetic emotion judgments, and vice versa, finding that each could explain less than half of the explainable variance in the other (r² = 0.49 [SE = 0.01, bootstrap] in emotions explained by qualities; r² = 0.44 [SE = 0.01, bootstrap] in qualities explained by emotions; Fig. 3, left panel). Thus, viewers’ experiences of aesthetic emotion and the qualities they perceive in works of visual art are related yet not redundant processes⁴⁴.

Figure. 3.

Aesthetic liking as a function of aesthetic emotions, aesthetic qualities, and the 25 dimensions. 45 aesthetic emotions and 58 aesthetic qualities have distinct but overlapping roles in predicting aesthetic liking. Along with aesthetic emotion and aesthetic quality judgments, participants reported overall aesthetic liking. Aesthetic emotion judgments were used to predict aesthetic quality judgments, and vice versa (left panel). Aesthetic emotion judgments, aesthetic quality judgments, the combination of emotion and quality judgments, and the 25 dimensions extracted from PPCA were separately used to predict mean aesthetic liking (right panel).

Second, we explored whether aesthetic liking could be predicted by aesthetic emotions, aesthetic qualities, or a combination of both using nonlinear regression. Aesthetic emotion judgments predicted liking with a correlation of r = 0.89 (SE = 0.011, bootstrap), aesthetic quality judgments with r = 0.85 (SE = 0.014, bootstrap), and all emotion and quality judgments together with r = 0.91 (SE = 0.011, bootstrap). However, by first extracting the reliable dimensions from both aesthetic emotion and aesthetic quality judgments using PPCA, then using the 25 resulting dimensions to predict aesthetic liking (a means to prevent overfitting), we achieved a correlation of r = 0.99 (SE = 0.0068, bootstrap; Fig. 3, right panel). The relative degree of these correlations demonstrates that the 25 dimensions (that combine emotions and qualities) predict aesthetic liking better than either the 45 emotion judgments, the 58 quality judgments, or all 103 judgments together. Thus, aesthetic emotions and qualities had an overlapping but distinct role in predicting participants’ liking of an artwork.

Together, these findings suggest that we conceptualize our experiences in response to art not only in terms of emotion concepts (awe, fear, serenity) but also in terms of complex aesthetic qualities of the visual art (mystical, striking, disorienting). These findings also confirm that the 25 dimensions extracted through PPCA capture the aesthetic experiences evoked by visual art with high fidelity.

Distribution: aesthetic experiences lie along continuous gradients rather than within discrete clusters (hypothesis 3)

Hypothesis 3 predicted that categories of experience evoked by visual art will lie along continuous gradients²⁸. To explore the distribution of aesthetic experiences, we generated a chromatic map of the scores of each of the 1457 visual artworks within the 25-dimensional space derived from PPCA using t-distributed stochastic neighbor embedding (t-SNE)⁴⁵. t-SNE is a method that preserves local distances between data points while separating more distinct data points by longer distances. Thus, t-SNE naturally groups visual artworks that convey similar experiences and can capture smooth, continuous variations within the 25-dimensional space despite being limited to two dimensions. The resulting map consists of spatial coordinates and colors for each artwork (see Fig. 4 and its interactive version https://barradeau.com/2021/emotions-map/). To generate coordinates for each artwork, t-SNE was applied 100 times to the data matrix, using default Matlab settings (1000 iterations, perplexity = 30, learning rate = 500, theta = 0.5), and the map with the lowest loss (Kullback–Leibler divergence) was further refined through an additional 1000 iterations for fine-tuning purposes. Of course, some information is lost in this process—this is why it is important to simultaneously view a second, independent channel of information, conveyed through the color assigned to each artwork. The color assigned to each artwork corresponds to a weighted average of the unique colors representing its top three scores on the 25 categorical judgment dimensions, unless fewer than 3 categories would be frequently selected for a given artwork.

Figure. 4.

Screenshot of interactive map visualizing distribution. t-SNE was used to visualize the 25-dimensional structure of aesthetic experiences evoked by visual art on an online interactive map. The resulting map reveals smooth gradients between specific experiences, such as a gradient from “mystical/mysterious” to “dreamy/whimsical” art.

The combined color representations and structure of the map reveal the smooth gradients that traverse many categories, such as the gradients between “love/adoration” and “calmness/serenity” or “distaste/disgust” and “chills/dread”. Replicating recent studies of experiences evoked by music and GIFs, and representations of emotion in the face, body, and prosody^26,29,31, the experiences evoked by visual art lie along continuous gradients between categories rather than within discrete clusters. These gradients are evident when we visualize smooth variations in the categorical judgment profiles of the 1457 visual artwork samples in the interactive version of the map. Critically, not all categories of aesthetic experience can be evoked simultaneously by a visual artwork sample. Aesthetic experiences are blended in a systematic rather than random fashion (e.g., no visual artwork sample was categorized as both “distasteful” and “adorable”). Thus, the feelings associated with visual art are neither entirely discrete nor arbitrarily blended but rather are distributed along specific gradients of aesthetic experience, which can be modeled using appropriate analysis methods (see Supplementary Methods 5 for details on t-SNE).

Discussion

Humans have been making visual art for tens of thousands of years, in part to share emotional and cognitive experiences that art archives. In this investigation, we took a data-driven approach and applied computational tools to test hypotheses derived from Semantic Space Theory that are germane to three long-standing questions: How many categories of aesthetic experience does visual art evoke (dimensionality)? What organizes the representation of aesthetic experiences in response to visual art, felt emotions or perceived qualities (conceptualization)? Are categories of aesthetic experience blended or divided by sharp boundaries (distribution)? Results show that aesthetic experiences in response to visual art can be mapped on a high-dimensional semantic space including 25 categories (dimensionality). People conceptualize experiences with visual art in nuanced terms, such as “intricate”, “whimsical”, “disorienting”, “tense”, and “dread”, which cover both the emotions people feel in response to the artwork and the feeling qualities they perceive in the artwork (conceptualization). Finally, people’s aesthetic experiences are often mixed, as they traverse the boundaries of different feeling states, which are bridged by gradients of blended experiences (distribution).

Our findings challenge previous conceptions of experiences associated with visual art. We demonstrate that visual art evokes upwards of 25 non-reducible categories of experience that extend to a class of imaginative states, such as “disorienting”, “dreamy”, and “whimsical”; are often profound and include states like “spirituality”, “psychedelic”, “mysterious”, and “awe”; and go beyond broad evaluative terms of pleasure or liking, providing evidence that aesthetic experiences extend beyond hedonism. Our research shows that the semantic space of aesthetic experiences is high- rather than low-dimensional, which counters the findings of several previous studies suggesting that aesthetic experiences can be reduced to a few dimensions^{22,24,32,33,42}. At the same time, our findings align with other studies of emotion-related behavior that employed data-driven methods to show high dimensionality in the semantic space of experiences in response to other art forms (e.g., music²⁷ and GIFs²⁹). Similarly, our finding that we conceptualize aesthetic experiences in terms of felt emotions and perceived qualities converges with evidence from semantic network analysis studies that map the concepts people use to describe visual art³⁸.

This broader, comprehensive taxonomy of aesthetic experiences opens new ways of thinking about and studying aesthetic experiences by putting on the map concepts that have not been studied before in the field of visual art (e.g., “psychedelic/ cosmic”, “spiritual”) and demonstrating the conceptual distinction between states (e.g., “distaste/disgust” and “strange/confusion”). The reliance on just a few categories of aesthetic experience constrains attempts to understand how these experiences are perceived and responded to by others or influence one’s behavior. For instance, a narrow focus on four types of aesthetic experiences with visual art revealed by previous research (pleasure, prototypical, epistemic, and negative aesthetic emotions) impedes progress in understanding the dynamics of aesthetic experiences and asking questions about previously uncharted aesthetic experiences: Are imaginative states experienced in the realm of visual art a training ground for advanced cognitive skills, like Theory of Mind? What neurophysiological processes underlie the broad range of aesthetic experiences in visual art? How may our responses to the subjects depicted in visual art (inequality, war, attachment, destruction of nature) influence human behavior?

To what extent do aesthetic experiences in response to visual art mirror emotional experiences in response to real-life events? As one answer to this question, in Table 1, we compare the dimensionality of the semantic spaces observed in previous studies of emotional expression and experience associated with real-life events, and aesthetic experience observed in the present study^26,29,31. A careful inspection of Table 1 reveals that many negative emotions—anger, anxiety, sadness, disgust, confusion, boredom—and positive emotions—amusement, desire, love, interest, awe—that people experience in daily life also arise as aesthetic experiences in response to visual art. Interestingly, pain, embarrassment, and shame did not emerge as distinct aesthetic experiences in response to visual art, nor did many positive states, such as elation, ecstasy, sympathy, triumph, and pride. In a speculative vein, we note that many of these states—pain, embarrassment, shame, triumph, pride—arise through self-focused cognitive processes related to social disapproval or approbation⁴⁶. Perhaps a point of art engagement is to enable experiences free of such social expectations, thereby creating space for more other-focused cognition. Moreover, unlike experiences elicited by real-life events, visual art evoked states related to the imagination, that is, what is possible but not necessarily true, hypothetical, or beyond the laws of psychology and physics³⁶. This was especially true of positive aesthetic experiences, such as “mystical/mysterious”, “psychedelic/cosmic”, “spiritual”, and “dreamy/whimsical”—fascinating mental states that go beyond the emotions of daily living.

Table 1.

Dimensionality of semantic spaces of expression/experience in real-life events and visual art.

	Vocal expression26	Facial/bodily expression31	Experience in response to GIFs29	Experience in response to visual art
Negative	Anger	Anger	Anger	Violent
	Distress	Distress	Anxiety	Tense/Anxiety
	Fear	Fear	Fear	Chills/Dread
	Sadness	Sadness	Sadness	Sadness/Tragic
	Confusion	Confusion	Confusion	Strange/Confusion
	Disgust	Disgust	Disgust	Distaste/Disgust
	Pain	Pain	Empathic Pain	–
	Embarrassment	Embarrassment	Awkwardness	–
	Contempt*	Contempt*	–	–
	Disappointment*	Disappointment*	–	–
	–	Shame	–	–
	–	Doubt	–	–
	–	–	Boredom	Boredom
	–	–	–	Brooding/Dreary
	–	–	–	Chaotic/Disorienting
Positive	Amusement	Amusement	Amusement	Amusement
	Desire	Desire	Sexual desire	Sensual/Desirous
	Adoration	Love	Adoration	Love/Adoration
			Romance	Intimate/Connectedness
	Awe	Awe	Awe	Wonder/Awe
			Admiration	Admiration/Absorbing
	Interest	Interest	Interest	Striking
	Surprise	Surprise	Surprise
	Elation	Elation	Joy	–
	Ecstasy	Ecstasy	Excitement	–
	Relief*	Relief*	Relief*	–
	Contentment*	Contentment*	Satisfaction*	–
	Realization*	Realization*	–	–
	Sympathy	Sympathy	–	–
	Triumph	Triumph	–	–
	–	Pride	–	–
	–	Concentration	–	–
	–	Contemplation	Entrancement	–
	–		Aesthetic Appreciation	–
	–	–	Nostalgia	Nostalgia
	–	–	Calmness	Calmness/Serenity
	–	–	–	Mystical/Mysterious
	–	–	–	Psychedelic/Cosmic
	–	–	–	Lively/Vibrant
	–	–	–	Spiritual
	–	–	–	Dreamy/Whimsical
	–	–	–	Intricate/Ornate
	–	–	–	Elegant

^*These emotions were not included in the current study because they did not emerge as distinct categories in previous studies of aesthetic experiences (see Supplementary Table 1).

It is interesting to consider how the rich array of aesthetic experiences maps onto a wide array of perceptual features in visual art. What are the perceptual underpinnings of these dimensions, in terms of formative elements (e.g., colors, lines, textures), semantic content (e.g., meaning, subjects), and contextual information (e.g., artist identity, human- vs. computer-made)? Inspection of the artworks that relate to each dimension suggests specific associations between experiences and perceptual features. For example, “psychedelic” experiences are often evoked by vibrant colors and optical illusions, such as Bridget Riley’s Movement in Squares; experiences of “distaste/disgust” are related to gory, perverse, and socially deviant themes, such as Frida Kahlo’s A Few Small Nips⁴⁷; “chaotic/disorienting” experiences are elicited by artworks of non-representational styles, such as Kandinsky’s abstractions, perhaps because they pose a challenge to meaning extraction⁴⁸; and “awe/wonder” experiences are evoked by ancient and religious art, such as Egyptian reliefs, or well-known masterpieces, like Botticelli’s Birth of Venus⁴⁹. Deep neural networks could partially predict aesthetic ratings from high- and low-level features in artworks⁵⁰. By applying machine learning methods to our collection of artworks, we could shed light on a broader set of features that shape aesthetic experiences.

A significant fraction of the observed responses to artworks were of profound nature, such as confusion, awe, love, dread, and spirituality—emotions that can expand our knowledge structures and deepen our moral landscape. This finding challenges the commonly adopted view that art tempers the passions, evokes mild feelings of pleasure, and elicits “soft and tender” emotions²². In contrast, our findings show that the scope of our experiences with art is vast and runs deep. Thus, experiences of moral reflection, cognitive growth, or inner transformation may be as worthy to study as feelings of mere pleasure^7,51,52. Decades of research on moral, epistemological, and self-transcendent emotions evoked by non-art entities have proved them to be strong predictors of human behavior⁵³. For instance, self-transcendent emotions such as awe are associated with the tendency to enhance the welfare of others and to prioritize the common good over self-interest⁵⁴. Furthermore, recent studies show that animated films that elicit awe enhance children’s prosocial behavior⁵⁵. However, we still do not know how artworks that evoke other profound experiences may affect our behavior, making downstream effects of aesthetic experiences fertile territory for future research.

Although our collection of artwork samples was culturally diverse, approximately two-thirds of artwork samples were from Western cultures. Also of note, our sample was participants from the United States. Would visual art elicit different experiences across cultures? Current evidence suggests that artworks reliably express emotional states characteristic of social situations interculturally. Analyses of folk songs from around the world are uncovering robust universals in the emotions expressed across at least four behavioral contexts, such as dancing, healing, soothing, and courtship^56,57. One analysis of the ancient arts from Mesoamerica that predate contact with Western Europeans found eight emotions expressed in figurines and sculptures that Western Europeans today could readily identify¹⁴. Traditional Hindu dance movements described in the Sanskrit treatise the Natyashastra express 15 emotions that are readily recognized by people from non-Hindu cultures unfamiliar with the tradition⁵⁸. Future research will need to extend the present methods to other cultures to shed light on how universal experiences elicited by visual art are⁵⁹.

The current results also raise the question of whether aesthetic experiences in response to visual art align with responses to other art media, such as music, poetry, sculpture, film, and the performing arts. Some of the dimensions we revealed here have been shown to describe experiences in response to music, such as “anxious/tense”, “amusement”, “calm”, “dreamy”, “desirous”, and “sadness”²⁷. However, a number of dimensions were unique to each medium. Perhaps, aesthetic experiences are structured around a common core of dimensions, or kinds of experience, and diversified according to the sensory modality of each art medium. This would imply that artworks combining different media (e.g., opera, dance, film) may have more diversified semantic spaces than single-media artworks (e.g., music, visual art). Answers to these questions will prove critical to research on the neurophysiological underpinnings of aesthetic experiences within a high-dimensional space⁵.

From our homes and streets to our workplaces and museums, visual artworks are omnipresent in our lives. Artworks show us the major themes of living—love, war, encounters with the Divine, death, suffering—swathed in human feeling, thereby humanizing the world before our eyes. Mapping how visual art makes us feel provides valuable insights into the everlasting human endeavor to create, seek, and savor art.

Author contributions

E.S., A.S.C., D.K., and R. C. designed research; E.S., A.S.C., R.C., and E.A. performed research; A.S.C. contributed new reagents/analytic tools and analyzed data; E.S. and D. K. wrote the paper.

Data availability

The current study uses copyrighted artworks that were made available by Google Arts & Culture. All study materials including links to the artworks used in the study, are available on OSF: https://osf.io/4ykvq/?view_only=c13b2430b42942229298c473aaa1f094. Data and analysis code necessary to reproduce the current results will be made available under protected access because the files include participant identifying information. Data and analysis code can be requested at: https://forms.gle/cufAfVcSPbWRNa5R7.

Competing interests

The authors declare no competing interests.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-024-69686-9.