In the working memory of the beholder: Art appreciation is enhanced when visual complexity is compatible with working memory

Abstract

What shapes art appreciation? Much research has focused on the importance of visual features themselves (e.g., symmetry, natural scene statistics) and of the viewer’s experience and expertise with specific artworks. However, even after taking these factors into account, there are considerable individual differences in art preferences. Our new result suggests that art preference is also influenced by the compatibility between visual properties and the characteristics of the viewer’s visual system. Specifically, we have demonstrated, using 120 artworks from diverse periods, cultures, genres and styles, that art appreciation is increased when the level of visual complexity within an artwork is compatible with the viewer’s visual working memory capacity. The result highlights the importance of the interaction between visual features and the beholder’s general visual capacity in shaping art appreciation.

Introduction

An important line of research in empirical aesthetics has been to determine how the physical features of an artwork influence preferences. Researchers have identified a variety of visual features—including symmetry, color, contrast, clarity, aspect ratio, prototypicality, image statistics, and complexity—that influence preferences for art (e.g. Berlyne, 1971; McManus et al., 1981; Graham & Field, 2007; McManus, 1980; Schloss & Palmer, 2011; Shortess et al., 2000). Despite much research suggesting the importance of visual features in aesthetic judgments, there remains both anecdotal and scientific evidence that “beauty is in the eye of the beholder.” For instance, preferences for specific types of art may depend on experience (e.g. Reber et al., 2004), expertise (e.g. Winston & Cupchik, 1992; Silvia, 2006) and emotional state (e.g. Eskine et al., 2012). However, some individual differences in aesthetic preferences appear idiosyncratic, suggesting that individual differences in visual processes may also influence art preference by modulating the appreciation of visual features. For instance, Chevrier and Delorme (1980) showed that individuals who more effectively isolated simpler figures from a complex whole preferred polygons of greater complexity. In the present study, we investigated how the beholder’s visual capacity may influence her appreciation of complexity in real artworks.

Complexity is a property present in any artwork or image, and has been shown to be associated with aesthetic preference, typically following an inverted-U function (e.g., Berlyne, 1971); that is, people tend to prefer artworks with greater visual complexity up to a certain level, at which point appreciation declines. Visual capacity, measured using a variety of visual working-memory tasks, is a heritable and relatively stable trait (e.g., Melby-Lervag & Hulme, 2013; Nagel et al., 2008) associated with the basic ability to temporarily hold visual information. We reasoned that part of enjoying an artwork derives from discovering various relationships among patterns within the artwork. A more complex artwork contains more potentially relatable patterns, but it also puts more demand on visual working memory to hold the more complex patterns to be related. It is thus plausible that an individual’s appreciation of an artwork is increased if it contains a level of visual complexity compatible with her visual working memory capacity. We therefore tested the hypothesis that individuals with higher visual working memory capacity would tend to prefer artworks of greater visual complexity, whereas individuals with lower visual working memory capacity would tend to prefer artworks of lesser visual complexity.

Methods

Participants

A group of 64 (25 female) Northwestern University undergraduate students with no formal art training participated in this study. All participants gave informed consent to participate for partial course credit, had normal or corrected-to-normal visual acuity, and were tested individually in a dimly lit room. Two of the participants were excluded from the analysis due to their unusually poor performance on the visual-object working memory task (more than 2 SDs below the mean, suggesting that they had sub-normal visual-object working memory capacity or that they made little effort in performing the visual-object working memory task).

Stimuli and Procedure

One hundred and twenty high-resolution images of fine art were selected from various online databases including the Catalog of Art Museum Images Online database. Works of art were chosen to span different art historical periods (from 15,000 BC to contemporary art) and diverse cultures (e.g. American, European, African, Indian, and Chinese), and were selected from both representational (e.g. landscapes or still life) and abstract genres (see Figure 1 for examples; see Supplemental Materials for the complete list). In order to reduce participants’ familiarity with the images, well-known artworks were not included. We verified during post-experiment debriefing that all participants either had never seen the images before or had seen them only briefly and were unable to recall any specific episodes.

Figure 1.

Example artworks used in our study. The horizontal and vertical artworks were presented in the same size in the experiment.

We measured image complexity using behavioral and computational methods. To behaviorally measure perceived complexity, we asked participants to rate all artworks on a scale from 1 (most simple) to 6 (most complex) in three separate blocks: the initial block with 50-ms upside-down presentations of the images, the next block with 500-ms upright presentations, and the final block with self-paced upright presentations. Processing of semantic information is substantially reduced when images are presented briefly in the upside-down orientation (e.g., Walther et al., 2009; Greene & Oliva, 2009). In contrast, under prolonged self-paced viewing of images, complexity judgments are likely to depend on semantic as well as visual factors. Thus, complexity ratings from the initial 50-ms-upside-down presentations are likely to primarily reflect visually perceived complexity (with little influences from semantic or affective factors), complexity ratings from the 500-ms-upright presentations are likely to include some semantic influences, and complexity ratings from the self-paced-upright presentations are likely to maximally reflect semantic influences. We complemented these behavioral measures of visual and semantic complexity with a computational measure of image complexity. We employed Rosenholtz et al.’s feature-congestion model (Rosenholtz, Li, & Nakano, 2007) because, to our knowledge, this model most closely predicts human perception of visual clutter/complexity. Further, the model allows separate assessments of clutter/complexity with respect to key visual features, luminance contrast, color, and orientation, in terms of their spatial variance evaluated across multiple scales. The use of behavioral and computational measures allowed us to elucidate what aspects of complexity interacted with working memory to influence art appreciation.

After completing complexity ratings, participants rated each artwork on its aesthetic value by considering how strongly they were moved by the artwork. We used the following instructions (adapted from Vessel et al., 2012).

“Imagine that the images you see are of artworks that may be acquired by a museum of fine art. The curator needs to know which artworks you find most compelling or moving. Make sure to consider your own individual response, not how someone else might respond to this piece. Your job is to give your gut-level response, based on how compelling, powerful, or moving you find the piece. Note: The artworks may cover the entire range from “beautiful” to “strange” or even “ugly.” Respond on the basis of how much this image “moves” you, not necessarily how much you “like” what you see, though this might be a factor you consider in your judgment.”

Participants rated each image using a scale from 1 (least compelling) to 6 (most compelling). The order of image presentation was randomized across the three complexity-rating conditions and the aesthetic-rating condition, but the same orders were used for each participant. This ensured that any effect of prior exposure on image preference (see Bornstein, 1989, for a review) would be the same across all participants.

At the end of the experimental session, visual-object working memory (VOWM) was measured using a 2-back procedure (adapted from Jaeggi et al., 2008). Each novel pattern consisted of a 3-by-3 rectangular array of achromatic squares (14° by 8.6° visual angle) with each square randomly assigned one of three luminance values (8.7, 86, or 122 cd/m²), presented against a pale-yellow (CIE[0.29,0.31], 113 cd/m²) background (Figure 2). Each pattern was presented for 500 ms, followed by the next pattern after a 2-s blank screen. Participants saw a sequence of 96 patterns, and were instructed to remember each pattern and press the spacebar whenever the current pattern was identical to the pattern they had seen two patterns previously (this occurred for 1/3 of the patterns). Participants were not instructed to use any particular strategy. The same stimulus order was used for each participant so that we could measure each participant’s VOWM with identical stimuli and procedure. The performance was measured in d’; reliability was estimated with Cronbach’s α (0.79) obtained by splitting the trials into two halves and computing d’ for each half. Note that we sought to simulate the visual-working-memory demand during art appreciation where one needs to sequentially hold and compare visual patterns across attention shifts and eye movements to discover coherent structures. Accordingly, our VOWM task measured the ability to sequentially encode, hold, and compare multiple patterns. A more typical change-detection type task (see Brady et al., 2011, for a review) would be less suitable here because it measures working memory capacity in terms of the number of items (e.g., objects, features, locations) that could be held at once.

Figure 2.

A schematic trial sequence of the visual-object working memory (VOWM) task.

All visual stimuli were displayed on a 19” Trinitron CRT monitor (at 1024 × 768 resolution and 85Hz refresh rate), and the experimental tasks were controlled using MATLAB software with Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997). Each artwork was either horizontally or vertically oriented, and it subtended 11.6° by 8.6° of visual angle at a viewing distance of 86 cm.

Results

Overall aesthetic ratings (averaged across all artworks) were not significantly correlated with VOWM (r[60]=−0.09, n.s., 95% CI[−0.33, 0.16]), providing no evidence to suggest that individuals with higher (or lower) VOWM generally appreciate artworks more (or less). Nor were overall complexity ratings (averaged across all artworks) significantly correlated with VOWM (r[60]=−0.01, n.s., 95% CI[−0.26, 0.24] for the 50-ms-upside-down condition, r[60]=0.02, n.s., 95% CI[−0.23, 0.27] for the 500-ms-upright condition, and r[60]=−0.07, n.s., 95% CI[−0.31, 0.18] for the self-paced-upright condition), providing no evidence to suggest that individuals with higher (or lower) VOWM generally perceive artworks to be more (or less) complex. However, these null results need to be interpreted with caution. We instructed participants to rate art appreciation and complexity on absolute scales (e.g., encouraging them to rate all images as non-moving or low complexity if necessary). Nevertheless, it is possible that participants still employed relative scales, spreading their ratings across the full scale. To the extent that we could not rule out this possibility, these null correlations suggesting that VOWM influences neither overall art appreciation nor overall perception of complexity needs to be interpreted with caution.

As expected we found considerable individual differences in aesthetic ratings as they shared only 8% of the variance based on the average pair-wise inter-participant correlation. This result is consistent with previous reports (Vessel & Rubin, 2010; Vessel, Starr, & Rubin, 2012). The primary goal of the current study was to test the idea that individual differences in VOWM may account for some of these large individual differences in art appreciation. In particular, we hypothesized that VOWM would systematically modulate the relationship between complexity and art appreciation. Specifically, we predicted that individuals with higher VOWM would appreciate artworks of greater complexity whereas those with lower VOWM would appreciate artworks of lower complexity.

To determine the level of complexity preferred by each participant, we evaluated the relationship between perceived complexity and art appreciation. This analysis was done within each participant because previous research suggests that, when judging the complexity of a natural scene, different individuals focus on different image features such as the number of objects or colors, the amount of clutter or open space, and the degree of organization (Oliva et al., 2004). Indeed, our participants likely focused on diverse image features when judging complexity as their complexity ratings shared only about 15% of the variance based on average pair-wise inter-participant correlations (14.4% for the 50-ms-upside-down condition, 16.0% for the 500-ms-upright condition, and 14.4% for the self-paced-upright condition). The participant-by-participant analysis allowed us to determine whether there was a consistent effect of VOWM on the appreciation of complexity despite the fact that different individuals’ visual systems may focus on different image features for computing complexity.

Several examples of the aesthetic-rating-versus-complexity-rating function are shown on the right side of Figure 3. These functions are based on the initial complexity ratings from the 50-ms-upside-down condition, indicative of visual complexity. According to our hypothesis, the peak of this function corresponding to the aesthetically preferred level of complexity should be shifted towards greater complexity for an individual with higher VOWM. We estimated this aesthetically-preferred complexity (APC) by fitting a quadratic curve to each participant’s function (yielding the mean goodness of fit, R²=0.82) and computing the center of mass (or centroid) under the fitted curve. Specifically, given f(x) represents the quadratic function fitted to the aesthetic rating and x represents the complexity rating, the aesthetically-preferred complexity is given by, $\mathit{APC}={\displaystyle {\sum }_{x=1}^{x=6}[x\times f(x)]}/{\displaystyle {\sum }_{x=1}^{x=6}f(x)}$. Note that the example functions shown in Figure 3 (right side) are generally consistent with our hypothesis; the aesthetically-preferred complexity is shifted to the right towards greater complexity for an individual with a higher VOWM (e.g., the top right plot) relative to an individual with a lower VOWM (e.g., the bottom right plot). If this association generally holds across our participants, the aesthetically-preferred complexity should be positively correlated with VOWM. This is indeed the case, r[60]=0.32 (95% CI[0.07, 0.53]), p<0.01 (Figure 3).

Figure 3.

Scatterplot showing the correlation between aesthetically-preferred complexity, APC, and visual-object working memory, VOWM (measured in d′). APC was computed for each participant as the center-of-mass of the quadratic fit to the aesthetic-rating-vs.-complexity-rating function (see text for details). On the right side examples of aesthetic-rating-vs.-complexity-rating functions for participants with high (top), medium (middle), and low (bottom) VOWM are shown along with the corresponding quadratic fits and APCs (R² indicating the goodness of fit).

Thus, we have shown that the aesthetically-preferred visual complexity (based on the initial complexity ratings with 50-ms-upside-down image presentations) is associated with VOWM. An important aspect of our hypothesis was that VOWM should be uniquely associated with the aesthetic appreciation of visual (rather than semantic) complexity. In support of this hypothesis, the aesthetically-preferred semantic complexity based on the complexity ratings with 500-ms and self-paced upright image presentations was not significantly correlated with VOWM (r[60]=0.22, n.s., 95% CI[−0.03, 0.45] for the 500-ms-upright condition, and r[60]=0.22, n.s., 95% CI[−0.03, 0.45] for the self-paced-upright condition). Further, when the aesthetically-preferred complexity values based on the three complexity ratings were simultaneously entered into a multiple-regression model to predict VOWM, only the values based on visual complexity ratings from the 50-ms upside-down presentations significantly contributed (t[60]=2.08, p<0.05, unstandardized β= 0.93, 95% CI[0.03, 1.83] for the 50-ms upside-down condition, t[60]=−0.28, n.s., unstandardized β=−0.16 95% CI[−1.34, 1.01] for the 500-ms upright condition, and t[60]=0.13, n.s., unstandardized β= 0.06 95% CI[−.94, 1.07] for the self-paced upright condition; overall regression model was marginal, R=0.34, F[3, 58]=2.567, p<0.06, likely because two of the predictors were uncorrelated with VOWM). The exclusive contribution of visual complexity cannot be attributed to differences in reliability across the three measures of complexity because the average pair-wise inter-participant correlations were equivalent for the three complexity ratings (see above) suggesting that they had equivalent reliability. Taken together, these results support the hypothesis that individuals with higher VOWM tend to prefer paintings with greater visual (rather than semantic) complexity.

To complement these analyses based on perceived complexity, we used Rosenholtz et al.’s feature-congestion model to examine the roles of image clutter/complexity with respect to luminance contrast, color, and orientation (Rosenholtz, Li, & Nakano, 2007). For each artwork, the model yielded three values of clutter/complexity, based on multi-scale spatial variance in luminance contrast, color, and orientation. We classified each value into six evenly-spaced bins so that the analysis was comparable to those based on the complexity ratings that had six levels. We computed three aesthetic-rating-versus-complexity functions for each participant, one based on complexity with respect to luminance contrast, one based on complexity with respect to color, and one based on complexity with respect to orientation. For each function, we obtained the aesthetically-preferred complexity and examined how it was associated with VOWM. The aesthetically-preferred complexity based on luminance-contrast based complexity was significantly correlated with VOWM (r[60]=0.36, p<.01, 95% CI[0.12, 0.56]; Figure 4), whereas those based on color-based complexity (r[60]=0.19, n.s., 95% CI[−0.06, 0.41]) and orientation-based complexity (r[60]=0.08, n.s., 95% CI[−0.17, 0.32]) were not. Further, when the aesthetically-preferred complexity values based on the three features were simultaneously entered into a multiple-regression model to predict VOWM, only the values based on luminance contrast significantly contributed (t[60]=2.76, p<0.01, unstandardized β=2.21, 95% CI[0.60, 3.81] for luminance contrast, t[60]=−0.33, n.s., unstandardized β=−0.24, 95% CI[−1.71, 1.22] for color, and t[60]=−0.93, n.s., unstandardized β=−0.43, 95% CI[−1.34, 0.49] for orientation; R=0.39, F(3, 58)=3.47, p<0.05, for the overall regression model). These results suggest that individuals with higher VOWM tend to prefer artworks with greater multi-scale spatial variance in luminance contrast.

Figure 4.

Scatterplot showing the correlation between aesthetically-preferred complexity, APC, and visual-object working memory, VOWM (measured in d′). APC was computed for each participant as the center-of-mass of the quadratic fit to the aesthetic-rating-vs.-complexity function, where the complexity of each artwork was computed using Rosenholtz et al’s (2007) feature-congestion model based on the luminance-contrast information (see text for details).

As aesthetically-preferred complexity is significantly associated with VOWM whether complexity is measured with ratings from 50-ms upside-down image presentations or with the feature-congestion model based on luminance contrast, a remaining question is whether the two measures of complexity reflect the same image statistics. To answer this question, we evaluated a multiple-regression model with the rating-based and feature-congestion-model-based aesthetically-preferred complexity values as two predictors of VOWM. Both significantly predicted VOWM (t[60]=2.65, p=0.01, 95% CI[0.19, 1.36] for the visual complexity ratings, and t[60]=2.85, p<0.01, 95% CI[0.45, 2.57] for the feature-congestion model; R=0.47, F(2,59)=8.45, p=0.001 for the overall regression model). These results suggest that VOWM is independently associated with multi-scale spatial variance in luminance contrast and perceived visual complexity that reflects other image factors.

Finally, of the 120 artworks we used, 35 were abstract and 85 were representational. When we computed the aesthetically-preferred complexity values separately for the two types of artworks, their correlations with VOWM were statistically equivalent for all measures of complexity (z[60]’s<.17, n.s.). Although the statistical power for this comparison is limited because we included many fewer abstract than representational artworks (as abstract artworks tended to be from a similar historical period and we attempted to include images from a wider range), we have no evidence to suggest that our results apply to only a specific type of artworks.

Taken together, these results demonstrate that VOWM is associated with appreciation for visual complexity in artworks. Individuals with higher VOWM tend to appreciate artworks with greater multi-scale spatial variance in luminance contrast and also appreciate artworks that are perceived to be of greater visual complexity.

Discussion

There are considerable individual differences in art appreciation. In our study in which a large number of artworks from diverse periods, cultures, genres and styles were used, a pair of individuals was only 8% in agreement about their aesthetic preferences. We investigated how the beholder’s general visual capacity may contribute to these large individual differences in art appreciation. We focused on visual complexity because it may add to enjoyment of art, as complex images contain numerous patterns and relationships that can be discovered and appreciated. At the same time, appreciating complex relationships may require strong visual working memory in order to assemble numerous relatable parts into a coherent interpretation. We thus hypothesized that individuals might aesthetically prefer artworks that convey the level of visual complexity broadly consistent with their strength of visual working memory. We supported this hypothesis by demonstrating that the aesthetically-preferred level of visual complexity was higher for individuals with higher VOWM and lower for those with lower VOWM.

As with any correlational results, the nature of the association needs to be carefully considered. It is possible, as we hypothesize, that higher VOWM may enable people to appreciate artworks of greater complexity. The opposite causation is unlikely because it is implausible that a tendency to prefer artworks of greater complexity would cause people to have higher VOWM capacity. What about the possibility that a third factor may contribute to both higher VOWM capacity and aesthetic preferences for greater complexity? Personality may be one such factor. For example, previous research has suggested that individuals who enjoy listening to complex music tend to be creative, tend to value aesthetic experiences broadly, and tend to consider themselves to be intelligent (Rentfrow & Gosling, 2003); at the same time, individuals who are creative and intelligent tend to have higher working memory (e.g., De Dreu, Nijstad, Baas, Wolsink, & Roskes, 2012; Luck & Vogel, 2013). Thus, greater creativity and/or higher intelligence may cause both preferences for greater image complexity and higher VOWM. Nevertheless, it is more likely that a relationship between preferences for complexity and creativity/intelligence is mediated by working memory rather than vice versa because it is more likely that working memory contributes to creativity/intelligence than vice versa. Perceptual experience is another factor that may potentially link higher VOWM to preferences for greater complexity. For example, an influential theory of aesthetic preference, the fluency theory (Reber, Schwartz, & Winkielman, 2004), posits that people tend to prefer images that are implicitly familiar to them. Hypothetically, people may tend to draw pictures and diagrams as well as organize objects in their common environments in levels of complexity compatible with their VOWM. Consequently, they may tend to frequently experience images and scenes that convey the level of complexity compatible with their VOWM. If so, people would become familiar with their VOWM-compatible levels of complexity, and the fluency theory would predict that they would prefer those familiar levels of complexity. To our knowledge, however, there is no reported evidence suggesting a systematic relationship between people’s VOWM and the complexity of their actions or environments.

It is difficult to directly demonstrate that higher VOWM causally enables people to appreciate images of greater complexity as it is difficult to experimentally manipulate VOWM. In fact, research has shown that working memory depends on genetic factors (e.g., Nagel et al., 2008) and is relatively stable over time, as training typically results in only short-lasting improvements (see Melby-Lervag & Hulme, 2013, for a meta-analysis). Nevertheless, because working memory capacity may temporarily reduce with stress (e.g., Qin et al., 2009) and more persistently reduce in old age (e.g., Burke & Barnes, 2006), one may predict that people who are stressed and the elderly may tend to prefer visually simpler artworks, though stress and aging affect many factors besides working memory. A future study might reversibly impair VOWM using a technique such as repetitive transcranial magnetic stimulation (e.g., Oliveri, Turriziani, Carlesimo, Koch, Tomaiuolo, Panella, M. et al., 2001; Mottaghy, Gangitano, Sparing, Krause, & Pascual-Leone, 2002) to see if a selective impairment in VOWM causes individuals to temporarily prefer images of lesser complexity. Although the causality remains uncertain, our results have demonstrated that higher VOWM is associated with the aesthetic appreciation of greater visual complexity both in terms of perceived complexity and complexity based on multi-scale spatial variance of luminance contrast that is not reflected in explicit ratings of visual complexity.

Research has suggested that aesthetic appreciation is increased by certain visual features in and of themselves (e.g. Berlyne, 1971; Graham & Field, 2007; McManus, 1980; McManus et al., 1981; Schloss & Palmer, 2011; Shortess et al., 2000) as well as by certain characteristics of visual processing such as fluent processing of an artwork based on the beholder’s perceptual experience (Reber et al., 2004), deeper processing of an artwork based on the beholder’s expertise with an artwork (e.g. Winston & Cupchik, 1992; Silvia, 2006) and effective parsing of an artwork based on the beholder’s ability to identify local features within a global context (Chevrier & Delorme, 1980). Others have theorized that art appreciation may depend on a match between the beholder’s cognitive or emotional goals and the semantic content of the artwork (Eskine et al., 2012; Silvia, 2005; Silvia 2006). Our results may bridge and extend these process-based and match-based views by suggesting that the aesthetic appreciation of visual complexity, a ubiquitous visual property, depends on a match between the level of visual complexity within an artwork and the level of integrative processes afforded by the beholder’s visual working memory capacity. This highlights the importance of the interactions between visual features and the characteristics of the beholder’s visual system in shaping art appreciation.