Music–color associations are mediated by emotion

Abstract

Experimental evidence demonstrates robust cross-modal matches between music and colors that are mediated by emotional associations. US and Mexican participants chose colors that were most/least consistent with 18 selections of classical orchestral music by Bach, Mozart, and Brahms. In both cultures, faster music in the major mode produced color choices that were more saturated, lighter, and yellower whereas slower, minor music produced the opposite pattern (choices that were desaturated, darker, and bluer). There were strong correlations (0.89 < r < 0.99) between the emotional associations of the music and those of the colors chosen to go with the music, supporting an emotional mediation hypothesis in both cultures. Additional experiments showed similarly robust cross-modal matches from emotionally expressive faces to colors and from music to emotionally expressive faces. These results provide further support that music-to-color associations are mediated by common emotional associations.

Researchers have attempted to identify systematic links between music and color. Perhaps the most direct connection comes from the fascinating phenomenon of music–color synesthesia (1–4). A small minority of individuals, including some distinguished artists (e.g., Kandinsky and Klee) and musicians (e.g., Scriabin and Rimsky-Korsokov) report diverse cross-modal experiences of color while hearing musical sounds (1). Scientific studies initially failed to establish general correspondences because synesthetic sound-to-color mappings appeared idiosyncratic (3).

Nonsynesthetic people also have music-to-color associations but do not actually experience colors while hearing music. Relatively low-level sound-to-color associations—e.g., higher pitches being associated with lighter colors (2, 5–7)—appear to hold for both synesthetes and nonsynesthetes (1). Reliable pitch–hue associations have been reported in children (8) although these effects were probably due to lightness, where spectral yellow and green (lightest) were associated with higher pitches, red and orange (midlightness) with midlevel pitches, and blue and violet (darkest) with lower pitches. There is evidence for other low-level auditory–visual associations such as timbre–saturation (9), loudness–brightness (7), and pitch–size (10, 11) [Spence (12)].

Other studies have investigated music–color correspondences at a higher level. Bresin found that music in the major mode was associated with lighter colors than music in the minor mode (13), but only two melodies were studied. Sebba reported that students used warmer, more saturated, lighter, and more highly contrasting colors in creating images while listening to a major Mozart selection than did students listening to a minor Albinoni selection (14). Again, only two musical selections were used, and students chose the musical selections rather than being randomly assigned, so students who are more inclined to choose major music may merely be more inclined to create images with warmer, more saturated, lighter, and more highly contrasting colors. Barbiere et al. found that “gray” was associated with sadder music whereas “red,” “yellow,” “green,” and “blue” were associated with happier music (15), but only four musical selections were studied and, more importantly, no actual colors were presented in the selection task, but only words.

How might music-to-color associations occur in nonsynesthetes? The two most plausible hypotheses are (i) the direct connection hypothesis that there are direct, unmediated associations between colors and musical sounds (3, 9, 16) and (ii) the emotional mediation hypothesis that color and music are linked through shared emotional associations (3, 8, 13–15, 17–19).* Although some empirical support has been claimed for hypothesis ii, the small sets of colors and/or descriptors (15, 17, 18), the small set of musical selections (13–15, 18), and the potential relevance of cultural comparisons preclude firm conclusions.

The present results demonstrate clear, robust connections between music and color that are widely shared across both individuals and cultures. They differ from previous findings in that they (i) associate specific dimensions of color (saturation, lightness, and yellowness-blueness) with specific high-level musical dimensions (tempo and mode), (ii) show clear evidence of mediation by emotional dimensions (happy–sad and angry–calm), and (iii) demonstrate a strong cultural invariance across US and Mexican participants. Specifically, people’s experiences of 18 brief, classical orchestral selections that varied in tempo (slow/medium/fast), mode (major/minor), and composer (Bach/Mozart/Brahms) are closely associated with the saturation, lightness, and yellowness of the colors they chose to “go best” with the music.

Experiment 1 Color, Music, and Emotion

We studied music–color associations in nonsynesthetes in the United States and Mexico for 18 classical orchestral selections using the 37 colors of the Berkeley Color Project (20, 21). The colors (Fig. 1) were chosen to vary systematically in hue, saturation, and lightness. The eight hues included four unique hues (red/green/blue/yellow), plus four intermediate hues with approximately equal amounts of adjacent unique hues (orange/chartreuse/cyan/purple). These hues were sampled at four cuts (saturation/lightness levels), with the saturated (S) colors being the maximally saturated colors that our monitor could produce, muted (M) colors being approximately halfway between each S-color and neutral-gray, light (L) colors being approximately halfway between each S-color and white, and dark (D) colors being approximately halfway between each S-color and black in Munsell space (Table S1). We also included white, black, and the three grays whose lightnesses were approximately the average lightness of the L-, M- (and S-), and D-colors (Fig. 1 has 38 colored squares because the gray for the S and M cut is the same).

Fig. 1.

The display of 37 colors that was presented during the music–color association task: red, orange, yellow, chartreuse, green, cyan, blue, and purple at four different lightness-saturation levels (saturated, light, muted, and dark), plus three grays, white, and black. (See text and Table S1 for details.) The gray corresponding to the saturated (Top Left) and muted (bottom right) cuts were the same because the saturated and muted cuts had similar mean lightnesses.

Participants listened to 18 50-s samples of orchestral music that varied in tempo (slow/medium/fast) and mode (major/minor) (Table S2) while viewing the 37-color array (Fig. 1) (tempi for different composers were not the same, however: e.g., the fast selection for Brahms was significantly slower than the fast selections for both Bach and Mozart). They were asked to choose the five colors, in order, that were most consistent with the music and then the five colors, in order, that were least consistent with the music (subsequent analyses have shown that essentially the same results are obtained if only the three most/least consistent colors are chosen).

We also measured participants’ emotional associations separately for each color and each musical selection by asking them to rate how strongly associated it was with each of eight emotional descriptors that were relevant for both music and color—happy, sad, angry, calm, strong, weak, lively, and dreary—using a line-mark ratings scale that ranged from −100 to +100. They also performed four color–appearance ratings of each of the 37 colors for the degree to which they were red-green (R/G), yellow-blue (Y/B), light-dark (L/D), and saturated-unsaturated (S/U) (20).

We examined the relation between the dimensional variations in our music samples (i.e., tempo and mode) and the dimensional structure of color by computing a music–color association (MCA) score for each of the 18 musical selections along each of four color appearance dimensions (R/G, Y/B, L/D, and S/U) as rated by the same participants. Conceptually, the MCA score for a given musical selection m on a given dimension d (say, saturation) is a linearly weighted average of the saturation ratings of the five colors chosen as most consistent with that music (C_d,m) minus an analogous weighted average of the saturation ratings of the five colors chosen to be least consistent with that music (I_d,m):

where c_j,d,m represents the value along dimension d of the jth color picked as most consistent with musical selection m, where j ranges from 1 to 5, and i_j,d,m represents the corresponding value of the jth color picked as most inconsistent with musical selection m. The MCA for a given selection on dimension d is then,

These values were computed for each subject in the United States and Mexico for each of the 18 musical selections and each of the four color appearance dimensions. Fig. 2 shows average MCA values for US participants as a function of tempo and mode, averaged over composers, for each of the four color dimensions. Separate ANOVAs for each color dimension showed that, averaged over composers, faster tempi were generally associated with more saturated, lighter, and yellower (warmer) colors [F(2,94) = 96.13, 32.54, 62.82, P < 0.001, respectively]. The same was true for major (vs. minor) mode [F(1,47) = 96.13, 51.31, 29.71, P < 0.001, respectively]. By the same token, slower tempi and music in the minor mode were associated with less saturated, darker/cooler colors. For major (but not minor) music, slow tempi were associated with greener colors than medium or faster tempi [F(1,47) = 69.61, 62.87 P < 0.001, respectively]. Main effects of composer were also present, with Brahms’s music being associated with less saturated, darker, bluer colors than Bach’s and Mozart’s music [F(2,94) = 11.55, 49.58, 16.97, P < 0.001, respectively], with Bach’s and Mozart’s music not differing from each other.

Fig. 2.

Dimensional color associations for music at slow/medium/fast tempi and in major/minor mode for the saturation, lightness, yellowness-blueness, and redness-greenness of colors chosen as most/least consistent with the music, as computed using the Music–Color Association index (MCA) defined by Eqs. 1–3. (Error bars represent SEMs.)

Reliable differences between composers were also present for many of the two- and three-way interactions (Fig. S1). The trends that are mentioned above varied somewhat for different composers, especially in the case of Brahms. No doubt many of these differences are due to the particular musical selections we chose, as they varied widely in musical features other than tempo and major/minor mode, such as loudness, orchestration and timbre, pitch height, melodic structure, harmonic structure, note rate (as opposed to tempo), dynamic range (loudness contrast), plus the enormous stylistic differences among music from the Baroque (Bach), Classical (Mozart), and Romantic (Brahms) eras.

Cultural dependency was assessed by comparing the results of exactly the same experiment for 49 Mexican participants at the University of Guadalajara. The pattern of results for tempo, mode, and composer were remarkably similar (compare Fig. 2 with Fig. S2), as indexed by very high correlations between cultures for each color appearance dimension: +0.88 for R/G, +0.96 for Y/B, +0.97 for L/D, and +0.95 for S/U. ANOVAs including both cultural datasets were performed for each of these dimensions, with culture (United States/Mexico) as a between-subject factor and composers, tempi, and modes as within-subject factors, with a Bonferroni-corrected critical alpha of 0.01 to adjust for multiple comparisons for the four color-appearance dimensions. The results revealed no interactions between culture and tempo for R/G, Y/B, L/D, and S/U [F(2,190) = 0.20, 0.95, 1.89, 2.33, P > 0.01, respectively], mode [F(1,95) = 1.36, 0.00, 5.60, 0.00, P > 0.01, respectively], or tempo × mode (Fs < 1.02). There were minor main effects of culture for the different color dimensions, for which Mexican participants chose somewhat lighter, yellower, and greener colors than US participants did [F(1,95) = 5.44, 8.07, 7.78, P < 0.05, 0.01, 0.01, respectively]. There were also minor interactions involving culture and composer for the R/G and S/U dimensions, but the patterns and ordering of the conditions were the same in both countries.

The emotional mediation hypothesis suggests that, as people listen to the music, they have emotional responses while listening to music and then pick colors with similar emotional content. For example, Bach’s fast-paced dance in F-major might convey a happy, energetic emotion, consistent with happy, energetic colors (e.g., saturated-yellow and light-yellow) whereas Brahms’ slow-paced Adagio in C-minor might convey sad, depressive feelings, consistent with sad, depressive colors, (e.g., black, dark-gray, and dark-cyan). This hypothesis was assessed by analyzing the emotional ratings provided separately for the colors and musical selections.

We computed the average ratings of each color for each of the eight emotional adjectives (Fig. S3). The ratings of three conceptually opposite pairs of emotional terms were nearly perfect inverses of each other in both the US and Mexican data, as indexed by high negative correlations between the ratings of the 37 colors for each of these pairs: happy/sad (r = −0.94, −0.94), strong/weak (r = −0.97, −0.87), and lively/dreary (r = −0.99, −0.95). Angry and calm, however, were less than fully opposite (r = −0.69, −0.13, P < 0.001, P > 0.05, respectively). Corresponding analyses of the average emotional ratings for the 18 musical selections (Fig. S4) revealed the same pattern: happy/sad (r = −0.98, −0.96), strong/weak (r = −0.99, −0.97), and lively/dreary (r = −0.95, −0.98) are effectively single, bipolar dimensions, but angry/calm is not (r = −0.51, −0.31, P < 0.05, P > 0.05, respectively).

We then performed separate emotional multidimensional scalings (MDSs) of the colors and the music. First, we computed the correlation of average emotional association ratings for each pair of colors over the eight emotional terms for the US data to use as input to mdscale (www.mathworks.com/help/stats/mdscale.html), an MDS program. The emotional ratings of S-yellow and S-orange, for example, are very similar (r = +0.98) because both are rated quite high on happiness, strength, and liveliness, and low on sadness, weakness, dreariness, and calmness. The resulting 37 × 37 correlation matrix served as an inverse ordinal distance measure in the scaled space.

We obtained a good 2D solution (Fig. 3A), accounting for 95% of the variance (stress = 0.08). Consistent with previous studies of color–emotion associations (22–24), the two emotional dimensions were interpretable as positive/negative valence (happy/sad) and high/low potency (strong/weak). Average happy/sad ratings were correlated +0.93 with the coordinate values of the colors along dimension 1 in Fig. 3, and strong/weak ratings were correlated +0.93 with the coordinate values of the colors along dimension 2. Similar dimensions are frequently obtained in MDS solutions of many other emotional stimuli (22, 23, 25, 26). We obtained similar results when the corresponding Mexican ratings were scaled in the same way.

Fig. 3.

Best-fitting solutions for emotional multidimensional scaling of (A) colors and (B) music based on emotion similarity as determined by the correlation between all possible pairs of colors (A) and musical selections (B) for ratings on eight emotional terms: happy, sad, angry, calm, strong, weak, lively, and dreary. Dimensions are labeled by the emotional terms whose ratings best fit the projections of the colors (A) and musical selections (B).

The analogous emotional MDS of the US data on the 18 musical selections yielded a similarly good 2D solution (Fig. 3B), accounting for 99% of the variance (stress = 0.02). The same two dimensions characterize the solution: positive/negative valence (happy/sad) and high/low potency (strong/weak). Happy/sad ratings correlated +0.96 with the coordinate values of the colors along dimension 1, and strong/weak ratings correlated +0.96 with the coordinate values of the colors along dimension 2. Again, similar results were obtained when the corresponding Mexican ratings were scaled in the same way.

The fact that the same two emotional dimensions emerged in separate MDS solutions for colors and musical selections supports the emotional mediation hypothesis. A more direct test can be formulated in terms of correlational analyses. In particular, emotional mediation implies that there should be a high correlation between the emotional ratings of the musical selections and the emotional ratings of the colors people chose as consistent/inconsistent with those selections. We computed the music–color association (MCA_e,m) for each emotional dimension e and musical selection m that is entirely analogous to the music–color association (MCA_d,m) defined by Eqs. 1–3, except that the emotional dimension e replaces color dimension d. This measure provided a combined, weighted index of, say, the happiness/sadness of the 10 colors chosen as being most/least consistent with each musical selection.

We then computed the correlation between the average rating for each emotional dimension for each of the 18 musical selections versus the MCA_e,m values of the colors chosen as positively/negatively associated with each musical selection. Fig. 4 shows the four scatter plots of these correlations for each (approximately) polar pair of emotional terms: happy/sad, strong/weak, lively/dreary, and angry/calm. The strength of these correlations is equally remarkable in the US and Mexican data: happy/sad (+0.97, +0.97), lively/dreary (+0.99, +0.96), strong/weak (+0.96, +0.96), and angry/calm (+0.89, +0.93), respectively. These results are clearly consistent with the emotional mediation hypothesis (3, 8, 13–15, 17–19). Because the evidence is purely correlational, however, it does not establish that the relation is causally mediated by emotion or in which direction it operates. It is also possible that there are redundant, direct associations between color and music. These issues are addressed in part by experiments 2 and 3.

Fig. 4.

Scatterplots and correlations between the emotional ratings of the 18 musical selections (x axis) and the emotional associations of the colors chosen as most/least consistent with them (y axis) for the four emotional dimensions studied: happy/sad, angry/calm, strong/weak, and lively/dreary.

Experiment 2 Color, Faces, and Emotion

If the strong links between colors and musical selections measured in experiment 1 are mediated by common emotional associations, then analogous results should emerge if we asked participants to pick the colors that are most/least consistent with any other set of stimuli that are strongly associated with the same emotional dimensions of happy/sad, angry/calm, and strong/weak. Perhaps the clearest and most transparently emotional visual stimuli that meet this criterion are images of individual human faces expressing different emotions to different degrees. The emotional mediation hypothesis implies that, when asked to match colors to facial expressions, the correlations between the emotional ratings of the faces and the emotional ratings of the colors chosen to go well/poorly with the faces will be quite high, analogous to the results of experiment 1 for color and music. Experiment 2 tests this prediction by repeating experiment 1’s cross-modal mapping task with emotional faces simply replacing the musical selections. The 14 face images we used came from Marian and Shimamura’s (27) gray-scale, morphed versions of the NimStim set (28), including male and female expressions of calm (neutral), 50% happy, 100% happy, 50% angry, 100% angry, 50% sad, and 100% sad (Fig. 5).

Fig. 5.

The 14 happy, sad, and angry emotionally expressive faces of a male and female in experiment 2. Participants chose the five colors that were most/least consistent with each face from the 37 colors shown in Fig 1. The 50% emotional faces were morphed halfway between the 0% (neutral) and corresponding 100% emotional face (27). Face images are from the NimStim database (28).

We calculated the face–color association measure (FCA_d,f) for each color dimension, d, and each face, f, analogously to the MCA_d,m measure for music, except that the 14 emotional faces were substituted for the 18 musical selections (Eqs. 1–3). The results are plotted in Fig. 6 for the four color appearance dimensions of S/U, L/D, Y/B, and R/G.

Fig. 6.

Average color appearance values of the colors chosen as going best/worst with emotional faces (FCA scores) are plotted as a function of the degree of emotionality (0%, 50%, and 100%) for saturation, lightness, yellowness-blueness, and redness-greenness. (Error bars represent SEMs.)

The results (Fig. 6) indicate that neutral/calm faces were paired with desaturated, moderately light colors that were slightly cool (e.g., low saturation blues and greens), sad faces were paired with darker, desaturated, cool colors (e.g., dark-bluish or dark-greenish grays), happy faces were paired with light, highly saturated, warm colors (e.g., vivid and pastel yellows, oranges, and reds), and angry faces were paired with dark, somewhat saturated reddish colors (e.g., dark reds) (see SI Text for statistical analyses of these data).

To examine the emotional mediation hypothesis, we computed the correlations between the ratings of the three emotional dimensions for the 14 faces and for the colors selected as most/least consistent with the corresponding faces using the color–emotion association for faces (FCA_e,f) for emotional dimension e and face f (Eqs. 1–3). These correlations were again strikingly high: +0.97 for happy/sad, +0.94 for angry/calm, and +0.85 for weak/strong. Although there is clearly a direct association between certain colors and certain facial expressions of emotions due to correlated changes in skin color (e.g., faces typically become redder when expressing extreme anger), other associations are not of this type. Faces do not typically get yellower or lighter when people become happier nor darker and bluer when they become sadder. The fact that the same overall pattern of correlations emerges further supports the emotional mediation hypothesis of the coupling between dimensions of color and music in experiment 1.

Experiment 3 Music, Faces, and Emotion

If the emotional mediation hypothesis is correct, as the results of experiments 1 and 2 suggest, then any two sets of stimuli that have strong associations with common emotional content should be systematically relatable. Because experiment 1 showed that music and color can be related through emotional associations and experiment 2 showed that color and faces can be related through the same emotional associations, it follows that music and faces should be analogously relatable. We tested this prediction in experiment 3 by playing the 18 musical selections from experiment 1 and asking participants to choose the two most/least consistent faces among a set of 13 faces that varied in the degree to which they expressed the emotions of happiness, sadness, anger, and calmness. We used the seven female faces from experiment 2 augmented by 25%- and 75%-morphed versions of the same happy, sad, and angry faces.

We calculated the music–face association (MFA_e,m) for each emotion, e, and each musical selection, m, analogous to the MCA_d,m measure for music–color associations (Eqs. 1–3), except that the 13 faces were substituted for the 37 colors, the emotional dimensions of happy, sad, and angry were substituted for the four color appearance dimensions, and only two faces were selected as being most/least consistent with each musical selection. Analogous to results from experiments 1 and 2, there were strong correlations between the emotional content of the music and the emotional content of the faces that were chosen as consistent/inconsistent with the music for happy/sad (r = +0.97), dreary/lively (r = +0.94), and weak/strong (r = +0.88), with weaker but reliable correlations for angry/calm (r = +0.51). These findings provide additional support for the emotional mediation hypothesis.

General Discussion

The results of experiment 1 showed remarkably strong associations between high-level dimensions of classical orchestral music by Bach, Mozart, and Brahms and appearance dimensions of color in nonsynesthetic participants. In particular, faster music in the major mode was generally associated with more saturated, lighter, yellower colors, whereas slower music in the minor mode was associated with more desaturated (grayer), darker, bluer colors. Three further results showed that these music-to-color associations were highly consistent with the emotional mediation hypothesis. First, separate 2D emotional MDSs of colors and musical selections were well explained by the same two emotional dimensions: happy/sad (positive/negative valence) and strong/weak (high/low potency). Second, the correlations between emotional ratings for the musical selections and a linearly weighted average of corresponding emotional ratings for the colors chosen to go best/worst with the music were remarkably high: happy/sad (+0.97), lively/dreary (+0.99), strong/weak (+0.96), and angry/calm (+0.89). Third, all of the previously mentioned results were essentially the same for participant samples in the United States and Mexico, as expected given prior results that color–emotion (24) and music–emotion associations may well be universal (29, 30). It is as yet unclear whether similar results will be evident in cultures that use nonwestern musical scales and structures.

The results of experiments 2 and 3 further support emotional mediation. In experiment 2, people produced analogous effects when choosing colors that are most/least consistent with emotionally expressive faces. First, there were highly reliable associations between emotionally expressive faces and particular dimensions of color appearance. Second, face–color associations produced similarly high correlations between the emotional content of the faces and the emotional associations of the colors chosen to be most/least consistent with them: 0.97 (happy/sad), 0.94 (angry/calm), and 0.85 (weak/strong). Experiment 3 showed analogous patterns of results when people made associations between the musical selections from experiment 1 and an augmented set of the emotional faces from experiment 2. For example, faster, major music was systematically associated with happier faces, and slower, minor music with sadder faces. Again, correlations between the emotional ratings of the musical selections and the weighted average of emotional ratings for the faces chosen as most/least consistent with the music were remarkably high: 0.97 (happy/sad), 0.94 (lively/dreary), and 0.88 (strong/weak).

The fact that the pattern of cross-domain matching results is so clearly and consistently related to emotion in all three studies provides strong support for emotional mediation as a mechanism of at least some cross-modal associations. There is also evidence for emotional mediation in fragrance-to-color associations (31). It does not rule out the possibility that there might also be direct or other sorts of associations. Spence, for example, argues that three types of cross-modal matching mechanisms have been established empirically: structural correspondences based on common neural coding, statistical correspondences based on natural covariation of cross-modal attributes, and semantically mediated correspondences based on common descriptive terminology (12). Although he mentions affectively mediated correspondences as an additional possibility, he does not endorse them as having a strong empirical basis. We believe that the foregoing results provide clear evidence of cross-modal correspondences based on emotion.

Even though the present evidence does not rule out cross-modal sensory associations based on covariation, theoretical parsimony suggests that they are unnecessary. Moreover, it is generally unclear what the basis for such direct associations might be, except for a very few cases, and those are actually related to emotional factors. It is realistic for angry faces to be associated with redder colors (experiment 2) because angry faces do tend to be redder than happy or sad faces due to increased blood flow, but happy faces are not yellower nor sad faces bluer. Neither are there any obvious reasons why fast, major music should be systematically associated with saturated, light, yellowish colors, or slow, minor music with desaturated, dark, bluish colors.

A great deal remains unknown about the nature and cause of these cross-modal associations. Might they be accounted for by some more abstract nonemotional features such as the three dimensions of the semantic differential (25): evaluation (good/bad), potency (strong/weak), and activity (active/passive)? Do people actually experience the emotions they associate with music and/or colors or are they merely cognitive associations? Will the correlational evidence of emotional mediation diminish when people’s affective experiences are compromised, either due to neurological deficits or pharmacological interventions? Do the results generalize to forms other than classical orchestral music and to cultures in which nonwestern music is the norm? What factors produce the emotional associations to music and what factors produce those to colors? And do music–color synesthetes have the same associations between music and colors as nonsynesthetes?

Materials and Methods

Participants.

There were 48 US and 49 Mexican participants in experiment 1, 24 US participants in experiment 2, and 16 US participants in experiment 3. All factors of each experiment were fully orthogonal, within-subject designs except culture (US/Mexican), which was between-subjects. All participants had normal color vision (screened using the Dvorine Pseudo-Isochromatic Plates) and gave informed consent. We did not explicitly screen for synethesia but interviewed participants about their experiences of colors in their daily lives, and none reported experiencing color while listening to music. We note that music–color synesthesia is very rare among the general population (0.2%) (32). The institutional review boards at the University of California, Berkeley (Committee for the Protection of Human Subjects) and at the University of Guadalajara (Academic Secretary Dr. Francisco Javier González Madariaga) approved the experimental protocol.

Experiment 1 Design, Displays, and Procedure.

The three tasks were completed by all 97 participants on different days, embedded in a battery of 32 tasks, so it was not obvious that they were related. Unless otherwise specified, all designs, displays, and procedures were the same for US and Mexican participants. The 37 colors (Fig. 1) are specified in Table S1 and the 18 musical selections in Table S2. Participants viewed the calibrated computer monitor (an 18-inch diagonal Dell M990, 1024 × 768 pixel resolution, in the United States and an 18-inch iMac monitor (Apple), 1680 × 1050 pixels, in Mexico) from ∼70 cm. The background was always neutral gray [International Commission on Illumination (CIE) x = 0.312, y = 0.318, Y = 19.26]. Participants listened to the music through speakers connected to the same Dell and iMac computers in the United States and Mexico, respectively.

Music–color associations.

Participants saw all 37 colors (always displayed as shown in Fig. 1) while listening to each of the 18 musical selections. [Data were subsequently collected from 30 different participants when the color array was rotated by 180°, thus reversing both up-down and left/right color positions. The results were nearly identical to the data reported above (r = 0.94, 0.94, 0.96, and 0.83 for saturation, light/dark, yellow/blue, and red/green dimensions, respectively). Between-subjects ANOVAs comparing them with the music-to-color associations reported above showed no significant main effects or interactions after Bonferroni corrections for multiple comparisons.] They initially heard each selection for 50 s, while they considered which colors were most consistent (went best) and most inconsistent (went worst) with the music. They were then prompted to click on the five best colors, with each color disappearing when clicked. All colored squares then reappeared, and participants clicked on the five worst colors. Each musical selection looped continually during the music–color association task, with fade-in/fade-out transitions at the start and end.

Color–emotion associations.

Participants rated each of the 37 colored squares (100 × 100 pixels) singly in random order for consistency with each of the eight emotional terms (happy, sad, angry, calm, lively, dreary, active, passive, strong, and weak) by sliding a cursor along a continuous scale and clicking to record their final position. The to-be-rated emotional term was centered above the square, and the 400-pixel response scale was centered 250 pixels below the square, with the left, right, and center points labeled “inconsistent,” “consistent,” and “neutral,” respectively. Responses were scaled from −100 to +100. Trials were blocked by emotional term, with block order randomized. Before each block, participants anchored the end points for the particular emotional term by viewing all colors simultaneously and considering which color was most/least consistent with that term. Trials were separated by 500-ms intertrial intervals.

Music–emotion associations.

Participants heard each musical selection in random order and rated its consistency with each of the same eight emotional terms using the same procedure as for color–emotion associations.

Experiment 2 Design, Displays, and Procedure.

In all respects not mentioned below, the methods of experiment 2 were the same as experiment 1. The 14 faces were the orthogonal combination of 2 genders (male/female) × 7 facial expressions created from the orthogonal combination of 3 emotions (happy, sad, and angry) with two levels of intensity (100% and 50% morphed) plus the single calm/neutral expression (27, 28). The colors were the 37 colors described above. The colors and faces were rated along three bipolar emotional dimensions: happy/sad, angry/calm, and strong/weak. Displays were programmed and presented using Presentation (www.neurobs.com) and displayed on a 19-inch Dell E196FPf monitor (resolution 1040 × 768 pixels). All three tasks were completed by the same participants during one experimental session in the following order.

Face–color associations.

Each display contained one face (6.7 inches × 8.3 inches) on the left of the screen with the display of 37 colors (Fig. 1) on the right of the screen. Participants were asked to click (in order) on the five colors that were most/least consistent with the face, as specified in experiment 1.

Color–emotion associations.

This task was the same as in experiment 1, except that participants rated each color along the following 3 bipolar dimensions: happy/sad, angry/calm, strong/weak.

Face–emotion associations.

Participants were presented with each face, one at a time, centered on the screen, which they rated along the same three dimensions as for colors. The color appearance rating data were those reported in experiment 1.

Experiment 3 Design, Displays, and Procedure.

In all respects not mentioned below, the methods of experiment 3 are the same as those of experiments 1 and 2. The 13 faces included the 7 female faces from experiment 2, plus morphed faces with 25% and 75% emotionality for each emotion (happy, sad, and angry). The music was the same as in experiment 1. Displays were presented on an 18-inch iMac (Apple) monitor (1680 × 1050 pixels) using Presentation software (www.neurobs.com). Participants completed all three tasks within the same experimental session in the following order: music–face associations, face–emotion associations, and music–emotion associations.

Music–face associations.

This task was the same as the music–color association task in experiment 1, except that participants were presented with the array of 13 faces (each was 0.7 inches × 0.9 inches), and their task was to click first on the two faces that were most consistent with the music, followed by the two faces that were most inconsistent with the music.

Face–emotion associations.

This task was the same as the color–emotion task in experiment 1 except that participants rated how consistent each face was with the emotional terms of experiment 1.

Music–emotion associations.

This task was the same as the music–emotion task in experiment 1.