Individual aesthetic preferences for faces are shaped mostly by environments, not genes

Summary

Although certain characteristics of human faces are broadly considered more attractive (e.g. symmetry, averageness), people also routinely disagree with each other on the relative attractiveness of faces. That is, to some significant degree, beauty is in the “eye of the beholder”. Here, we investigate the origins of these individual differences in face preferences using a twin design, allowing us to estimate the relative contributions of genetic and environmental variation to individual face attractiveness judgments or face preferences. We first show that individual face preferences (IP) can be reliably measured and are readily dissociable from other types of attractiveness judgments (e.g. judgments of scenes, objects). Next we show that individual face preferences result primarily from environments that are unique to each individual. This is in striking contrast to individual differences in face identity recognition, which result primarily from variations in genes [1]. We thus complete an etiological double dissociation between two core domains of social perception (judgments of identity versus attractiveness) within the same visual stimulus (the face). At the same time, we provide an example, rare in behavioral genetics, of a reliably and objectively measured behavioral characteristic where variations are shaped mostly by the environment. The large impact of experience on individual face preferences provides a novel window into the evolution and architecture of the social brain, while lending new empirical support to the long-standing claim that environments shape individual notions of what is attractive.

Results and Discussion

To understand the origins of individual preferences for certain faces, we collected face attractiveness ratings from 547 identical twin pairs (monozygotic or MZ twins) and 214 same-sex nonidentical twin pairs (dizygotic or DZ twins) (see Fig 1A). Consistent with the large literature regarding universal aspects of face preferences and attractiveness [2–7], there was substantial agreement across participants about which faces were more and less attractive: mean ratings from male participants, for example, correlated at ceiling with mean face ratings from female participants, across both female faces (r(100) = 0.99; 102 female faces) and male faces (r(96) = 0.99; 98 male faces; see Fig 1B–C). Average aesthetic preferences, however, can mask substantial individual differences [8–10] (see Fig 1D–E). Selecting two participants at random produced an average of only 48% agreement (and 52% disagreement) in face preferences (see Fig 1F), even after removing apparent disagreements that could be explained away as self-inconsistency (Supplemental Information). This estimate is consistent with previous literature [11, 12] as well as with the everyday experience that on the one hand, fashion models can “make a fortune with their good looks,” while on the other hand, friends can “endlessly debate about who is attractive and who is not” [11]. To capture individual differences in face preferences, we first estimated the proportion of variation in each participant’s ratings that was unique to that participant and not explained by average ratings, based on the correlation between each participant’s ratings and the average person’s ratings [10–13]. For our subsequent analyses we transformed correlations between individual participant and average face ratings to z-scores using Fisher’s r-to-z transformation to remove the inherent skew in correlation values [14] and then removed variance attributable to differences in response consistency or intra-individual variability (see Experimental Procedures). This procedure yielded reliable individual preference scores (IP scores; see Fig S1A) for each participant, with larger numbers indicating greater agreement between the participant’s ratings and the mean ratings for each of the faces. These IP scores for faces were only modestly correlated with IP scores for abstract objects (r(2197) = 0.07, 95% CI: 0.03 – 0.11) and scenes (r(2197) = 0.28, 95% CI: 0.24 – 0.32; see Fig S1B) [13]. To get at face-specific preferences, we regressed out both object and scene IP. These face-specific IP scores remained highly reliable (split-half reliability = 0.88; test-retest reliability = 0.75; see Fig S1C, Experimental Procedures, and Supplemental Information). We used standardized (z-scored) face-specific IP scores (hereafter called face IP scores) for all subsequent analyses.

Figure 1. Common and individual preferences for faces.

(A) To calculate common and individual preferences for faces, participants rated the attractiveness of 200 faces. The mean ratings by male participants were very highly correlated with the mean ratings by female participants when rating both (B) female faces and (C) male faces, indicating face preferences held in common across participants. When looking at the correlation between individual participant ratings and mean ratings, however, individual differences in face preferences become evident. In (D) and (E), data are displayed from two individual participants. The first participant (D) shows very high agreement with mean ratings. The second participant (E) had lower agreement with mean ratings. Notably, these two participants had similar response consistency of face ratings (inset graph with blue dots) based on a subset of faces (60/200) that were rated a second time. By comparing inter-participant correlations with intra-participant correlations for the 60 faces that were rated twice (see Supplemental Information: Individual vs. Common Preferences), we estimate that 48% of the variation in face ratings can be explained by common preferences (those that overlap between two typical individuals), with the remaining 52% of the variation in face ratings attributable to individual face preferences (those that do not overlap between two typical individuals). In all plots, each dot represents one face stimulus and Pearson correlation coefficients are given in the upper-left or lower-right corners. Individual participant ratings in (D) and (E) are jittered for visibility. See also Table S1, and Reliability Analysis in Supplemental Information.

Next, we estimated the contributions of genetic and environmental factors to face IP by comparing the correlation of face IP scores among MZ twins with the correlation of face IP scores among DZ twins. Although MZ and DZ twins share family environment to a similar extent, MZ twins share, on average, twice as much of their genetic variation as DZ twins. The correlations for face IP scores between MZ twins and between DZ twins can thus be used to estimate the proportion of variation in face IP that can be explained by variations in genes, shared environments, and unshared environments. We calculated a maximum likelihood correlation of 0.22 (95% CI: 0.14 – 0.29) for MZ twins and 0.09 (95% CI: −0.06 – 0.24) for DZ twins. These two correlations did not significantly differ (Fisher r-to-z transformation; p = 0.1), indicating that most of the variance in face IP is likely attributable to environmental factors. To obtain a more precise estimate of the contributions of genetic and environmental factors to face IP, we fit a standard ACE twin model that includes additive genetic influences (A), shared environmental influences (C), as well as unshared environmental influences and measurement error (E) using structural equation modeling techniques [15]. Controlling for age and sex, the ACE model attributed 22% of variance to (A) additive genetic factors, 0% to (C) shared environmental factors, and 78% to (E) individual or unshared environment/measurement error (see Fig 2B and Table 1). We compared the full ACE model with reduced AE, CE, and E models (respectively setting the contributions of the C parameter, A parameter, and A+C parameters to zero) and determined the AE model yielded the best fit based on Akaike’s Information Criterion (see Table 1 and Table S2). The AE model gave similar point estimates for both A (22%) and E (78%) parameters, but with tighter confidence intervals (see Fig 2C and Table 1). We conclude that most of the reliable variation in face IP was explained by the influence of unshared or individual environment with a relatively small contribution from genetic variation and little to no contribution from shared environment.

Figure 2. Genetic and environmental contributions to face preferences (Face IP) and face recognition.

Shown are (A) face IP scores for monozygotic (MZ) twin pairs and dizygotic (DZ) twin pairs. Scores from one twin are plotted on the x-axis, with scores from that person’s co-twin on the y-axis. Maximum likelihood correlations are shown. MZ correlations represent the combined impact of shared genetic variation and shared environments (family resemblance), whereas the difference between MZ and DZ correlations can be used to estimate the contribution of genetic variation, specifically. (B) Maximum likelihood model-fitting was applied to MZ and DZ twin data to estimate A (additive genetic), C (shared environmental) and E (unshared environmental) contributions to face IP scores. Error bars give 95% confidence intervals around each estimate (bounded by zero on the lower end). (C) The best-fit model included contributions from both additive genetic (A) and individual or unshared environmental (E) factors. Estimates of genetic and environmental contributions to both face recognition and face IP scores (face preferences) are shown, with error bars indicating 95% confidence intervals around each estimate. Arrows indicate the upper boundary for ‘A’ estimates, based on the test-retest reliability of each measure. Based on AE model estimates, (1) almost all of the reliable variation in face recognition is due to variations in genes and (2) most of the reliable variation in face IP is attributable to variations in environments. This dissociation in heritability suggests that there are distinct genetic and etiological mechanisms underlying these two core social-perceptual phenotypes. See also Figure S1 and Table S2.

Table 1. Reliability, twin correlations, and variance component estimates for Face IP.

Internal/split-half reliability was estimated based on Spearman-Brown corrected correlations between face IP scores calculated from ratings on odd- versus even-numbered trials. Test-retest reliability was calculated in a separate sample based on alternate forms test-retest (see Supplemental Information: Reliability Analysis). Maximum likelihood correlations are shown for monozygotic (MZ) and dizygotic (DZ) twin pairs. Model fit parameters are given for the full ACE model, estimated using OpenMx software [62]. Parameter estimates represent the contribution of additive genetic (A), shared environmental (C), and individual or unshared environmental (E) factors. The AE model was selected as the best-fit model, based on Akaike’s Information Criterion. Parameter estimates for the reduced AE model are also shown. Parameter estimates for all models are given in Table S2 of Supplemental Information.

Reliability
internal (split-half)	0.88
test-retest	0.75
Twin Correlations (95% CI)
MZ	0.22 (0.14 – 0.29)
DZ	0.09 (−0.06 – 0.24)
Model fit: −2LL; AIC (p-value)
ACE	12711.44; 9695.44
AE	12711.44; 9693.44 (p = 1)
CE	12713.54; 9695.54 (p = 0.15)
E	12741.39; 9721.39 (p < 0.001)
Full Model: ACE estimates (95% CI)
A	0.22 (0 – 0.29)
C	0 (0 – 0.24)
E	0.78 (0.71 – 0.86)
Best-fit Model: AE estimates (95% CI)
A	0.22 (0.14 – 0.29)
E	0.78 (0.71 – 0.86)

Our results provide a rare example of a complex, objectively measured, highly reliable, and specific behavioral characteristic that is shaped predominantly by environmental factors. Although high estimates of unshared environment contributions to social cognition and behavior are reported in twin studies [16], these estimates often occur in the context of low or unknown reliability [17, 18]. High measurement error (low reliability) spuriously reduces estimates of familial resemblance from both genetic and shared environmental factors and spuriously inflates estimates of unshared environmental contributions. Apparent examples of high unshared environment contributions are often confounded with measurement error [19]. Given the high reliability of face IP, even when estimated conservatively (via an alternate forms test-retest procedure), we conclude that the contribution of unshared environmental factors to face IP cannot be explained by unreliable measurement. Instead, our findings support the notion that individual aesthetic face preferences are truly shaped primarily by individual life experiences [20, 21].

The observed results isolate a highly specific environmental influence that impacts face IP independently of scene IP and abstract object IP. But does this environmental influence act specifically on face attractiveness judgments? Alternatively, it might act broadly on any judgment that involves a face, or on any social judgment. As a strong test of specificity, we consider the case of face identity recognition. Face attractiveness judgments and face identity recognition both involve social evaluation of faces, in the visual domain. Moreover, both require processing of invariant face characteristics, which are known to rely upon inferior occipital and inferior temporal brain regions [22], and deficits in both have been found to coexist in patients [22–24]. If the etiology of face identity processing were to differ from that of face IP, then that would provide strong evidence that the observed environmental effect is specific not only to social stimuli in general; nor to faces in particular; nor even to judgments of invariant face characteristics; but rather to a particular subset of judgments of invariant face characteristics. We previously measured face recognition in another sample of MZ and DZ twins drawn from the same Australian Twin Registry [1] (see also [25]). While highly reliable, the face recognition measure was no more reliable than our face IP measure (Cambridge Face Memory Test scores: internal reliability = 0.89, test-retest reliability = 0.70; Face IP scores: internal reliability = 0.88, test-retest reliability = 0.75). Yet despite equal precision of measurement, a sample drawn from the same population, and similarly robust evidence for independence from various non-face categories, we found little to no impact of environment on face recognition ability. Genetic variation accounted for most or all of the reliable face recognition variance, in contrast with face IP (68% vs 22% heritability; p of difference < 1E-14; see Figure 2C and Table 1). Indeed, looking across the behavioral genetic literature, face IP is among the most environmental objectively measured behavioral traits, whereas face identity recognition is among the most heritable [1, 26]. We conclude from this etiological dissociation that the observed environmental effect is highly specific to face attractiveness judgments.

Previous evidence has indicated that preferences for particular faces or face characteristics are shaped by a range of factors, including personality preferences [27], the rater’s own facial characteristics [28], features of the socioeconomic and cultural environment [29–33], previous visual experience [34–38], and history of social learning [18, 39–43]. Individual preferences for faces are also correlated among friends and spouses [12]. In our sample, most of the variations in face preferences were explained by the contribution of unshared environment – those aspects of the environment that are unique to individuals and not shared between twins. Our data suggest that individual life history and experience are a driving force behind individual face preferences [21].

Does this mean that shared environments are not important for individual face preferences? Not necessarily. Our study was conducted with a relatively homogeneous sample of Australian twins [44]. Given the sociocultural homogeneity of our sample, the low contribution of genetic variance to face IP is particularly noteworthy: estimates of genetic contributions tend to be higher where environments are less variable [45].

We have demonstrated, in the context of a sensitive behavioral genetic investigation, what scholars in the humanities and arts have long claimed: that, at least for faces, our environments play a substantial role in shaping our preferences and particular notions of attractiveness [20]. Our results further establish that the important environments are individual-specific; that is, they are not consistent across family members. Moreover, we demonstrate a developmental dissociation in the fundamental etiology of two core domains of social perception and face processing: whereas variations in face attractiveness judgments result primarily from variations in environments, variations in face identity judgments result primarily from variations in genes. Our results provide a window into understanding the developmental and biological origins of the social brain, and those aspects of our genes and environments that make us each unique.

Experimental Procedures

Participants

To understand the genetic and environmental contributions to individual face preferences, we recruited 796 twin pairs through the Australian Twin Registry [46]. We classified twin zygosity through latent class analysis via a standard self-report questionnaire [47]. After exclusions (see Supplemental Information), our final sample comprised 547 MZ twin pairs (mean age = 45.2; 415 female) and 214 same-sex DZ twin pairs (mean age = 45.8; 160 female). We also analyzed data from a set of 660 singletons (mean age = 41; 445 female) who completed the same measures as our twin sample. As the correlations between dependent measures calculated using the combined twin and singleton samples and calculating average ratings using only the singleton sample were extremely high (r’s > 0.99), we combined the two samples in order to maximize the precision of our estimates. We note that our results were the same when calculated using only the twins sample, only the singletons sample, or with both samples combined. The study was reviewed and approved by the Committee for the Use of Human Subjects at Harvard University and the Australian Twin Registry. All participants gave informed consent before taking part in the study.

Behavioral Testing and Data Analysis

All tests were administered through our website, TestMyBrain.org [48]. Participants were sent a link to the study and participated at a time of their choosing from their own personal computers. Participants were given feedback about how their ratings compared to the average person. We have found that this feedback-as-incentive model produces high quality data that is comparable to data collected in traditional lab settings, even for demanding tests of social perception [48]. For the measures included in this manuscript, average ratings between twin participants and a separate sample tested in the lab (n = 31) were highly correlated (r = 0.96), indicating comparability between unsupervised web versus lab-based assessments.

Individual preference scores were estimated based on the correlation between a participant’s ratings and the average ratings for each stimulus (see Figure 1D–E), transformed to z scores using Fisher’s r-to-z transformation [14]. We then regressed out z-transformed response consistency scores to produce a general face preference score that was not related to differences in intra-individual variability (see Figure S1A and Supplemental Information). Finally, we regressed out individual preference scores for objects and scenes (see Figure S1B–C). These face-specific scores were then standardized (z-scored). Summary statistics for face, object, and scene preference measures are in Table S1 of the Supplemental Information. We also conducted internal reliability and test-retest reliability analysis to estimate the degree to which variations in face IP scores reflect variations in stable, phenotypic characteristics (internal reliability = 0.88; test-retest reliability = 0.75; see Supplemental Information). We used standard maximum-likelihood based behavioral genetic model-fitting procedures, implemented via OpenMx, to estimate genetic and environmental contributions to face IP (see Supplemental Information).