Adaptation improves discrimination of face identity

Abstract

Whether face adaptation confers any advantages to perceptual processing remains an open question. We investigated whether face adaptation can enhance the ability to make fine discriminations in the vicinity of the adapted face. We compared face discrimination thresholds in three adapting conditions: (i) same-face: where adapting and test faces were the same, (ii) different-face: where adapting and test faces differed, and (iii) baseline: where the adapting stimulus was a blank. Discrimination thresholds for morphed identity changes involving the adapted face (same-face) improved compared with those from both the baseline (no-adaptation) and different-face conditions. Since adapting to a face did not alter discrimination performance for other faces, this effect is selective for the facial identity that is adapted. These results indicate a form of gain control to heighten perceptual sensitivity in the vicinity of a currently viewed face, analogous to forms of adaptive gain control at lower levels of the visual system.

1. Introduction

Adaptation causes aftereffects that selectively change the perception of a stimulus following exposure to a prior stimulus. Aftereffects are widespread phenomena in the visual system, occurring not only with basic properties such as luminance, contrast, spatial frequency, orientation and colour, but also with higher level representations of shapes and objects, such as faces. For at least some basic features, adaptation can be regarded as a process by which the visual system adjusts to the prevailing conditions of the environment. As an example, luminance adaptation in the retina serves to maintain contrast sensitivity across a wide range of illumination, by centring the limited sensitivity range of retinal cells on the ambient levels of the current environment, a process termed retinal gain control (e.g. [1]). Evidence for a similar function for adaptation at various levels of cortical processing has been mixed. Some studies found enhanced discrimination of orientation, contrast and direction of motion following adaptation [2–6] but others did not [7–9].

In contrast to the long-standing literature on adaptation to features such as colour, motion and contrast, adaptation to faces has only recently been described [10–14]. This has generally been shown with perceptual bias paradigms, in which viewing of one face causes perceptual decisions about ambiguous faces to be biased away from the adapting face and towards other faces, a ‘repulsive’ aftereffect [15–23]. We have also shown that adaptation affects the minimum luminance contrast necessary for recognizing faces [24–26]. In this report, we ask whether adaptation also affects the thresholds for discriminating between different faces. If so, this might indicate a type of ‘face gain control’, in which the sensitivity range for face perception is adjusted dynamically to enhance performance in the vicinity of current stimulus conditions.

2. Methods

(a). Observers

Eleven observers with normal or corrected-to-normal vision participated (four males, ages 22–36). All but one (I.O.) of the observers were naive to the purposes of the experiment. The protocol was approved by the review boards of the University of British Columbia and Vancouver Hospital, and informed consent was obtained in accordance with the principles in the Declaration of Helsinki.

(b). Apparatus

Stimuli were displayed on a SONY Trinitron 17-inch GDM-G500 monitor at 1024 × 768 resolution and refresh rate of 100 Hz. Viewing distance was 99 cm. Cambridge Research Systems (CRS) VSG Toolbox for Matlab was used to present the stimuli via a CRS VSG 2/3 card. Displays were gamma-corrected by means of an automated calibration procedure using the VSG software and an OptiCAL photometer (Model OP200-E) by CRS. Average luminance was 40 cd m⁻².

(c). Stimuli

Face stimuli displaying a neutral expression were obtained from the Karolinska Database of Emotional Faces [27]. All face images were first converted to greyscale using Adobe Photoshop CS 8.0 (www.adobe.com). The images were then manipulated using Matlab (www.mathworks.com). An oval aperture was superimposed on the facial images, outside of which the display was a uniform grey set at the mean luminance of 40 cd m⁻². The tip of the nose and the pupils were used as anchors to align faces horizontally and vertically. All faces had the same pose (frontal), orientation (vertical upright) and eye colour (brown) and were without obvious distinguishing marks such as moles and visible hair to minimize discrimination based on trivial features. Luminance values inside the oval aperture were normalized such that the average was set to the mean luminance of 40 cd m⁻² and the root-mean-squared (r.m.s.) contrast (the standard deviation of luminance values divided by mean luminance) to 1. These images were the standard templates whose contrasts were later set to a fixed 30 per cent for both adapting and test faces throughout the experiment. The horizontal and vertical extents of the cropped test faces were 5.1 × 7.2° visual angle, respectively, at the viewing distance of 99 cm. Adapting stimuli were 50 per cent larger than test stimuli (i.e. 7.7 × 10.8°), to minimize contributions to adaptation from lower level image properties.

(d). Adapting stimuli

There were four possible adapting stimuli: three female faces (labelled A, B and C) and a blank stimulus (figure 1a). The blank adaptor was used in the baseline condition. The choice of the three adapting faces (A, B and C) was not arbitrary. The degree of similarity between the different adapting faces is a critical factor in the experiment. If they are too similar, each may partially activate the neural units preferring the other, with the result that adapting to one face may inadvertently affect the representations of the second face, making it difficult to detect a difference between same-adaptor and different-adaptor conditions. We used an ideal observer simulation [28–30] to determine a physical similarity measure, and chose adapting faces sufficiently distinct from each other. We ran the ideal observer in a 2-alternative forced-choice face recognition task for pairs of faces in added zero-mean Gaussian white noise, and measured recognition contrast thresholds. The ideal observer had knowledge of the two alternative faces, their occurrence frequency (equal), the contrast of the added Gaussian white noise, and the contrast of the face stimuli at each trial. The response of the ideal at each trial was based on the maximum Bayesian posterior probability across the templates, which is equivalent to minimum Euclidean distance between the stimulus and the alternative templates as given by for i = 1, 2, where S is the noisy stimulus, c is the stimulus contrast and F_i is the template for face i. The contrast thresholds of the ideal observer were measured through a 40-trial fixed-length Quest procedure. Further details of this simulation can be found in Fox et al. [16].

Figure 1.

Illustration of the stimuli and experimental procedure. (a) Faces of three females (A, B and C) were called the base faces. These were used as adapting stimuli (in addition to a blank adapting stimulus) and as the base faces used to generate the three different sets of test stimuli. (b) Production of test set A. A test set was generated by morphing from a base face (A in this example) towards two different endpoint faces, X and Y, which were not base faces. All test stimuli produced this way still looked predominantly like the base face, but contained a subtle resemblance to either face-X or face-Y. The magnitude of this resemblance was determined by the %morph-distance. In this example, morph images AX and AY both contain 70% face-A, and 30% face-X or face-Y. (c) An experimental trial. Discrimination thresholds around a base face (e.g. A) were measured using a two-interval two-alternative forced-choice paradigm. Each trial started with a 10 s adapting period displaying one of the three base faces A, B or C, or a blank stimulus. Next, observers were shown two-morph test stimuli, AX and AY, in a random sequential order, and asked to indicate whether the first or the second stimulus resembled face-Y more. The morph image pairs were always from either side of the base face (i.e. one an AX mix and one an AY mix), and constrained to be equidistant in %morph-distance from it. The %morph-distance between AX and AY in each trial was determined by the Quest procedure.

We obtained the ideal observer contrast thresholds for all pairs of faces in a set of seven potential stimulus candidates. This gave us 21 possible face-pairs. The specific value of the ideal observer's contrast threshold for a given pair of faces is arbitrary as it depends on the power of the added noise. However with fixed noise power across all pairs, the relative magnitude of the contrast thresholds gives us a measure of similarity. A higher contrast threshold indicates higher similarity, as the two faces are harder to discriminate and thus require more contrast for the ideal observer to do so. The most dissimilar pairs of faces are those with the lowest contrast thresholds. From the set of 21 pairwise contrast thresholds (figure 2), we selected the three faces that produced the lowest thresholds (i.e. as distinct from one another as possible) with the constraint that all three pairwise thresholds (between the pairs A-B, B-C and C-A) were approximately equal (i.e. equally distinct from one another). The three faces that satisfy both conditions (F₅, F₆ and F₇, in figure 2, outlined in the red rectangle) were chosen as the adapting stimuli (base faces A, B and C in figure 3).

Figure 2.

Selection of the adapting stimuli based on the ideal observer method. Our aim was to choose three base faces as dissimilar from one another as possible. The ideal observer's contrast threshold for discriminating between two faces provides a measure of physical similarity: the lower the threshold, the more distinct (i.e. easy to discriminate) the faces. We started out with seven faces: F₁, F₂, … , F₇, as potential candidates. We plot the ideal observer's thresholds for all 21 resulting pairs in this group. We chose a triad of faces that (1) had the lowest pairwise discrimination thresholds, and (2) for which the three different pairwise thresholds were similar, so that all three adapting faces were equally distinct from all the others. The pairwise thresholds for the three faces chosen, F₅, F₆ and F₇ (shown in figure 1a as base faces A, B and C) are outlined by the dashed rectangle.

Figure 3.

Discrimination thresholds around adapted and unadapted faces. (a) Threshold ratios are plotted for same-face and different-face adapting conditions. Threshold ratio is the threshold for a given adapting condition divided by the baseline threshold: hence lack of adaptation would give a threshold ratio of 1. The data shown are geometric averages of the threshold ratios of all subjects across all three base faces. Error bars represent 95% bootstrap confidence intervals. (b) Same data shown for each individual, as a scatter-plot of threshold-ratio pairs for the two adapting conditions, same-face and different-face. Eight out of 11 data points fall below 1 in the same-face condition indicating that adapting to the same-face reduced thresholds for majority of the observers.

(e). Test stimuli

The three adapting faces A, B and C were also used as base faces to produce test stimuli. To measure discrimination thresholds around these three adapting faces, we generated three sets of test stimuli, with a morphing technique using FantaMorph 3.0 (www.fantamorph.com). Each of the three test sets used a different one of the three adapting faces (A, B or C) as its base face. In each series, the base face was morphed in two ‘directions’, towards endpoint face X and towards endpoint face Y (figure 1a,b). Endpoint faces X and Y were different females than base faces A, B and C. The same endpoint faces (X and Y) were used in each of the three series. Figure 2 also displays the ideal observer results for the endpoint faces (X was F₃ and Y was F₄). Unlike the base faces, the specific choice of the endpoint faces in relation to the base faces and to each other was not critical for the experimental design. Having two endpoint faces that differ substantially in their similarity to a base face would add noise to the threshold measurements in the main experiment, but would not impact the difference between conditions, particularly as all adaptation data are referenced to a baseline condition with a blank screen (see §2g). Nevertheless, as seen in figure 2, faces F₃ and F₄ were reasonably comparable in their discriminability from the base faces F₅, F₆ and F₇.

Thus, each of these three sets was generated by gradually morphing its base face (e.g. face-A) towards endpoint face-X in one direction, producing AX morphs (face-A slightly modified to resemble face-X) for the A series, for example, and towards endpoint face-Y, producing AY morphs, in the other direction. As a result, the ‘A’ test set, for example, included 20 AX and 20 AY morphs, for a total of 40 morph images. The first 10 of these in each direction differed from the base face in 1 per cent steps, and the next 10 varied in additional 2 per cent steps, towards the endpoint X or Y faces. For a naming convention, the test stimuli were denoted by the amount of the base face and endpoint face in the image, e.g. A₉₉X₁ for a 99% A-1% X face morph. Thus, for base face A varying towards endpoint face-X, the series consisted of ‘A₉₉X₁’, ‘A₉₈X₂’, … ,‘A₉₀X₁₀’ for the first 10 images in the series, and ‘A₈₈X₁₂’, … ,‘A₇₀X₃₀’ for the last 10 images in the series.

Our strategy was to determine how accurately a subject can discriminate an AX morph from an AY morph of the same degree of change (e.g. ‘A₉₀X₁₀’ versus ‘A₉₀Y₁₀’), and likewise for BX versus BY morphs, and CX versus CY morphs. Our goal was to see whether the discriminative ability for a morph series is affected by prior adaptation to its base face, or to one of the other base faces.

(f). Procedure

Face discrimination morph-distance thresholds at the 82 per cent correct level were measured in a two-interval two-alternative forced-choice paradigm. Two morph images ‘equidistant’ in terms of %-morph distance from the set's base face (e.g. A₉₀X₁₀ and A₉₀Y₁₀) were shown in random order in the two intervals. The task was to indicate which interval contained the AY combination, i.e. the image that looked more like endpoint face (Y), by pressing 1 (for first interval) or 2 (for second interval) on the computer keypad. Auditory feedback indicated whether the answer was correct or not. Prior to starting the experiment, the observer was instructed on which endpoint face to report on (Y), and this was fixed throughout the entire experiment. The design of this face discrimination experiment was modelled after an orientation paradigm used to measure discrimination thresholds around vertical orientation [2].

Discrimination thresholds were determined by using the Quest procedure implemented in the Psychophysics Toolbox [31,32] in Matlab. The discrimination threshold, obtained at the end of a fixed-length 40-trial run, was defined as the magnitude of the total modification to the base test-face, expressed in %morph-distance, necessary to reach the criterion success rate of 82 per cent. (Note: if the AX image at threshold was a 95%–5% morph between face-A and face-X, the AY image would also be a 95%–5% morph between face-A and face-Y and the threshold %morph-distance would be 10%, not 5%.) Discrimination thresholds were measured in three different adapting conditions: (i) baseline condition, where the adapting stimulus was a blank, (ii) same-face condition, where the adapting stimulus was the same as the base face used in the test set, and (iii) different-face condition, where the adapting stimulus was a different face than the base face used in the test set (i.e. face B or C if the test set used face A).

Each trial started with a 10 s adapting period, followed by a noise mask (50 ms), a fixation cross (150 ms), a blank screen (150 ms), stimulus interval 1 (150 ms), a blank screen (150 ms) and stimulus interval 2 (150 ms; figure 1c). This last was replaced by a blank screen that remained until the observer entered their response by pressing key 1 or key 2.

The experiment was completed in three sessions. In each session, only one test set was used (A, B or C), with session order counterbalanced across observers. Each session started with a training block where observers practiced the discrimination task for that session's test set, without an adaptation period. Training continued for at least 12 consecutive 40-trial runs, or until discrimination performance reached an asymptote determined by visual inspection of the learning curve. This was followed by a 2 min warm-up block used to familiarize the observers with the experimental procedure. Finally, observers performed the experimental block, consisting of four randomly interleaved Quest runs each separately measuring the discrimination threshold for four different adapting stimuli (faces A, B or C, or blank). Each of the three experimental sessions took about an hour to complete.

(g). Data analysis

The 12 adaptor/test pairs given by three base test-faces × four adapting stimuli (i.e. three adapting faces and a blank) were categorized into same-face, different-face and baseline (blank adaptor) conditions. Discrimination threshold ratios for the same-face and different-face conditions were computed by dividing the %morph-distance threshold in each instance by its corresponding baseline (blank adaptor) threshold. Group data were computed as the geometric average of threshold ratios across subjects. Non-parametric bootstrap confidence intervals were obtained by re-sampling the data across observers a large number of times [33]. The re-sampled group averages were then sorted and the lowest and highest α/2 percentile values yielded the desired α% confidence intervals.

3. Results

Thresholds for the same-face condition were significantly reduced compared with baseline thresholds (threshold ratio = 0.88, p < 0.05, Wilcoxon signed-rank test), indicating enhanced discrimination sensitivity around the adapted face (figure 3a). By contrast, the average threshold-ratio for the different-face condition was 0.98, which was not significantly different from 1 (p > 0.2, Wilcoxon signed-rank test): thus, this shows no evidence that adaptation affects discrimination in the vicinity of unadapted faces. Figure 3b shows data for individual subjects, as a scatter plot of threshold ratios for same-face versus different-face conditions. We found that eight out of 11 observers show thresholds that were lower than baseline in the same-face condition, indicating that improved discrimination effect occurred for the majority of the observers, with most showing greater improvement for same-face than for different-face conditions.

4. Discussion

Our results show that face adaptation for 10 s enhances fine discrimination in the vicinity of the adapted face compared with baseline performance. In the current context, a low-level origin for this effect is unlikely: in general, lower level visual aftereffects are retinotopic and require maintained fixation throughout the adapting period, whereas our observers were allowed and even encouraged to shift their gaze during adaptation, to avoid fixating on a single point in the face and rather to scan the face in a relaxed manner. In addition, the fact that our adapting stimuli were viewed at a different size compared with the test stimuli, a common strategy in studies of face adaptation to minimize lower level effects [17,34–36], further argues against an account based on adaptation to lower level features for these face aftereffects.

A fundamental question regarding adaptation is its purpose in the visual system. While many studies have discussed what adaptation can tell us about the nature of face representations, few have considered whether it imparts any benefits to face perception. One study has reported that adaptation to gender or ethnicity does not appear to confer any advantage during rapid serial visual presentation or visual search for detection of these adapted facial categories [37]. As yet, few studies have examined adaptation effects on discrimination performance in face perception. One study has suggested changes in face discrimination after adaptation, but examined this around the average face, rather than around the adapted face [36], an observation that thus may not be directly relevant to our objective. A second study evaluated discrimination around an average face, and found that adapting to the average face did not improve sensitivity [38]. A third examined effects on discriminating gender between pairs of images taken from a male/female morph continuum [37]. In their experiment 4C, adapting to one end of the continuum did not appear to confer an advantage to discriminating pairs near that end compared with pairs near the opposite end. While this could reflect a difference between representations for facial gender and those of identity, technical issues would first have to be considered in comparing results. Both of these latter two studies differed from the present experiment in various methodological aspects—including the adaptation protocol, adapting and test image duration and task. It is possible that any or all of these differences account for the inability of these prior studies to find an effect on discrimination. In particular, the careful consideration we gave to maximizing the dissimilarity between different adapting faces may be an important methodological detail. Using this methodology, we were able to show that adapting to a face enhances our ability to distinguish between pairs of highly similar faces, as long as they are in the vicinity of the adapting face, possibly by increasing the dissimilarity between the representations of similar faces around the adapting face.

Better identification of faces of a particular race has been reported recently after adapting to the average face of that race [39]. This result reveals the effects of adaptation on the facial representations at the population level, e.g. changes on how faces of a particular race are coded. Although this study used a 5 min adapting duration, the adaptive changes shown may underlie such longer term phenomena as the other-race effect. To our knowledge our present results are the first report of improved discrimination of subtle changes on an individual face, resulting from adapting to that particular face.

As pointed out [39], the mechanism by which the adaptation enhances discrimination remains to be clarified. Our prior study of the dynamics of aftereffects on contrast recognition thresholds [26] has suggested that there are multiple adaptation effects with complex dynamics, including initial facilitation and later suppression of the adapted representation, as well as lateral inhibition of non-adapted representations. Another potential adaptation effect is that the tuning curves of units responsive to the adapted face are sharpened by brief exposure, leading to better sensitivity to small changes in facial structure. Sharpening of stimulus tuning has been discussed as an adaptive mechanism in other perceptual contexts [40–42], has been demonstrated neurophysiologically in the direction and speed selectivity in neurons of macaque V5 region [43,44], and has been incorporated in some models of face adaptation [26]. Alternatively, it has been suggested that improved discrimination of individuals after adapting to their ethnic category might be due to an ‘orthogonalization’, in which ‘adaptation reduces responsiveness to common information shared by all faces in a population … thus freeing up resources to code distinctive information, which is what matters for face recognition’ [39, p. 967]. At a categorical level such as ethnicity, one might speculate that adapting to either an average face for that category or specific exemplars of that category might both lead to a similar orthogonalization for ethnicity-related dimensions, which in our case might have predicted enhanced discrimination for both our adapted and unadapted faces, since they shared the same ethnicity and gender. However, one can ask whether orthogonalization for the identity-specific dimensions of an exemplar might lead to improved discrimination in its vicinity in face-space. If so, suppression or ‘reduced responsiveness’ may predict similar benefits to discrimination as sharpening of tuning. However, against this explanation is the finding that 10 s of adaptation elevates contrast recognition thresholds not only for adapted faces but also, and to an equal degree, for unadapted faces of the same ethnic category, possibly through lateral inhibition [26]. Hence, improved discrimination around adapted faces but not around unadapted faces may be difficult to attribute to reduced responsiveness when the latter is similar for both adapted and unadapted faces.

Enhanced discriminative power in the vicinity of an adapted stimulus has potential advantages in terms of heightened sensitivity to variations in stimulus properties close to the current percept. In some cases, such as retinal light adaptation, improved discrimination around the adapted level can serve to extend the operating range of the system [1,45]. With faces, recognition expertise depends on long-term experience, as revealed by such phenomena as the face inversion effect [46], a consequence of the bias of natural encounters with faces to the upright orientation, and the other-race effect [47], a consequence of the higher frequency of encounters with faces of the same ethnicity as that of the observer. Short-term face adaptation may be responsible for dynamic fine-tuning and calibration of face discrimination, which may enhance the ability to rapidly and efficiently distinguish individuals of similar appearance in ongoing social encounters [48]. This suggests that, like the adaptive mechanisms in the retina that serve to maintain contrast sensitivity through a gain control for background luminance, the higher level face recognition system of visual cortex may also have a gain control to adjust discriminative power for the current set of stimulus conditions, namely the faces being viewed, a potentially important benefit of adaptation.