The Effects of Face Inversion on Perceiving- and Sensing-Based Change Detection

Abstract

Face perception is more difficult when faces are inverted compared to when they are upright. However, it is not known whether face inversion disrupts the ability to make perceiving-based discriminations (i.e., the ability to identify a specific feature change), or sensing-based discriminations (i.e., the ability to detect there was a change without the ability to identify what changed). In the current study, we used confidence-based receiver operating characteristics (ROCs) in a change detection test to examine the effect of face inversion on perceiving and sensing. In Experiment 1, face inversion led to a reduction in the probability of perceiving but did not impact sensing-based discriminations. In Experiment 2, we replicated these results, and verified that the findings based on ROC estimates paralleled participants’ phenomenological experiences of perceiving and sensing. Furthermore, the perceiving-based face inversion effect was found to reflect a reduction in the ability to accurately report specific feature changes. These findings indicate that face inversion does not reduce the ability to sense there was a change in the absence of identification, but rather it reduces the ability to consciously identify specific characteristics of faces in service of perceiving-based discriminations. In addition, they suggest that sensing responds to global differences across the visual image, rather than to changes in holistic processing of the visual input. These results further our understanding of the face inversion effect and clarify the nature of the processes underlying visual perception.

Suppose you were presented with two pictures of the same face and were asked to decide whether the two images were identical or slightly different. In some cases, you may be able to readily perceive specific details that differ between pictures, such as the presence of a goatee in one picture but not the other. In other cases, however, you may sense that there is a difference, but lack the ability to identify what the difference is. This example illustrates that visual discriminations can be based on two different types of perceptual processes: perceiving and sensing. A growing body of research has indicated that perceiving- and sensing-based perceptual judgements are functionally distinct (Aly, Ranganath, & Yonelinas, 2013, 2014; Aly & Yonelinas, 2012; Dehaene, Changeux, Naccache, Sackur, & Sergent, 2006; Rensink, 2000, 2004). For example, in a series of change detection experiments, Aly and Yonelinas (2012) found that whereas sensing-based responses varied continuously with respect to response confidence, perceiving-based responses were limited to high-confidence responses. Moreover, sensing-based responses dominated performance when the perceptual changes were global (e.g., the major features of one image were contracted inward very slightly), whereas perceiving-based responses dominated performance when the change involved a single local feature of an image. These results are important in showing that factors or variables that influence overall perceptual performance may do so in very different ways – either by impacting perceiving- or sensing-based perceptual processes.

One well established finding in the perceptual literature is that it is much more difficult to make perceptual judgements about faces when they are inverted compared to when they are presented in an upright orientation (i.e., the face inversion effect; Diamond & Carey, 1986; Scapinello & Yarmey, 1970; Yin, 1969). The face inversion effect is generally thought to reflect the fact that faces are processed holistically and this type of processing is disrupted when faces are presented in unfamiliar orientations (Bartlett & Searcy, 1993; Farah, Tanaka, & Drain, 1995; Freire, Lee, & Symons, 2000; Searcy & Bartlett, 1996; Sekuler, Gaspar, Gold, & Bennett, 2004; for reviews see Maurer, Grand, & Mondloch, 2002; McKone & Yovel, 2009; Rossion, 2008; Tanaka & Simonyi, 2016; Valentine, 1988). The term holistic processing is used to refer to the simultaneous integration of feature and metric spacing information into a unified perceptual representation that is largely processed as an unparsed whole (McKone & Yovel, 2009; Richler, Palmeri, & Gauthier, 2012; Tanaka & Farah, 1993; Wilford & Wells, 2010), whereas the terms configural and featural refer to the type of information that can be manipulated in a stimulus, as opposed to perceptual processes (Barton, Keenan, & Bass, 2001; Rossion, 2008).

Although many agree that holistic face processing is disrupted upon inversion, the characteristics of this disruption continue to be debated. Some contend the face inversion effect is qualitative – it disproportionately affects the processing of configural information compared to featural information (Diamond & Carey, 1986; Freire et al., 2000; Leder & Bruce, 2000; Leder & Carbon, 2006; Maurer et al., 2002; Rhodes, Brake, & Atkinson, 1993; Rossion, 2008; Valentine, 1988). While others argue for a quantitative view of the face inversion effect – processing of configural and featural information are both adversely affected by inversion to a similar degree (Curby, Goldstein, & Blacker, 2013; McKone & Yovel, 2009; Richler et al., 2012; Sekuler et al., 2004; Yovel & Kanwisher, 2004). Still other work has focused not on configural versus featural information, but on the extent to which holistic processing is exclusive to faces, as opposed to any objects of expertise, by examining the effect of inversion on non-face mono-oriented objects (Curby, Glazek, & Gauthier, 2009; Curby et al., 2013; Epstein, Higgins, Parker, Aguirre, & Cooperman, 2006; Farah et al., 1995; Wilford & Wells, 2010; Yin, 1969). While there is now an extensive body of literature that has provided important insights into the functional processes involved in the face inversion effect, it remains unknown whether face inversion impacts perceiving- or sensing-based perceptual discriminations.

So how might perceiving and sensing be impacted by face inversion? Does face inversion reduces one’s ability to perceive specific featural changes, or does it reduce the ability to sense there was a change? Elucidating this would further our understanding of face perception by revealing what types of perception are, and are not, involved in producing the face inversion effect. In addition, determining how face inversion impacts perceiving- and sensing-based perceptual processes will allow us to further characterize these processes by showing how they respond to disruptions in holistic visual processing.

Based on prior work we expected that inverting a face would preferentially disrupt sensing-based discrimination. That is, as described above, Aly and Yonelinas (2012) found that sensing-based responses increased when the perceptual changes were global (i.e., when the images were pinched inward or expanded outward such that a large portion of the image was slightly altered) compared to when the perceptual changes were local (i.e., when a single feature was added or removed). To the extent that global manipulations impact the configural information in images, rather than just a single feature, these results suggest that sensing may be based on the configural information within the visual image. Thus, if the face inversion effect is due mainly to disrupted holistic processing of configural information, we would expect that face inversion would disrupt sensing more than perceiving, particularly for faces with widespread global changes. In addition, Wilford and Wells (2010) found that participants were better at detecting that a change had occurred for faces compared to houses, but were better at explicitly identifying the specific change that had occurred for houses than for faces. Moreover, this interaction was absent when the stimuli were inverted, suggesting that general change detection, rather than specific change identification, may be disrupted by face inversion. These results provide further evidence that face inversion may preferentially reduce sensing-based perception.

However, another possibility is that face inversion may affect perceiving-based discriminations by disrupting the ability to identify specific features that differ between faces. In support of this possibility are studies showing that experts are better than novices at making fine perceptual discriminations about stimuli from their area of expertise, such as for own-race faces (Rhodes, Brake, Taylor, & Tan, 1989), famous faces (Buttle & Raymond, 2003), dogs (Diamond & Carey, 1986), cars (Curby et al., 2009), videogames (Clark, Fleck, & Mitroff, 2011), radiographic (X-ray) films (Myles-Worsley, Johnston, & Simons, 1988), and football related images (Werner & Thies, 2000). Thus, extensive prior experience with upright faces may promote efficient visual search and so may facilitate the identification of featural details within the upright face. Indeed, it has been suggested that an orientation-dependent holistic representation of upright faces facilitates rapid and efficient processing of the face by providing top-down attentional guidance of visual search (Barton et al., 2001; Endo, 1986; Malcolm, Leung, & Barton, 2004; Richler et al., 2012; Rossion, 2008). Similarly, inversion is believed to disrupt the formation and use of holistic or object-based representations leaving only a slower, part-based search strategy available that no longer benefits from top-down attentional guidance (Barton, Radcliffe, Cherkasova, Edelman, & Intriligator, 2006; Curby et al., 2013). In this way, face inversion might act to reduce the likelihood of perceiving specific changed features.

In order to separate perceiving- and sensing-based perceptual discriminations we utilized a signal detection based model of receiver operating characteristics (ROCs; Macmillan & Creelman, 2005; Swets, 1973; Yonelinas, 1994, 2001). On each perceptual trial (see Figure 1A) participants were sequentially presented with two faces and were required to make a ‘same’/’different’ discrimination using a 6-point confidence scale ranging from ‘sure same’ to ‘sure different’. For half of the trials the two faces were identical (i.e., same trials), and for the other half they were slightly altered (i.e., different trials). Participants were presented with faces that were either upright or inverted 180° and that had either global, widespread changes or local, discrete changes (Figure 1B).

Figure 1.

(A) Change detection task trial sequence example for a different trial from the inverted local condition. Trial example is not drawn to scale. (B) Examples of face stimuli with global changes (top) and local changes (bottom).

ROCs were generated by plotting the hit rate (i.e., the probability of correctly responding ‘same’ when the two faces were the same) on the y-axis, against the false alarm rate (i.e., the probability of incorrectly responding ‘same’ when the two faces were different) on the x-axis, across varying levels of response confidence (Figure 2A). The leftmost point of the ROC represents the highest confidence ‘same’ response and points extending rightward represent cumulative hit and false alarm rate probabilities as each consecutive level of response confidence is included.

Figure 2.

(A) ROC generated by plotting hits and false alarms. Sensing (S) captures the degree of curvilinearity of the ROC and is measured by d’, whereas perceiving a difference (P_d) captures the upper intercept and is measured as a probability (i.e., the distance from the 1, 1 intercept). (B) The Dual Process Signal Detection (DPSD) model. Probability density functions for evidence that stimuli are the same or different. Sensing-based discrimination is measured as the distance between the means of the same and different distributions (i.e., d’). Perceiving-based discrimination is measured as the height of the perceiving distribution (i.e., solid perceiving distribution). If face inversion disrupts sensing, the distance between the means of the same and different distributions should decrease. If face inversion disrupts perceiving, the height of the perceiving distribution should decrease (i.e., dashed perceiving distribution). We note that the DPSD model is a mixture model, rather than a traditional signal detection model as described in many classic texts like Macmillan and Creelman (2005). That is, we assume that if perceiving occurs subjects will base their response on that information and effectively ignore sensing strength, and so it is only when perceiving fails to occur that the sensing strength distributions will be utilized. In contrast, traditional signal detection approaches will sometimes assume that different strength distributions may be averaged together, rather than mixed, to produce an overall strength distribution. These two classes of models can produce very similar ROCs, although mixture models tend to produce slightly flatter ROCs (for a discussion see Yonelinas & Parks, 2007).

The observed ROCs were fit to a Dual Process Signal Detection (DPSD) model using maximum likelihood estimation in order to estimate the contributions of perceiving and sensing (Figure 2B) (Aly & Yonelinas, 2012; Goodrich & Yonelinas, 2016; Yonelinas, 1994, 2001). The model assumes that overall perceptual performance reflects a mixture of two different types of trials. On some proportion of trials participants are assumed to be able to identify a feature that differs between the two images (i.e., the perceived trials), whereas on other trials they are unable to identify any specific features that differ (i.e., non-perceived trials). However, for the non-perceived trials, participants may still correctly decide that the images were different if they sense that there is a difference but cannot identify the precise features that differ. The perceive/sense distinction captures several phenomenological aspects of visual perception. For example, one of the reviewers of the current paper described an experience that illustrates this distinction quite clearly; “my father once grew a mustache, which he wore for about a year. One day, he shaved it off without telling anyone. When he came home from work, I said ‘you look so young. Did you dye your hair or something?’ That is, I sensed the change, but I did not perceive the detail that changed”. In addition, the idea that perceiving seems to occur in some cases but fails in others is illustrated by the phenomenon of bistable illusions, such as the common example where an ambiguous visual image is perceived either as a rabbit or a duck, and the observer reports seeing one image or the other with no in between.

According to the DPSD model, sensing is assumed to reflect the classical signal detection process underlying the common d’ sensitivity metric. That is, it is assumed that there is a continuous perceptual sensing signal measuring the degree to which the two faces match/mismatch each other. In this way, the same trials will have some mean level of matching signal with some variability around the mean (i.e., the same distribution on the left of Figure 2B), whereas the different trials will have a higher mismatch signal (i.e., the different distribution that is shifted to the right of the same distribution in Figure 2B). The perceptual discriminability afforded by sensing is measured as the distance between the means of the same and different distributions (i.e., d’ which is measured in z-scores). The better that participants are at discriminating between same and different trials on the basis of sensing, the further apart those distributions will be from one another. Participants are assumed to map sensing onto confidence such that higher mismatch strength leads to higher mismatch confidence and higher match strength leads to higher match confidence. In addition to sensing, however, if the participant can identify some qualitative difference between the two faces then these trials are assumed to be consciously perceived as different and so are expected to result in a high confidence ‘different’ response (i.e., the perceiving distribution on the far right extreme of Figure 2B). The shape of the perceiving distribution is presented as a narrow normal distribution for convenience, but its true shape is unknown and its only constraint is that the perceiving distribution is assumed to lead to high-confidence ‘different’ responses that are as high, or higher, than the highest confidence ‘different’ responses based on sensing. The notion is that if participants can identify a specific difference in the two faces, they can be sure that the two faces are not the same. In this way, perceiving can be measured as a simple probability which indicates the proportion of different trials that are perceived as ‘different’.

The DPSD model can be written as a set of equations that can be fit to the observed ROC data to derive estimates of sensing (i.e., sensitivity, measured as d’) which captures the degree of curvilinearity of the function, perceiving (measured as probability P_d) which captures the upper intercept of the ROC, and response criterion parameters separating each of the six levels of response confidence (see Aly & Yonelinas, 2012; Goodrich & Yonelinas, 2016; also see Koen, Barrett, Harlow, & Yonelinas, 2016). It is expected that participants will correctly identify a different trial if they perceive a specific feature that differs across the two images (P_d for ‘perceive different’). However, even if they fail to perceive a specific difference (i.e.,1 - P_d), they may still make a correct response on the basis that they sense there was a difference (S_d), which reflects the proportion of different trials that exceed the sensing response criterion. Thus, P(‘different’|different) = P_d + (1- P_d)*(S_d). It is also expected that participants will incorrectly identify some proportion of same trials as being ‘different’ (S_s, which reflects the proportion of same trials that exceed the sensing response criterion). Thus, P(‘different’|same) = S_s. Sensing is assumed to reflect an equal-variance signal-detection process and, hence, S_d and S_s will be a function of the distance between the means of the same and different item distributions (d′) and the response criterion (c).

Prior work has indicated that the DPSD model provides a better account of perceptual discriminations than two other common alternative signal detection based ROC models: The Equal Variance Signal Detection model (EVSD) and the Unequal Variance Signal Detection (UVSD) model (Aly & Yonelinas, 2012; Goodrich & Yonelinas, 2016; Parks & Yonelinas, 2009; Yonelinas & Parks, 2007). Although the current studies were not designed to contrast these different models, the fits and parameter estimates for those models are reported after the results of the two experiments are described, as they are useful in further characterizing the current findings.

In Experiment 1, we used confidence-based ROCs in a same/different discrimination test to determine the effect of face inversion on perceiving- and sensing-based change detection. Contrary to our initial expectations, face inversion led to a reduction in the probability of perceiving differences between faces but did not impact the contribution of sensing-based discriminations. The pattern of results was observed whether the changes were local or global, although the effects were largest for global changes. Experiment 2 further confirmed these results and verified that the findings based on ROC estimates of perceiving and sensing paralleled participants’ phenomenological experiences of perceiving and sensing. Moreover, an examination of participants’ ability to explicitly identify the changes indicated that the perceiving-based face inversion effect reflected a reduction in the ability to accurately report specific featural changes.

2. Experiment 1

2.1. Methods

2.1.1. Participants

Ninety-four undergraduates from the University of California, Davis participated in Experiment 1 in exchange for psychology course credit. Three participants were excluded from analyses: two for contraindicative medical histories and one for chance performance. All remaining participants reported having normal or corrected-to-normal vision. In total, 91 participants make up this data set (74 female; mean age=20; range 18–45 years). Participants were randomly assigned to view one of four face stimulus sets (see Materials): upright global (n=22), upright local (n=24), inverted global (n=21), and inverted local (n=24).

Power analyses were conducted using effect sizes from a study by Aly and Yonelinas (2012, Experiment 3A) which examined the nature of perceiving and sensing for scenes that were manipulated either globally or locally in the same manner as the face stimuli used in the current study. Based on the effect sizes of perceiving (Cohen’s d=1.02, two-tailed) and sensing (Cohen’s d=0.87, two-tailed) from that study, we required sample sizes of 17 and 22 to detect a possible face inversion effect on perceiving and sensing, respectively, with at least 80% power.

This experiment, and the following experiment, were approved by the University of California, Davis Institutional Review Board and informed consent was obtained from all participants prior to testing.

2.1.2. Materials

The stimuli consisted of 160 greyscale frontal photographs of male and female faces with neutral expressions, which were cropped at the neck. The male faces were found from Internet searches and from databases available courtesy of Michael J. Tarr (Center for the Neural Basis of Cognition and Department of Psychology, Carnegie Mellon University, http://www.tarrlab.org/). The female faces were obtained from the University of Texas at Dallas Center for Vital Longevity face database (Minear & Park, 2004). The original images were used to create separate sets of altered images for the global and local change conditions. Global changes involved configural alteration of the original images by either slightly contracting (‘pinching’) them inward or slightly expanding (‘spherizing’) them outward. Local changes involved featural alteration of the original images by either adding or removing a facial feature (e.g., moles, facial hair, scars, etc.) or by lightening or darkening a facial feature (e.g., eyes, lips, etc.). All of the original and altered images were then duplicated and rotated in the picture plane 180°. This process resulted in four sets of face stimuli with distinct orientation-change profiles: upright global, upright local, inverted global, and inverted local (see Figure 1B for examples of the face stimuli). For each of the four stimulus sets, two separate trial lists, with half same and half different trials, were created to ensure that each image was tested on both same and different trials across participants. Order of image presentation was controlled for by counterbalancing stimulus pairings and image order across trials and lists, and same and different trials were always presented in random order.

2.1.3. Procedure

Participants were randomly assigned to one of four between-subject conditions that varied only in the stimulus set used. All other procedures were identical across conditions. Participants completed a total of 160 trials presented in random order – half same and half different trials. Prior to the experiment, participants scrolled through four sample face pairs (two pairs of same faces and two pairs of different faces) to familiarize themselves with the types of changes to expect, and then completed four practice trials. For each participant, all testing was completed in a single 30-minute session.

Stimuli were presented on a black background at a viewing distance of approximately 55 cm. Each trial began with a centrally-presented fixation cross (+) for 1500 ms, followed by a face image for 1500 ms, and then a dynamic noise mask for 50 ms. The dynamic noise mask comprised three separate noise masks presented sequentially for 17 ms, 17 ms, and 16 ms. The corresponding identical (same trials) or alternate (different trials) face was then presented and participants had to indicate how confident they were that the two faces were the same or different. Participants made ‘same’/’different’ judgements using a 6-point confidence scale which was visible at the bottom of the screen. Specifically, participants indicated their level of confidence that the faces were different (1=sure different, 2=maybe different, 3=guess different) or the same (6=sure same, 5=maybe same, 4=guess same). Responses were self-paced and input using the numbers 1 through 6 on a keyboard. The second face image and the response scale remained on the screen until a response was made, after which the next trial would initiate. An example of a ‘different’ trial from the inverted local condition is illustrated in Figure 1A.

2.1.4. Data Analysis

In this and the subsequent experiment, same/different confidence ratings from the change detection task were used to plot ROCs for each participant, and aggregate ROCs were plotted for group comparisons. This is done by plotting hits (y-axis) and false alarms (x-axis) across varying levels of response confidence. The leftmost point of the ROC represents the highest confidence ‘same’ response and points extending rightward represent cumulative hit and false alarm rate probabilities. Intermediate points of the ROC represent lower confidence ‘same’ (from left) and ‘different’ (from right) responses, with decreasing confidence as the midpoint of the ROC is approached. ROCs were fit to the DPSD model using maximum likelihood estimation in order to estimate the free parameters of perceiving and sensing (for additional details on how these ROC parameter estimates are obtained see Aly & Yonelinas, 2012; Goodrich & Yonelinas, 2016; Yonelinas, 1994, 2001). According to the DPSD model, perceiving and sensing make independent, yet joint, contributions to perception and they differentially influence the shape of the ROC. The probability of perceiving is reflected by the upper x-intercept of the ROC – the further left it is shifted, the higher the obtained estimate of perceiving-based responding. On the other hand, the estimate of sensing is reflected by the degree of ROC curvilinearity – the further the ROC curves away from the chance diagonal, the greater the obtained estimate of sensing-based responding (see Figure 2A).

Bayesian analyses – producing Bayes factors – were conducted on the ROC estimates for sensing using noninformative Jeffreys priors for the population variance and a Cauchy prior for the standardized effect size. Because we tested evidence for the null hypothesis, we inverted the Bayes factor (i.e., 1/BF) such that larger values indicate greater evidence for the null. By convention, a Bayes factor BF₀₁>3.16 indicates substantial evidence in favor of the null hypothesis and a Bayes factor BF₀₁<0.316 indicates substantial evidence in favor of the alternative hypothesis (Jeffreys, 1961).

To test the effects of orientation and global/local changes on overall discrimination accuracy and estimates of perceiving and sensing, we conducted 2 (orientation: upright/inverted) × 2 (change: global/local) ANOVAs. When Levene’s test indicated unequal variances the adjusted degrees of freedom are reported.

2.2. Results and Discussion

Visual examination of the aggregate ROCs (Figure 3A) shows that face inversion led to a decrease in perceptual discrimination for both the global and local change conditions, as indicated by ROCs closer to the chance diagonal for inverted faces (empty squares) compared to upright faces (filled circles). Overall discrimination accuracy was measured using d’ (i.e., zHIT - zFA) and indicated that performance was significantly lower for inverted faces (M=1.01, SE= 0.06) than for upright faces (M=1.43, SE=0.06), as shown by a main effect of orientation, F(1,87)=25.20, p<.001, ${\eta }_p^2=0.23$. There was neither a main effect of the type of change (p=.432) nor an orientation × change interaction (p=.583), indicating that inversion disrupted overall change detection performance for both locally and globally manipulated faces.

Figure 3.

ROC change detection performance in Experiment 1. (A) Aggregate ROCs for upright and inverted faces with global changes and for upright and inverted faces with local changes. Filled circles = upright faces; empty squares = inverted faces. (B) ROC parameter estimates of perceiving and sensing, measured as probability and d’, respectively. Error bars depict ±1 SE. ***p < .001

Given the face inversion effect seen for discrimination accuracy, ROC parameters were examined to assess the contributions of perceiving and sensing to performance (Figure 3B). For perceiving, there was a main effect of orientation, F(1,87)=59.41, p<.001, ${\eta }_p^2=.41$, such that estimates of perceiving were significantly reduced for inverted faces (M=0.24, SE=0.02) compared to upright faces (M=0.48, SE=0.02). There was also a main effect of the type of change, F(1,87)=34.63, p<.001, ${\eta }_p^2=.29$, with greater estimates of perceiving for local (M=0.45, SE=0.02) than for global changes (M=0.27, SE=0.02), meaning local changes coincided with a greater contribution of perceiving to performance than did global changes. The orientation × change interaction reached significance as well, F(1,87)=9.11, p=.003, ${\eta }_p^2=.10$, which reflected the fact that the reduction in perceiving for global changes (upright: M=0.43, SE=0.03; inverted: M=0.10, SE=0.03) was more than double that seen for local changes (upright: M=0.52, SE=0.02; inverted: M=0.38, SE=0.03). Note that post-hoc t-tests revealed reliable differences in perceiving between upright and inverted faces for both global, t(31.41)=6.48, p<.001, and local changes, t(46)=4.08, p<.001. Thus, inversion disrupted perceiving-based perception of faces with either type of change, but the decrement in performance was especially pronounced when faces differed globally rather than locally.

For sensing, there was a main effect of the type of change, F(1,87)=34.02, p<.001, ${\eta }_p^2=.28$, with greater estimates of sensing for global (M=0.77, SE=0.05) than for local changes (M=0.37, SE=0.05). However, there was no evidence that face inversion affected sensing as indicated by a nonsignificant effect of orientation (p=.982) and a nonsignificant orientation × change interaction (p=.538). Consistent with these results, Bayesian analysis provided substantial support (BF₀₁=4.54) for the null hypothesis that face inversion does not affect sensing-based change detection. Therefore, while global changes coincided with a greater contribution of sensing to performance compared to local changes, inversion did not affect sensing-based perception of faces with either type of change.

In contrast to our initial expectations, the results of Experiment 1 indicated that inverting faces disrupted perceiving-based discriminations but did not impact sensing-based discriminations. The selective effect of inversion on perceiving was observed whether the perceptual change involved a global difference in the configuration of the faces or an alteration of a localized individual feature. The reduction in perceiving-based responding was greater for faces that had been globally rather than locally altered which implies that inversion was particularly disruptive when the visual changes were more configural in nature, consistent with theories proposing face inversion is more detrimental for configural than featural information (Bartlett & Searcy, 1993; Tanaka & Farah, 1993; Young, Hellawell, & Hay, 2013). These findings suggest that face inversion does not disrupt the sensing of a difference between faces (i.e., change detection), rather it selectively disrupts the ability to consciously perceive specific differences (i.e., change identification). Thus, sensing-based responses do not appear to be particularly sensitive to the holistic properties of the visual materials. Rather, the results support the idea that inversion disrupts orientation-dependent holistic processing leaving only an inefficient part-based search process available and, thus, leads to a decrement in identifying specific details.

3. Experiment 2

Experiment 2 was conducted to test the replicability of the unexpected results of Experiment 1 with a different sample of participants. In addition, the study was designed to test the generalizability of those results using additional measurement methods. Specifically, we repeated the upright and inverted local conditions from Experiment 1, but with the addition of two more response components. First, following their ‘same’/’different’ confidence judgement, participants were asked to introspect about their subjective experience during that judgement and indicate whether they felt they had perceived a specific difference that they could report, or they just sensed that the faces were the same or different. This allowed us to directly assess whether the parameters derived from the ROC method corresponded to subjective reports of perceiving and sensing, and to determine if the inversion-related decrease in ROC estimates of perceiving was accompanied by a decrease in subjective reports of perceiving. Second, if participants made a ‘different’ response, they were asked to report what aspect of the face they believed had changed. In this way, we could assess whether decreases in parameter estimates of perceiving were accompanied by decreases in the objective ability to identify the specific changes in the presented stimuli.

Note that in Experiment 2 we only included the face stimuli with local changes because these types of changes allowed us to unambiguously determine if participants could identify the specific changes that were made to the faces. In contrast, with the globally changed faces it was difficult to objectively identify specific feature changes given that the change manipulation was subtle and extended across the extent of the face.

3.1. Methods

3.1.1. Participants

We decided to increase our sample sizes from Experiment 1 due to the additional measures collected for each participant in Experiment 2. Therefore, we aimed for 30 participants per condition, and slightly oversampled to account for later exclusion of participants due to data quality concerns. Power analyses based on the effect sizes of d’ (Cohen’s d=0.87, two-tailed) and perceiving (Cohen’s d=1.06, two-tailed) from the local conditions of Experiment 1 indicated that this would be sufficient to provide 90% power to detect a face inversion effect on d’ and 98% power to detect a face inversion effect on perceiving.

Sixty-eight undergraduates from the University of California, Davis participated in Experiment 2 in exchange for psychology course credit. Nine participants were excluded from analyses (seven from the upright condition and two from the inverted condition): two for not using the subjective response options as instructed and seven for subjective sensing estimates that were negative or undefined. All remaining participants reported having normal or corrected-to-normal vision. In total, 59 participants make up this data set (45 female; mean age=20; range 18–40 years). Participants were randomly assigned to view one of two face stimulus sets (see Materials): upright local (n=27) and inverted local (n=32).

3.1.2. Materials

The stimuli consisted of the upright local and inverted local face stimulus sets used in Experiment 1. Only the local change stimuli were used because the manipulations made for these images were unambiguous and better suited for assessing the qualitative, verbal descriptions that accompany perceiving-based responses. The trial lists and counterbalancing procedures were the same as those used in Experiment 1.

3.1.3. Procedure

Participants were randomly assigned to one of two between-subject conditions that varied only in the stimulus set used. For each participant, all testing was completed in a single 60-minute session. All other procedures were identical to Experiment 1 with the exception that on each trial, following their confidence judgement, and while the second face image was still present, participants also provided a ‘perceive’ or ‘sense’ response based on their subjective perceptual experience. ‘Perceive’ and ‘sense’ responses were input using designated keys on a keyboard. Additionally, if a ‘different’ response was given (i.e., a 1, 2, or 3 confidence response), participants verbally described the basis for their response, which was recorded by the experimenter. For example, a ‘perceive different’ response may be followed by a detailed description of what exactly changed between the two faces, whereas a ‘sense different’ response would be followed by a report of just knowing the faces were different without being able to describe exactly what changed. For the trial example shown in Figure 1A, a typical ‘perceive different’ response could be “He had a goatee in the first image, but it was not there in the second image”. Conversely, a typical ‘sense different’ response could be “I just know his face was different in the second image, but I don’t know how or why”.

Instructions for making ‘perceive’/’sense’ responses were thoroughly explained to participants prior to testing. Participants were told to make a ‘perceive’ response only if they had the conscious experience of perceiving that the two faces were exactly the same or different and were able to provide details about how the faces were the same or different. Participants were told to make a ‘sense’ response if they sensed, or just felt, that the two faces were the same or different but could not provide any details about what made the faces the same or different. Even if participants were highly confident in their judgement, if they could not provide any details about how the faces were the same or different, they were instructed to give a ‘sense’ response. For the practice trials only, participants were asked to justify their ‘perceive’/’sense’ responses to ensure that they understood the distinction between them. If the distinction was not fully understood the instructions were repeated.

3.1.4. Data Analysis

The same/different confidence ratings were used to plot ROCs and to derive DPSD model estimates of perceiving and sensing, as in the previous experiment. Because several participants in Experiment 2 produced truncated ROCs (e.g., they made no high-confidence false alarms), which biases parameter estimation, the model was modified to correct for the truncation by relaxing the assumption that the perceive parameter contributes only to the highest confidence response bin (see Yonelinas & Parks, 2007). Bayesian analyses were conducted on both the ROC estimates and subjective responses for sensing to test evidence for the null hypothesis as in Experiment 1.

Estimates of subjective perceiving and sensing were derived from the ‘perceive’/’sense’ responses independent of the confidence responses (Aly & Yonelinas, 2012). To incorporate hits and false alarms, perceiving (P_d) was estimated as:

$${\text{P}}_{\text{d}}=P("{\text{P}}_{\text{d}}"|\mathrm{different})-P("{\text{P}}_{\text{d}}"|\mathrm{same})$$

Sensing (S) was estimated as the probability of making a ‘sense’ response, given that a ‘perceive response was not made. This was done separately for same and different trials:

$${\text{S}}_{\mathrm{hits}}=\frac{P(\text{"}{\text{S}}_{\text{d}}\text{"}|\mathrm{different})}{[1-P(\text{"}{\text{P}}_{\text{d}}\text{"}|\mathrm{different})]}$$

$${\text{S}}_{\mathrm{false}\mathrm{alarms}}=\frac{P(\text{"}{\text{S}}_{\text{d}}\text{"}|\mathrm{same})}{[1-P(\text{"}{\text{P}}_{\text{d}}\text{"}|\mathrm{same})]}$$

The probability of sensing was then converted into a d’ score using the inverse of the standard normal cumulative distribution (i.e., ‘normsinv’) and subtracting false alarms from hits:

$$\text{S}\left(\mathrm{d\text{'}}\right)=\text{normsinv}\left({\text{S}}_{\mathrm{hits}}\right)-\text{normsinv}\left({\text{S}}_{\mathrm{false}\mathrm{alarms}}\right)$$

To score the verbal descriptions for different trials in which the participant gave a ‘different’ response (i.e., hits), the participant’s transcribed report was compared to a detailed written description of the aspect that had actually been altered. When the report corresponded to the specific alteration that had occurred, the response was scored as a ‘correct detail’. When the report corresponded to another (unaltered) aspect of the image, the response was scored as an ‘incorrect detail’. When the report included no details about what had changed (i.e., when participants felt the images were different but could not describe how), the response was scored as ‘no detail’. Verbal descriptions for same trials in which the participant gave a ‘different’ response (i.e., false alarms) were scored as either ‘detail’ or ‘no detail’. Detail accuracy was then calculated by subtracting the proportion of same trials in which a change detail was reported (i.e., detail false alarms) from the proportion of different trials in which a correct change detail was reported (i.e., correct detail hits). This was done separately for ‘perceive’ and ‘sense’ subjective responses, for both upright and inverted faces. Detail accuracy was used as an index of participants’ conscious access to veridical change information.

To test the effect of face inversion on ROC estimates and subjective estimates of perceiving and sensing, two-tailed independent samples t-tests were conducted. Additionally, correlations between the ROC and subjective estimates of perceiving and sensing were assessed to determine their correspondence. Finally, two-tailed independent samples t-tests were conducted to test the effect of face inversion on detail accuracy.

3.2. Results and Discussion

Visual examination of the aggregate ROCs (Figure 4A) shows that face inversion led to a decrease in performance as reflected by an ROC that was closer to the chance diagonal for inverted faces. Overall discrimination accuracy (d’) was significantly worse for inverted faces (M=1.38, SE=0.08) than for upright faces (M=1.61, SE=0.06), t(57)=2.27, p=.027, replicating the face inversion effect for local changes in Experiment 1. Examination of the ROC parameters (Figure 4B) also echoed the results from Experiment 1. ROC estimates of perceiving were significantly reduced, t(57)=2.41, p=.019, for inverted faces (M=0.46, SE=0.02) compared to upright faces (M=0.54, SE=0.03), whereas ROC estimates of sensing were not significantly different (p=.338).

Figure 4.

ROC change detection performance in Experiment 2. (A) Aggregate ROCs for upright and inverted faces with local changes. Filled circles = upright faces; empty squares = inverted faces. (B) ROC parameter estimates of perceiving and sensing. Error bars depict ±1 SE. *p < .05

An examination of subjective reports of perceiving and sensing also indicated that face inversion led to a reduction in perceiving and did not impact sensing (Figure 5). Subjective ‘perceive’ responses were significantly reduced, t(57)=2.48, p=.016, for inverted faces (M=0.44, SE=0.02) compared to upright faces (M=0.51, SE=0.02). There was no significant difference (p=.172) in subjective ‘sensing’ responses as a function of face orientation. However, we note that there was a numerical reduction in sensing – particularly for subjective responses – and so we tested evidence for the null hypothesis using Bayesian analyses. The Bayes factors for both ROC estimates (BF₀₁=2.55) and subjective reports (BF₀₁=1.70) of sensing suggested that the data were more probable under the null than the alternative hypothesis. However, given the size of the Bayes factors (Jeffreys, 1961), this is not strong support for the null but rather provides only anecdotal evidence that the null hypothesis is more plausible than the alternative hypothesis.

Figure 5.

Subjective change detection performance in Experiment 2. Subjective parameter estimates of perceiving and sensing for upright and inverted faces with local changes. Error bars depict ±1 SE. *p < .05

If our ROC estimates of perceiving and sensing accurately reflect participants’ subjective experiences of these distinct perceptual sub-processes, then there should be a direct correspondence between the ROC and subjective estimates (Figure 6). Indeed, there were strong positive correlations between the ROC and subjective estimates for perceiving (upright: r=0.85, p<.001; inverted: r=0.78, p<.001) and for sensing (upright: r=0.51, p=.007; inverted: r=0.51, p=.003).¹ This is consistent with previous work (Aly & Yonelinas, 2012, Experiment 4a) and demonstrates that our inferred ROC estimates of perceiving and sensing correspond to individuals’ subjective experiences of perceiving- and sensing-based perception for both upright and inverted faces. It should be noted that, even though the sensing correlations are weaker than the perceiving correlations, they are still moderately strong (rs>.50) suggesting a significant overlap in the underlying processes.

Figure 6.

Correlations between ROC parameter estimates and subjective parameter estimates for perceiving and sensing, for upright and inverted faces with local changes, in Experiment 2. Filled circles/solid lines = upright faces; empty squares/dashed lines = inverted faces. ***p < .001, **p < .01

In order to verify that face inversion did disrupt the ability of the participants to identify specific perceptual features that differed between faces, we assessed their ability to correctly report exactly what had changed in the face. An examination of the proportion of trials in which participants could accurately report the changed visual detail (i.e., hits) and the proportion of trials in in which participants inaccurately reported a changed visual detail (i.e., false alarms) was conducted. For the items that were correctly perceived as different (upright proportion=0.55; inverted proportion=0.52), participants were much more likely to correctly report the specific detail that changed (upright=0.48; inverted=0.45) than they were to report an incorrect detail (upright=0.06; inverted=0.06). In contrast, for items correctly judged as different on the basis of sensing (upright=0.14; inverted=0.13) participants were no more likely to report the correct detail that changed (upright=0.06; inverted=0.05) than they were to report an incorrect detail (upright=0.06; inverted=0.05). Similarly, for the items that were incorrectly perceived as different (upright proportion=0.04; inverted proportion=0.08), participants were more likely to report a specific detail they believed had changed (upright=0.04; inverted=0.07) than they were to report no detail (upright=0.00; inverted=0.01). For items incorrectly judged as different on the basis of sensing (upright=0.11; inverted=0.11) participants were also more likely to report a detail they believed had changed (upright=0.09; inverted=0.07) than they were to report no detail (upright=0.02; inverted=0.04). Thus, face inversion selectively influenced perceiving-based rather than sensing-based responses.

To determine whether the effect of face inversion on perceiving was associated with a reduced ability to extract accurate information about detailed differences in upright versus inverted faces we calculated detail accuracy scores by subtracting the proportion of same trials in which a change detail was reported from the proportion of different trials in which a correct change detail was reported. This was done independently for subjective ‘perceive’ and ‘sense’ responses, for both upright and inverted faces. The detail accuracy scores (see Figure 7) for ‘perceive’ responses were significantly greater than zero for upright and inverted faces (ps<.001), whereas the detail accuracy scores for ‘sense’ responses were significantly less than zero for upright faces (p=.016) and not significantly different from zero for inverted faces (p=.098). We then assessed whether the perceiving-based face inversion effect would be echoed in the detail accuracy scores. Detail accuracy for ‘perceive’ responses was significantly lower for inverted faces (M=0.38, SE=0.02) than for upright faces, (M=0.45, SE=0.02), t(57)=2.09, p=.041, but did not differ between upright and inverted faces for ‘sense’ responses (p=.494). This suggests that the observed perceiving-based face inversion effect is due to decreased access to veridical change information for inverted compared to upright faces.

Figure 7.

Detail accuracy for items judged to be changed on the subjective basis of perceiving and sensing in Experiment 2. Accuracy was measured as the proportion of change trials in which the changed detail was correctly identified minus the proportion of non-change trials incorrectly identified as having a changed feature. Error bars depict ±1 SE. *p < .05

These findings corroborate the results of the ROC analysis in showing that the estimates of perceiving and sensing were highly correlated with the subjective reports of perceiving and sensing and, in addition, they converged in showing that the effects of face inversion were only significant for perceiving-based responses. Nonetheless, we do acknowledge that we cannot rule out some small effect of face inversion on sensing. However, only in the case of perceiving-based responses were participants able to correctly identify the specific featural changes. Moreover, the results verified that face inversion did in fact lead to a significant reduction in the ability to correctly identify the specific perceptual changes that had been made. Thus, it seems that it is the high-confidence, conscious, perceptual identification of discrete differences between faces that is disrupted by face inversion and this, in turn, results in overall worse discrimination accuracy for inverted relative to upright faces. Altogether, these results confirm and bolster our conclusions from Experiment 1.

4. ROC Model Comparisons

The ROC analyses in Experiments 1 and 2 made use of the Dual Process Signal Detection (DPSD) model to estimate perceiving- and sensing-based responses. The model was supported by the observed convergence of the derived parameters with those observed in the subjective reports and the objective measure of feature change detection, and this joins results from previous studies that have provided evidence for the validity of that model (Aly et al., 2013; Aly & Yonelinas, 2012; Goodrich & Yonelinas, 2016). Nevertheless, it is useful to examine the ROCs in light of alternative ROC models.

For example, the Equal Variance Signal Detection (EVSD) model assumes that a strength/discriminability (d’) parameter alone describes the shape of the ROC (Macmillan & Creelman, 2005; Swets, 1973; Yonelinas & Parks, 2007). To assess whether the results could be accounted for using the simpler, single-parameter EVSD model, we compared it to the DPSD model, as well as another common two-parameter signal detection model: the Unequal Variance Signal Detection (UVSD) model (Swets, 1973; Yonelinas & Parks, 2007). The UVSD model assumes the two parameters that describe the shape of the ROC are strength/discriminability (d’) and variance ratio (V_d). Strength represents the discriminability of same and different items – as measured by the distance between the means of the same and different item distributions (Figure 2B) – and is reflected by the degree of ROC curvilinearity. The variance ratio (V_d) represents the ratio of the variance for the different item distribution relative to the variance for the same item distribution, and the estimate of V_d is reflected by the degree and direction of ROC asymmetry. If the variances of the same and different item distributions are similar, then the ROC will be symmetrical and V_d will be equal to one. If the variance of the same item distribution is greater than that of the different item distribution, then the ROC will be asymmetrically pushed up on the right side of the function and V_d will be greater than one. In prior studies, the V_d parameter of the UVSD model has behaved similarly to the P_d parameter of the DPSD model such that an increase in high-confidence perceiving responses can lead to an increase in the variance of the different item distribution (Aly & Yonelinas, 2012; Goodrich & Yonelinas, 2016)

For both Experiments 1 and 2, we directly compared the EVSD model to the more complex UVSD and DPSD models using the change in log-likelihood ratio G² test for nested models as a comparative fit index. Because the additional parameter (i.e., perceiving for DPSD and variance ratio for UVSD) in the more complex models can vary between participants, G² test statistics were calculated separately for each participant and then summed, for each model. Across Experiments 1 and 2, the DPSD model provided a significantly better fit to the observed data than the EVSD model. This was true for both upright faces [Experiment 1: G²(1)=455.74, p<.001; Experiment 2: G²(1)=324.41, p<.001] and inverted faces [Experiment 1: G²(1)=222.23, p<.001; Experiment 2: G²(1)=420.31, p<.001]. The UVSD model also provided a significantly better fit than the EVSD model for both upright faces [Experiment 1: G²(1)=467.99, p<.001; Experiment 2: G²(1)=355.78, p<.001] and inverted faces [Experiment 1: G²(1)=228.75, p<.001; Experiment 2: G²(1) =408.57, p<.001]. Altogether, these results indicate that single parameter estimates of discrimination, such as d’, are not appropriate for assessing perceptual performance in this task. Rather, two separable components are required. Moreover, this provides further evidence that perceiving-based and sensing-based judgements do not simply reflect differences in the strength of perceptual evidence.

Because the DPSD and UVSD ROC models are not nested they cannot be directly compared. However, we did contrast their fit using Bayesian Information Criteria (BIC) as a goodness-of-fit index in which smaller values indicate better fit, and differences of less than two are not considered meaningful (Kass & Raftery, 1995). BIC contrasts from both Experiments 1 and 2, indicated the two models fit the observed data equally well (all ∆BIC<2; see Table 1 for BIC values). The parameters produced by the DPSD and UVSD models from Experiments 1 and 2 are presented in Table 1, as well as those produced by the EVSD model for comparison. An examination of the parameter estimates in Table 1 shows that the DPSD model indicated face inversion led to a decrease in perceiving (Pd) but did not impact sensing (S). Moreover, the effect of face inversion was larger for globally manipulated faces than for locally manipulated faces. The UVSD-derived parameters indicated that face inversion led to a decrease in V_d (the variance of the different item distribution) for global changes, but a decrease in d’ (sensitivity) for local changes. Face inversion also led to a decrease in d’ for the local changes examined in Experiment 2. Why face inversion differentially impacted the two parameters of the UVSD model in the global and local change conditions is not clear. However, the results are broadly consistent with the DPSD model results in the sense that the condition in which face inversion led to the largest decrease in perceiving (i.e., the global change condition), also led to the largest decrease in variance ratio. This is consistent with prior work that has found that manipulations that lead to a decrease in the highest confidence ‘different’ responses tend to reduce estimates of P_d and estimates of V_d (Aly & Yonelinas, 2012; Goodrich & Yonelinas, 2016; Yonelinas & Parks, 2007).

Table 1.

Experiment 1 and 2 Parameter Estimates and Fit Statistics by ROC Model

Experiment 1
		Local Changes		Global Changes
		Upright	Inverted	Upright	Inverted
DPSD	Pd	0.52	0.38***	0.43	0.10***
	S	0.39	0.35	0.75	0.80
	BIC	39.10	39.10	39.86	39.67
UVSD	Vd	1.55	1.45	1.42	1.09***
	d’	0.92	0.68**	1.09	0.90
	BIC	39.24	38.92	39.15	39.56
EVSD	d’	1.32	1.03**	1.39	0.93***
	BIC	34.16	33.84	34.08	34.49
Experiment 2
		Local Changes
		Upright	Inverted
DPSD	Pd	0.54	0.46*
	S	0.52	0.44
	BIC	40.48	39.86
UVSD	Vd	1.50	1.48
	d’	1.08	0.92*
	BIC	39.32	40.23
EVSD	d’	1.44	1.22**
	BIC	34.24	35.16

Note. Parameter estimates and Bayesian Information Criteria (BIC) goodness-of-fit indices obtained by fitting each ROC model to the observed data for Experiment 1 (upper) and Experiment 2 (lower). Smaller BIC values indicate better fit and ∆BIC < 2 are not considered meaningful. The DPSD and UVSD model BIC values cannot be directly compared to the EVSD model BIC values due to differences in the number of free parameters. Abbreviations: DPSD = dual process signal detection; UVSD = unequal variance signal detection; EVSD = equal variance signal detection model; P_d = perceiving; S = sensing; V_d = variance ratio; d’ = sensitivity. Significance values indicate a significant difference between upright and inverted faces.

^***p < .001

^**p < .01

^*p < .05

The three ROC models were further assessed by examining how well each model’s parameters tracked the subjective reports of perceiving and sensing, and the detail accuracy scores from Experiment 2 (see Table 2). Given the large number of correlations that we examined we only discuss correlations exceeding a significance level of p<.01. For the DPSD model, estimates of perceiving correlated highly with subjective reports of perceiving and detail accuracy scores, whereas estimates of sensing were highly correlated with subjective reports of sensing. These results confirm that the perceiving and sensing parameters derived from the ROC analysis are validated by the subjective reports and the objective measures of correct change identification performance. For the UVSD model, both d’ and V_d correlated with subjective reports of perceiving and with detail accuracy scores, whereas only d’ correlated with subjective reports of sensing. For the EVSD model, the estimates indicated that overall discrimination (d’) was correlated with subjective reports of perceiving, subjective reports of sensing, and our objective measure of correct change identification.

Table 2.

Experiment 2 Correlations for ROC Model Parameter Estimates, Subjective Reports, and Detail Accuracy

	DPSD		UVSD		EVSD
	Pd	S	Vd	d’	d’
Subjective P	0.83***	0.27	0.54***	0.78***	0.84***
Subjective S	0.19	0.49***	0.09	0.42*	0.45**
Detail Accuracy	0.74***	0.38	0.40*	0.78***	0.79***

Note. Pearson correlation coefficients for each ROC model’s parameter estimates, the subjective reports of perceiving and sensing, and detail accuracy scores. Abbreviations: DPSD = dual process signal detection; UVSD = unequal variance signal detection; EVSD = equal variance signal detection model; P_d = perceiving; S = sensing; V_d = variance ratio; d’ = sensitivity.

^***p < .0001

^**p < .001

^*p < .01.

The model analysis indicated that the perceiving and sensing estimates from the DPSD model were the most selective in predicting subjective reports and the likelihood of correctly identifying a changed detail, whereas V_d and d’ showed selectivity to a lesser extent. That is, knowing an individual’s ROC estimates of perceiving and sensing is highly informative about that individual’s subjective experience and their conscious access to veridical change information. On the other hand, knowing an individual’s ROC estimates of V_d and d’ is much less informative because both parameters are highly associated with the subjective experience of perceiving a specific change and the objective probability of correctly reporting that change. The finding that the DPSD model was more useful in predicting subjective reports and objective change identification performance argues in favor of that model. But on the basis of model fits of the ROC data alone, the DPSD and UVSD models fit equally well, and so the ROC results in the current study can be well accommodated by both models. However, we note that previous studies that have directly contrasted these two models in perception and working memory have found that the DPSD model often provides a better fit (Aly et al., 2013; Aly & Yonelinas, 2012; Goodrich & Yonelinas, 2016).

5. General Discussion

In two change detection experiments, we examined the effect of inverting a face on the ability to detect subtle perceptual differences between the two faces. Consistent with many previous studies we found that overall discrimination accuracy was worse for inverted faces than for upright faces. However, the current studies revealed that this performance deficit was driven by reductions in perceiving-based responses rather than sensing-based responses. Thus, face inversion did not reduce the likelihood that the subjects sensed there was a difference between the faces, but rather it selectively reduced the ability to perceive the specific details of the faces that had changed. This was true for faces that differed either globally or locally, although the magnitude of the face inversion effect for globally changed faces was more than double that observed for locally changed faces (Experiment 1). The results for locally changed faces were replicated in Experiment 2, verifying that the face inversion effect was driven by differences in perceiving, but not sensing, whether the perceptual sub-processes were measured via confidence-based ROCs or subjective reports. Furthermore, the perceiving-based face inversion effect coincided with a decreased ability to accurately report the specific details that differed between faces when they were inverted compared to upright. Altogether, these findings suggest that the face inversion effect is the result of disrupted perceiving-based perception and diminished conscious access to veridical change information for faces that are upside down.

The finding that face inversion disrupted perceiving to a greater extent than sensing is counter to our initial expectation that inverting faces would primarily disrupt the ability to make sensing-based discriminations. This expectation was based on previous work showing that global manipulations that impacted configural information in the visual materials, rather than just a single feature, selectively increased sensing-based discriminations (Aly & Yonelinas, 2012). To the extent that face inversion disrupts holistic processing of configural information (Diamond & Carey, 1986; Farah et al., 1995; Freire et al., 2000; Leder & Bruce, 1998; Leder & Carbon, 2006; Searcy & Bartlett, 1996; Tanaka & Simonyi, 2016), we expected to see a sensing-based face inversion effect. The current results clearly contradicted this expectation and instead indicated that face inversion had little to no effect on sensing, rather it primarily disrupted perceiving. Importantly, sensing was not simply insensitive to experimental manipulation, as Experiment 1 indicated that sensing was significantly greater in the global than in the local change conditions (also see Aly & Yonelinas, 2012 for similar results with objects, fractals, and buildings).

It is important to acknowledge that the conclusions of the current studies were based in part on results from a relatively new dual process model-based ROC analysis of perception. However, the conclusions that we drew from that approach were also independently verified by both subjective reports and by objective measures of perceptual detail recall. That is, in Experiments 1 and 2 the dual-process ROC approach indicated that face inversion selectively reduced perceiving-based responses rather than sensing-based responses. In Experiment 2, we also assessed performance by directly asking subjects to report when they consciously perceived a specific change and when they made their judgements on the basis that they just sensed the faces were the same or different. Those reports also indicated that face inversion selectively reduced perceiving rather than sensing. In addition, we asked subjects to report the specific aspects of the faces that had changed and again we verified that face inversion selectively reduced the ability to report specific details that had changed without impacting the responses that were not associated with accurate detection of a changed feature. Thus, the primary conclusions of the current study are supported by both the modelling approach and by direct behavioral measures.

Nonetheless, a direct assessment of model fits to the observed ROCs indicated that both the DPSD model and the UVSD model provided equally good statistical fits to the observed ROCs. Although the current study was not designed to contrast these alternative models, we believe that the DPSD model of perception is a useful one, and that there are several reasons to prefer the dual-process approach when examining visual perception. First, as just described, the current results validated the DPSD model parameters by showing that these parameters were available to subjective reports (i.e., they were phenomenologically real) and they accurately predicted the ability to detect specific details (i.e., they tracked the objective ability to identify the specific local changes that were manipulated in the faces). In contrast, neither of the UVSD model parameters were found to selectively track these subjective or objective reports. In addition to the current results, previous perception studies have directly contrasted the DPSD and UVSD model fits to ROCs in visual perception tasks, and these studies have found that the DPSD model provided a significantly better fit to the observed ROCs (Aly et al., 2013; Aly & Yonelinas, 2012).

So why does face inversion disrupt performance primarily by reducing perceiving-based rather than sensing-based discriminations? We interpret the results to suggest that face inversion interfered with the participants’ ability to use their extensive prior experience with upright faces to guide efficient visual search for changed features in inverted faces. This is in line with prior work indicating that experts are better than novices at making fine perceptual discriminations about stimuli from their area of expertise (Buttle & Raymond, 2003; Clark et al., 2011; Curby et al., 2009; Diamond & Carey, 1986; Myles-Worsley et al., 1988; Rhodes et al., 1989; Werner & Thies, 2000). Moreover, it is broadly consistent with a number of theoretical proposals about the nature of the face inversion effect. For example, it has been argued that the orientation-dependent holistic representation of upright faces facilitates rapid and efficient processing across the entire extent of the face by providing top-down attentional guidance of visual search (Barton et al., 2001; Endo, 1986; Malcolm et al., 2004; Richler et al., 2012; Rossion, 2008). Similarly, it has been suggested that face inversion disrupts the formation and use of holistic or object-based representations leaving only a slower, part-based search strategy available that no longer benefits from top-down attentional guidance (Barton et al., 2006; Curby et al., 2013). Although these accounts do not make specific predictions about how sensing-based responses should be impacted by face inversion, they are consistent with the finding that face inversion acts to reduce the likelihood of perceiving specific changed features.

Additional support for this interpretation comes from eye-tracking studies of change detection which find that upright faces elicit a higher concentration of fixations on the eyes and nose regions, whereas inverted faces elicit a higher concentration of fixations on the mouth and nose regions (Barton et al., 2006; Xu & Tanaka, 2013). That is, viewing behavior was deployed in a spatially similar manner for upright and inverted faces (i.e., towards the top half of the image), but the visual information present at the oft-fixated location was not equivalently informative for upright and inverted faces (i.e., eyes-nose region vs. mouth-nose region). Additionally, when viewing inverted faces, participants make significantly more saccades (Xu & Tanaka, 2013), have longer scanning durations, and exhibit more random scan patterns (Barton et al., 2006); again suggesting visual search patterns are more efficient for upright compared to inverted faces. Moreover, upright faces may be searched more consistently across the face pairs than are inverted faces (e.g., the viewing patterns for the first and second faces in a pair may overlap more for upright faces). Any of these differences could reduce the likelihood that participants attend to and focus on the changed features in the inverted faces and so result in a reduction in perceiving. Future studies examining eye movements related to perceiving and sensing will be useful for assessing these possibilities.

The current finding that face inversion impacted perceiving rather than sensing appears to be inconsistent with an earlier result in which face inversion was found to have a greater negative impact on change detection than on change identification (Wilford & Wells, 2010). However, there are several methodological differences that may have led to these apparent differences. For example, all of the changes in Wilford and Wells (2010) were featural so their results are most comparable to our local change condition – the condition for which we observed a smaller effect of inversion. In addition, in our change detection task and the change detection task used by Wilford and Wells (2010), half of the trials were same and half were different, whereas in their change identification task all of the trials were different. Thus, participants may have adopted a serial search strategy in the identification task which could have reduced the magnitude of the face inversion effect in their change identification condition.

Although the perceiving-based face inversion effect was seen for faces with either local or global changes, the magnitude of this effect was more than twice as large for globally differing faces. While this is consistent with research indicating that face inversion disrupts perceptual processing of configural (global) more than featural (local) information (Diamond & Carey, 1986; Freire et al., 2000; Leder & Bruce, 2000; Leder & Carbon, 2006; Maurer et al., 2002; Rhodes et al., 1993; Rossion, 2008; Searcy & Bartlett, 1996; Valentine, 1988), it is also possible that the manipulations used for our local face stimuli inadvertently dampened the face inversion effect for local changes. That is, in a comprehensive review, McKone and Yovel (2009) found that the magnitude of feature inversion effects can be reduced if the feature change includes a color/brightness manipulation. Given that some of the faces in the local change condition of the current study involved changes in brightness this may have reduced the observed inversion effect seen for faces with local feature changes. Further studies directly contrasting the effects of different feature changes will be important in determining the factors that impact the magnitude of face inversion effects.

Our findings speak to the effects of face inversion on perceiving and sensing in a change detection task in which the face images are either identical or not, and all faces were presented as front views. However, previous studies have found that configural processing is viewpoint independent and that face inversion effects are still present for faces presented from different viewpoints (Favelle & Palmisano, 2012; Hills, Sullivan, & Pake, 2012; McKone, 2008). In these studies, face perception and recognition require identifying differently angled views of the same face as being the same person, which adds image differences that are irrelevant to the task. To date, research examining perceiving and sensing in perception has used only viewpoint invariant stimuli. How perceiving and sensing would behave in a change detection task with stimuli that vary in viewpoint is unknown and future studies are needed to test how irrelevant differences across images affect these processes.

In the current study, we focused on the effects of inverting faces on perceiving- and sensing-based perception. However, prior work has indicated that inversion effects are also observed with other mono-oriented objects such as buildings (Diamond & Carey, 1986; Scapinello & Yarmey, 1970; Yin, 1969) or for objects of expertise (Curby et al., 2009; Diamond & Carey, 1986; Rhodes et al., 1989). Future studies will be needed to determine whether the inversion effects observed with other types of materials also selectively impact perceiving. We have begun to address this issue in a follow-up to Experiment 1, in which we examined the effect of inversion on buildings rather than faces. Although the effects of inversion were quite subtle for buildings, building inversion disrupted perceiving-based responses rather than sensing-based responses, and these effects were most pronounced for buildings with global changes compared to buildings with local changes. Thus, the inversion effects on perceiving that we observed for faces may well turn out to be relatively general for mono-oriented objects.

The face inversion effect has been an area of intense research since Yin’s (1969) seminal work investigating the influence of orientation on perception and memory for a variety of mono-oriented objects, including faces. While this is far from the first study to examine the face inversion effect and its possible underlying mechanisms, it is the first to do so using confidence-based ROCs to assess how this phenomenon influences the contributions of known perceptual sub-processes (i.e., perceiving and sensing) to discrimination performance. Using a psychophysical approach, along with subjective reports, to dissociate these sub-processes we showed that disrupted holistic processing of inverted faces coincides with a reduction in perceiving-based judgements concerning discrete, localized differences and diminished conscious access to such differences. These results are important in further understanding the face inversion effect and what types of perception are, and are not, involved in holistic processing. In particular, we found that not all perceptual processes are equally affected by inversion. This highlights the limitations of single parameter measures of performance when examining the face inversion effect, given the selective effect of inversion on perceiving-based discriminations and the better fit of a model with two separable components. We hope that future studies of the face inversion effect, and face processing in general, are encouraged to employ continuous response designs (e.g., confidence-based ROCs) and dual process models of performance (e.g., perceiving and sensing) to elucidate the complex and intricate nature of face perception.

Context Paragraph.

Previous work has shown that perceptual and working memory decisions can be based on two different processes that independently, yet jointly, contribute to overall performance: perceiving and sensing. Moreover, perceiving-based responses contribute more to performance for images (fractals, faces, buildings) that have been locally manipulated by changing a specific feature, whereas sensing-based responses contribute more to performance for images that have been globally manipulated by changing the relational aspects between features. Given that the face inversion effect (i.e., worse discrimination performance for inverted vs. upright faces) is often thought to reflect a disruption in holistic processing, we aimed to determine whether the face inversion effect would also coincide with lower estimates of sensing for inverted compared to upright faces.