Evaluating comparative and equality judgments in contrast perception: Attention alters appearance

Katharina Anton-Erxleben; Jared Abrams; Marisa Carrasco

doi:10.1167/10.11.6

. Author manuscript; available in PMC: 2011 Aug 16.

Published in final edited form as: J Vis. 2010 Sep 9;10(11):6. doi: 10.1167/10.11.6

Evaluating comparative and equality judgments in contrast perception: Attention alters appearance

Katharina Anton-Erxleben ¹, Jared Abrams ², Marisa Carrasco ³

PMCID: PMC3156576 NIHMSID: NIHMS312622 PMID: 20884501

Abstract

Covert attention not only improves performance in many visual tasks but also modulates the appearance of several visual features. Studies on attention and appearance have assessed subjective appearance using a task contingent upon a comparative judgment (e.g., M. Carrasco, S. Ling, & S. Read, 2004). Recently, K. A. Schneider and M. Komlos (2008) questioned the validity of those results because they did not find a significant effect of attention on contrast appearance using an equality task. They claim that such equality judgments are bias-free whereas comparative judgments are bias-prone and propose an alternative interpretation of the previous findings based on a decision bias. However, to date there is no empirical support for the superiority of the equality procedure. Here, we compare biases and sensitivity to shifts in perceived contrast of both paradigms. We measured contrast appearance using both a comparative and an equality judgment. Observers judged the contrasts of two simultaneously presented stimuli, while either the contrast of one stimulus was physically incremented (Experiments 1 and 2) or exogenous attention was drawn to it (Experiments 3 and 4). We demonstrate several methodological limitations of the equality paradigm. Nevertheless, both paradigms capture shifts in PSE due to physical and perceived changes in contrast and show that attention enhances apparent contrast.

Keywords: attention, appearance, psychophysical methods, contrast perception, spatial vision

Introduction

Attention and appearance

Selective attention enhances processing of behaviorally relevant aspects of a visual scene at the cost of others. Thereby attention enables us to deal with the enormous amount of available sensory information. Attending covertly (without eye movements) to a particular location in space affects performance both when attention is allocated voluntarily—endogenous or sustained attention—or involuntarily—exogenous or transient attention (Egeth & Yantis, 1997; Posner, 1980; Yantis, 2000). Attention improves accuracy, spatial resolution, and processing speed for stimuli at the attended location (e.g., Cameron, Tai, Eckstein, & Carrasco, 2004; Carrasco, Giordano, & McElree, 2006; Carrasco & McElree, 2001; Carrasco, Penpeci-Talgar, & Eckstein, 2000; Dobkins & Bosworth, 2001; Hawkins et al., 1990; Ling & Carrasco, 2006a; Liu, Stevens, & Carrasco, 2007; Nakayama & Mackeben, 1989; Posner, 1980; Talgar, Pelli, & Carrasco, 2004; Yeshurun & Carrasco, 1998, 1999) while it impairs performance at the unattended locations (Giordano, McElree, & Carrasco, 2009; Ling & Carrasco, 2006b; Lu & Dosher, 1998; Luck et al., 1994; Montagna, Pestilli, & Carrasco, 2009; Pestilli & Carrasco, 2005; Pestilli, Viera, & Carrasco, 2007; Posner, 1980).

Whereas these effects of attention on visual performance are well established, a recent series of experiments demonstrates that attention also modulates the phenomenological appearance of several low-level stimulus features, a proposition that has been debated for more than a century (James, 1890/1950; von Helmholtz, 1910). To test subjective perception, a paradigm has been developed in which observers perform a task contingent upon a comparative judgment between two stimuli, while an uninformative cue directs spatial attention to one of the stimuli (Carrasco, Ling, & Read, 2004). For example, observers are asked to report the orientation of the higher contrast stimulus. By pressing one key, observers convey information regarding both properties, and they implicitly report their subjective experience of contrast. Changes in apparent contrast are measured in terms of shifts of the point of subjective equality (PSE), at which the two stimuli appear equal. This 2 × 2 alternative forced choice (AFC) paradigm enables an objective and rigorous study of attention and subjective experience (Luck, 2004; Treue, 2004). Using this paradigm, attention has been shown to alter appearance of contrast (Carrasco, Fuller, & Ling, 2008; Carrasco et al., 2004; Fuller, Park, & Carrasco, 2009; Fuller, Rodriguez, & Carrasco, 2008; Hsieh, Caplovitz, & Tse, 2005; Liu, Abrams, & Carrasco, 2009; Störmer, McDonald, & Hillyard, 2009), spatial frequency (Abrams, Barbot, & Carrasco, 2010; Gobell & Carrasco, 2005), gap size (Gobell & Carrasco, 2005), color saturation (Fuller & Carrasco, 2006), size of a moving object (Anton-Erxleben, Henrich, & Treue, 2007), motion coherence (Liu, Fuller, & Carrasco, 2006), flicker rate (Montagna & Carrasco, 2006), and speed (Fuller et al., 2009; Turatto, Vescovi, & Valsecchi, 2007). An additional advantage of the 2 × 2 AFC paradigm is that it provides concurrent assessment of appearance and performance: In addition to altering appearance, attention improves performance at the cued location, indicating that attention has been successfully manipulated (Abrams et al., 2010; Anton-Erxleben et al., 2007; Carrasco et al., 2004; Fuller & Carrasco, 2006; Fuller et al., 2008; Ling & Carrasco, 2007; Liu et al., 2006, 2009).

In principle, the effect on appearance could also be consistent with a bias that leads observers to preferentially select the cued side of space. Several kinds of control experiments have ruled out such a cue, response or decision bias. First, when the order of cue and stimulus is reversed (postcue), a simple cue bias should persist. Instead, the effect disappears, supporting an attentional interpretation (Anton-Erxleben et al., 2007; Carrasco et al., 2008; Fuller et al., 2009; Gobell & Carrasco, 2005; Liu, Pestilli, & Carrasco, 2005). Second, when the cue–stimulus interval is lengthened to 500 ms, the effect of the cue disappears, consistent with the limited time scale of transient attention (Carrasco et al., 2004; Ling & Carrasco, 2007; Liu et al., 2006; Turatto et al., 2007). Third, a bias could arise because attention makes the secondary task (e.g., orientation discrimination) easier, so that observers might develop a tendency to select the attended stimulus, regardless of its apparent contrast. However, when observers are asked to report the orientation of the stimulus of lower, rather than higher, contrast, they choose the cued test stimulus less frequently. According to a bias explanation, observers should have chosen the cued stimulus more often regardless of the direction of the question. Instead, this result is consistent with a genuine effect of attention on appearance (Anton-Erxleben et al., 2007; Carrasco et al., 2004; Fuller & Carrasco, 2006; Gobell & Carrasco, 2005; Ling & Carrasco, 2007; Liu et al., 2009; Montagna & Carrasco, 2006; Turatto et al., 2007).

Furthermore, a bias should be of similar magnitude independent of the stimulus properties and visual field location, but this is not the case. For instance, attention increases perceived saturation but it does not affect apparent hue, even though it improves performance in both cases (Fuller & Carrasco, 2006). Also, the effect of attention on apparent size varies inversely with standard stimulus size (Anton-Erxleben et al., 2007). Moreover, the effect of attention on apparent contrast is greater at the lower than at the upper visual vertical meridian (Fuller et al., 2008). These studies support the view that the effects of attention are not the result of a bias but rather an effect on phenomenological appearance.

Despite the evidence provided by these previous studies, the question of attentional modulation of appearance remains a matter of debate: an alternative interpretation proposes that decision biases occur in any experiment combining a cue with a comparative judgment such as evaluating which of two stimuli appears higher in contrast. Schneider and Komlos (2008) proposed that the decision bias is tied to the deployment of attention. They suggest that attention might simply change the stimulus’ salience, so that the cued stimulus is prioritized leading observers to preferentially select it for their judgment. According to this account, the increase in salience need not be related to an increase in apparent contrast. They argue that the results from control experiments with a postcue and with a longer cue-target SOA are consistent with their salience-based account. Furthermore, they contend that in the reversed instructions control experiments, even when observers are asked to select the stimulus of lower contrast, observers could still associate the cued target with higher salience and simply invert their responses.

Schneider and Komlos (2008) have suggested that a paradigm that uses an equality judgment, in which observers are asked to report whether two stimuli are the same or different in contrast, provides a definite answer to settle the question of whether attention alters apparent contrast. They argue that in a comparative paradigm, PSE and criterion are confounded, whereas in an equality judgment the PSE is independent of criterion shifts. Using the equality task, they did not find a significant effect of attention on contrast appearance and thus concluded that the effects of attention in previous studies (Anton-Erxleben et al., 2007; Carrasco et al., 2004, 2008; Fuller & Carrasco, 2006; Fuller et al., 2008; Gobell & Carrasco, 2005; Ling & Carrasco, 2007; Liu et al., 2006; Montagna & Carrasco, 2006; Turatto et al., 2007) were due to a bias. The approach of using an equality judgment to disentangle an attentional effect on perception from a decisional account is intriguing. However, an examination of the validity and effectiveness of the equality paradigm is essential. It must be demonstrated that the equality task is indeed bias-free and at least as sensitive to shifts in perceived contrast as the comparative task.

Evaluating comparative and equality paradigms

Evaluating comparative and equality paradigms is relevant for visual psychophysics beyond the field of attention: comparative judgments have been used to study many different visual phenomena; for instance, adaptation effects on perceived speed (Hammett, Champion, Morland, & Thompson, 2005; Ledgeway & Smith, 1997; Stocker & Simoncelli, 2009), contrast (Hammett, Snowden, & Smith, 1994), and shape (Gheorghiu & Kingdom, 2007), perceived spatial frequencies at different eccentricities (Davis, Yager, & Jones, 1987) and at different contrasts and durations (Georgeson, 1985), speed perception (Smith & Edgar, 1990; Stone & Thompson, 1992), size perception (Charles, Sahraie, & McGeorge, 2007), vertical and horizontal meridian asymmetries (Fuller et al., 2008; Montaser-Kouhsari & Carrasco, 2009), illusions (Carlson, Moeller, & Anderson, 1984), cue combination (Ho, Landy, & Maloney, 2008), lightness constancy (Todd, Norman, & Mingolla, 2004), and perception of 3D surface geometry (Ernst & Banks, 2002; Knill & Saunders, 2003). Should the equality judgment indeed give more bias-free estimates and at the same time be sufficiently sensitive to changes in appearance, previous interpretations of many results might have to be reevaluated.

Several psychophysical studies have investigated sensitivity and biases of comparative and equality tasks (e.g., Coltheart & Curthoys, 1968; Farell, 1985; Fetterman, Dreyfus, & Stubbs, 1996; Gorea, Caetta, & Sagi, 2005; Hautus & Lee, 1998; McKee, Klein, & Teller, 1985; Ratcliff & Hacker, 1981; Wichmann & Hill, 2001a, 2001b). In a comparative 2-AFC judgment, a bias for giving one over the other response, i.e. a shift in criterion, and a shift in PSE are indistinguishable without additional control experiments (e.g. Anton-Erxleben et al., 2007; Carrasco et al., 2004, 2008; Schneider & Bavelier, 2003; Schneider & Komlos, 2008; Shore, Spence, & Klein, 2001). Reliability of PSE estimation is affected by the slope of the psychometric function (McKee et al., 1985) as well as stimulus placement along the stimulus dimension of interest (McKee et al., 1985; Wichmann & Hill, 2001a).

Generally, in an equality judgment, the relation between criterion settings and the parameter estimates can be complex. The equality judgment replaces the one criterion of the comparative judgment with two criteria for “different because greater” and “different because lesser” responses. In the equality judgment employed by Schneider and Komlos (2008), the psychometric function is an approximately bell-shaped distribution of “same” responses, with the PSE corresponding to the peak of this distribution. The two criteria on each flank of the PSE determine the width and amplitude of the distribution, reflecting the general tendency for responding “same.” Indeed, in an equality judgment of letter strings, instructions to the observers (e.g., “say same only if you are sure the stimuli are the same”) affected not only speed and accuracy but also the overall proportion of “same” versus “different” responses (Ratcliff & Hacker, 1981). In Schneider and Komlos’ (2008) paradigm, a bias for giving one over the other response would in principle change width and amplitude without affecting the location of the PSE (Schneider & Komlos, 2008). This assumption however only holds when the two criteria are symmetric. Asymmetric criterion settings can bias the location of the PSE. Empirically, the symmetry of the two criteria is often violated and should therefore be verified in each experiment (Petrov, 2009). However, in Schneider and Komlos’ (2008) implementation of the equality paradigm symmetry of criteria was assumed, but the two criteria could not be independently calculated. Thus, the assumption that criterion shifts do not affect PSE estimation in their equality paradigm has yet to be tested.

Even if criteria were symmetric, a change in width and amplitude of the psychometric function likely influences the reliability of PSE estimation: If observers were strongly biased either to respond “same” or “different,” the distribution would become very shallow. Such a strong bias is particularly likely to occur if the underlying proportion of “same” and “different” trials is unbalanced, as was the case in Schneider and Komlos’ (2008) study, where “same” trials were presented only on ~11% of the trials. Such extreme probabilities lead to conservatism: If observers are unaware of the true prior odds, they overestimate the probability of occurrence of the rare event, so that their estimated odds are roughly equal to the cube root of the true prior odds (Green & Swets, 1966; Maloney, 2002). Thus, criterion settings could influence the sensitivity of the paradigm so that an effect of attention might be present but simply not captured. Furthermore, the assumption that the different parameters of a psychometric function are independent from each other has been directly tested and found to be unwarranted: The slope parameter of a psychometric function affects threshold estimation; specifically, a shallow slope impairs reliability of the threshold estimate (McKee et al., 1985). Differences in reliability between the comparative and equality paradigms have not been tested in previous studies.

Furthermore, a study that compared performance in judging a visual stimulus’ duration using comparative and equality judgments revealed that accuracy in the equality task was significantly lower than in the comparative task, indicating greater difficulty of the equality task (Fetterman et al., 1996).

Aims of the current study

Based on the lack of an attentional effect with an equality paradigm, Schneider and Komlos (2008) proposed that attention does not alter appearance but rather biases decisions. However, three alternative explanations could account for their null finding: (1) There is no empirical support for the superiority of equality judgments over comparative judgments. It is possible that the equality paradigm in general is not bias-free or not sensitive to attentional effects on appearance or both. (2) Their study, in particular, might lack sufficient statistical power, and with increased power they could have found an effect with the equality task. (3) Their study lacks a baseline condition, which is essential to assess the effect of attention—instead, they compare perceived contrast in the attentional condition to the physical contrast, assuming that contrast perception without attention is veridical. However, this is an empirical question and the assumption of veridicality is not warranted (Carrasco et al., 2008; Treue, 2004).

The aims of the present study are twofold: First, we systematically investigate sensitivity and biases in comparative and equality judgments. We evaluate and compare the sensitivity and biases of the equality task as employed by Schneider and Komlos (2008) and the comparative task implemented by Carrasco and colleagues (2004) to changes in physical contrast (Experiments 1 and 2). Given that in these experiments the contrast differences are physically present, they allow us to evaluate strengths and weaknesses of the two tasks without a potential influence of attention on either appearance or decision processes. Given that comparative judgments have been used to study a variety of visual phenomena (e.g. Carlson et al., 1984; Charles et al., 2007; Davis et al., 1987; Ernst & Banks, 2002; Fuller et al., 2008; Georgeson, 1985; Gheorghiu & Kingdom, 2007; Hammett et al., 1994, 2005; Ho et al., 2008; Knill & Saunders, 2003; Ledgeway & Smith, 1997; Montaser-Kouhsari & Carrasco, 2009; Smith & Edgar, 1990; Stocker & Simoncelli, 2009; Stone & Thompson, 1992; Todd et al., 2004), this systematic comparison has implications beyond the question of whether attention alters appearance.

Second, we evaluate and compare the effect of attention on apparent contrast with equality and comparative tasks (Experiments 3 and 4). If the equality paradigm were bias-free and at least as sensitive as the comparative paradigm (Experiment 1) and if the equality paradigm were not to reveal significant effects of attention (Experiment 3), then these findings would challenge our interpretation of previous results regarding the effects of cueing on the appearance of contrast (Carrasco et al., 2004, 2008; Fuller et al., 2008, 2009; Hsieh et al., 2005; Ling & Carrasco, 2007; Liu et al., 2009; Störmer et al., 2009) and of other visual attributes (Abrams et al., 2010; Anton-Erxleben et al., 2007; Carrasco et al., 2008; Fuller & Carrasco, 2006; Fuller et al., 2009; Gobell & Carrasco, 2005; Liu et al., 2006; Montagna & Carrasco, 2006; Turatto et al., 2007).

Experiments 1 and 2: Physical contrast increments

In Experiments 1 and 2, observers judged the contrasts of two simultaneously presented stimuli. We incremented the contrast of one stimulus to simulate the hypothesized effect of attention by changing the physical contrast. Any method that is sensitive to a change in apparent contrast must be able to capture corresponding differences in physical contrast. We chose three different levels of contrast enhancement (3%, 6%, and 9% away from the reference contrast) to test whether the equality and comparative paradigms would differ in the smallest effect size they could detect. The magnitude of the three contrasts increments was chosen so that the medium level roughly corresponded to the effect size reported with exogenous or transient attention (Carrasco et al., 2004, 2008; Fuller et al., 2008, 2009; Ling & Carrasco, 2007). The only difference between the two experiments was the instruction: In Experiment 1, observers were asked to report if the contrasts were the same or different; in Experiment 2, they were asked to indicate the location of the stimulus of higher contrast.

Experiments 1 and 2 were designed to assess biases and sensitivity of comparative and equality judgments such as the one used by Schneider and Komlos (2008); therefore, we kept our equality task very similar to theirs. However, in order to maximize the sensitivity of both methods, we varied stimulus contrast in finer steps than previous studies (Carrasco et al., 2004, 2008; Fuller et al., 2008, 2009; Ling & Carrasco, 2007; Schneider & Komlos, 2008).

Methods

Observers

Nine observers (4 male and 5 female; mean age = 28 years, SD = 5) participated in both experiments for remuneration (as well as experiments 3 and 4, see below); the order of all four experiments was randomized so that no two observers encountered the same order. All observers were naive to the purpose of the experiments and had normal or corrected-to-normal vision. Some observers were experienced and others were inexperienced in visual psychophysical tasks. The institutional review board of New York University approved all procedures.

Apparatus

Experiments were performed in a dark experimental room. Stimuli were presented on a calibrated and linearized CRT monitor (IBM P260) with a viewable area of 40 × 30 cm, a resolution of 1 cm/deg (corresponding to 32 pixels/deg), and a refresh rate of 106 Hz. Stimuli were presented on a medium gray background (53 cd/m²). Observers used a chin rest positioned at a distance of 57 cm from the monitor. Experiments were run on an Apple Macintosh computer (iMac). Stimulus presentation and recording of the observers’ responses was controlled by a custom MATLAB (The MathWorks, Natick, MA) script using the Psychophysics Toolbox extension (Brainard, 1997; Pelli, 1997).

Stimuli and trial sequence

Figure 1A shows the trial sequence for Experiments 1 and 2. Each trial started with the presentation of a dark (<1 cd/m²) fixation square of 0.2 deg side length at the center of the screen. After 510 ms, a test and a standard stimulus were presented simultaneously at 4 deg eccentricity left and right of fixation for 50 ms. Within each condition, the contrast of the standard stimulus was fixed whereas the contrast of the test stimulus was varied (see Figure 1B). Stimuli were stationary 4-cpd Gabors with a Gaussian envelope of 1 deg diameter at half height. In each trial, both Gabors were tilted either 45° to the left or to the right of vertical. The phase of each Gabor was randomly varied. Both phase shifts and tilts were introduced to minimize adaptation. In the baseline condition (no increment), the standard stimulus had a Michelson contrast of 26% while the test contrast was varied in 19 steps on a logarithmic scale between 10% and 67% (Figure 1B). In six increment conditions, either the test or the standard contrast was incremented by either ~0.05, ~0.1, or ~0.15 log contrast steps (Figure 1B). For the 26% contrast standard stimulus these increments correspond to ~3%, 6%, or 9% contrast. The seven different conditions (six increments and one baseline) were interleaved and presented with the same frequency in random order within each block. In each trial, the test contrast was randomly chosen from the 19 values with equal probability.

Experimental design. (A) In both increment Experiments (1 and 2), each trial started with a fixation period of 510 ms; then two Gabors were presented for 50 ms at 4 deg eccentricity left and right of fixation. In the equality judgment (Experiment 1), observers were asked to indicate if the contrasts of the two stimuli were the same or different, while in the comparative judgment (Experiment 2), they had to report which stimulus had higher contrast. (B) Illustration of the increment conditions in Experiments 1 and 2. For each, the numbers on the left show the increment magnitude (log contrast), the axes on the right illustrate the resulting standard (thick tick mark) and test contrast (thin tick marks). No increment (baseline, black), test incremented by 0.05 (dark blue), 0.1 (medium blue), 0.15 (light blue) log contrast steps, or standard incremented by 0.05 (dark red), 0.1 (medium red), or 0.15 (light red) log contrast steps. (C) In the attention Experiments (3 and 4), the fixation period was followed by a cue which was flashed for 70 ms at fixation (neutral condition) or at 1.5 deg above the center of one of the Gabors (test cued and standard cued conditions). After an interstimulus interval of 50 ms, the two Gabors were presented for 50 ms, and observers reported if the contrasts were the same or different or which stimulus had higher contrast in the equality (Experiment 3) and comparative (Experiment 4) judgments, respectively. Drawings are not to scale.

Procedure

Experiments consisted of two 1-hour sessions performed on different days. Sessions were divided into 4 blocks of 350 trials, so that each observer completed 2800 trials. This corresponds to ~21 trials per data point (133 combinations of condition and test contrast). In the beginning of each block, observers received written instructions on the screen, asking either “Do the two stimuli have the same or different contrast?” (Experiment 1) or “Which stimulus has higher contrast?” (Experiment 2). In all experiments, a 2-AFC paradigm was used. In the equality task (Experiment 1), observers were asked to indicate if both stimuli appeared to have the same or a different contrast by pressing either the “s” or the “d” key, respectively. In the comparative task (Experiment 2), they pressed the left-arrow and right-arrow keys to indicate that the left or the right stimulus had higher contrast, respectively. Responses could be given immediately after offset of the Gabors and the fixation square; response time was unlimited.

Analysis

For the equality experiment (Experiment 1), we analyzed the proportion of “same contrast”—responses as a function of the logarithm of the physical test contrast for each increment condition. Using non-linear regression in MATLAB (The MathWorks, Natick, MA), these data were fit with a scaled Gaussian function with the parameters mean (which is equivalent to the PSE), standard deviation, and amplitude (scale factor). For the comparative experiment (Experiment 2), we analyzed for each increment condition the proportion of trials in which an observer responded “test had higher contrast” as a function of the logarithm of the physical test contrast. We fit these data with a cumulative Gaussian function with the parameters mean and standard deviation. For both models, the standard deviation parameter was constrained to positive values, while the amplitude parameter in Experiment 1 was constrained to be between 0 and 1, as the frequency of responding “same” can never be more than 100%. Data points at different contrast levels were weighted proportionally to their number of repetitions. Effects of judgment type and increment were analyzed with a repeated measures ANOVA using SPSS (SPSS Inc., IL). Whenever Mauchly’s test of sphericity indicated a violation of the assumption of sphericity, we report Greenhouse–Geisser-corrected p-values; otherwise, uncorrected p-values are reported. Unless indicated otherwise, all subsequent pairwise comparisons were two-tailed.

Results

Goodness of fit

The scaled Gaussian model fit the data from the equality judgment (Experiment 1) reasonably well; average R² was 0.8539 (SEM = 0.0357). The data from the comparative experiment (Experiment 2) were well described by the cumulative Gaussian model (average R² = 0.9690, SEM = 0.0070). For all further analysis in Experiments 1 and 2, we excluded one observer who had R²s < 0.7 in several conditions in the equality experiment (Experiment 1).

Estimation of PSE

Figure 2A shows the average results of the equality experiment (Experiment 1). For each increment condition, the proportion of “same” responses is plotted as a function of baseline test contrast, together with the scaled Gaussian fit to the average data. Figure 2B shows the average data for the comparative task (Experiment 2). For each increment condition, we plot the proportion of trials in which observers reported that the test stimulus had higher contrast as a function of baseline test contrast together with the cumulative Gaussian fit to the average data.

Results Experiments 1 and 2 (Physical increments). (A) Experiment 1: Frequency of reporting same contrast as a function of no-increment test contrast (baseline) for each increment condition averaged across observers. Increment conditions: test incremented by 0.15 (light blue, *), 0.1 (medium blue, ×), or 0.05 (dark blue, +) log contrast steps, no increment (black, ○), standard incremented by 0.05 (dark red, □), 0.1 (medium red, ◆), or 0.15 (light red, △) log contrast steps. Error bars are standard error of the mean. Solid lines are fits to the average data. (B) Experiment 2: Frequency of reporting higher contrast as a function of baseline test contrast for each increment condition averaged across observers. Same format as A. (C and D) PSE in the test-incremented and standard-incremented conditions as a function of the PSE in the no increment condition for each observer in the equality (C) and comparative (D) experiment. Same symbols and color code as in A and B.

In the equality experiment, the PSE in the “no increment” (baseline) condition is at 23.6%, i.e., at a lower contrast than the true standard contrast (black curve, true standard contrast 26%). When the contrast of the standard stimulus is incremented by 0.05 (dark red), 0.1 (medium red), or 0.15 (light red) log contrast steps, the PSE increases to higher test contrasts of 26.0%, 29.3%, and 32.6%, respectively. When the contrast of the test stimulus is incremented by 0.05 (dark blue), 0.1 (medium blue), or 0.15 (light blue) log contrast steps, the PSE decreases to lower baseline test contrasts of 21.2%, 19.0%, and 17.1%, respectively. Table 1 summarizes the parameter estimates from the Gaussian fits.

Table 1.

Average parameter estimates with standard error across observers for the increment experiments. The POE is given in units of log contrast (leftmost column) and percent contrast (in parentheses). The remaining cells reflect the mean (PSE), standard deviation (SD), and amplitude parameters for the equality paradigm, and then the mean (PSE) and standard deviation (SD) parameters for the comparative paradigm. In each cell, the top row refers to the parameter estimate in units of log contrast and the bottom row to the standard error (in italics); for the PSE, the corresponding contrast value in percent contrast is given in parentheses.

Parameter	POE	Equality (Experiment 1) n = 8			Comparative (Experiment 2) n = 9
Parameter	POE	PSE	SD	Amplitude	PSE	SD
Test increment 3	−0.725 (19)	−0.767 (17.1)	0.173	0.776	−0.733 (18.5)	0.147
		0.021	0.048	0.083	0.013	0.052
Test increment 2	−0.679 (21)	−0.722 (19.0)	0.181	0.788	−0.687 (20.6)	0.145
		0.037	0.045	0.082	0.017	0.043
Test increment 1	−0.633 (23)	−0.674 (21.2)	0.180	0.780	−0.632 (23.3)	0.134
		0.032	0.046	0.077	0.012	0.044
No increment	−0.587 (26)	−0.629 (23.6)	0.188	0.778	−0.594 (25.5)	0.162
		0.042	0.053	0.083	0.012	0.044
Standard increment 1	−0.541 (29)	−0.586 (26.0)	0.186	0.732	−0.547 (28.4)	0.146
		0.036	0.039	0.089	0.018	0.061
Standard increment 2	−0.495 (32)	−0.534 (29.3)	0.164	0.742	−0.503 (31.4)	0.133
		0.032	0.049	0.086	0.014	0.053
Standard increment 3	−0.449 (36)	−0.487 (32.6)	0.162	0.689	−0.446 (35.8)	0.115
		0.021	0.044	0.092	0.016	0.046

Open in a new tab

In the comparative experiment, the PSE in the “no increment” (baseline) condition corresponds more closely to the true standard contrast (black curve, PSE 25.5%). When the contrast of the standard stimulus is incremented by 0.05 (dark red), 0.1 (medium red), or 0.15 (light red) log contrast steps, the PSE increases to a test contrast of 28.4%, 31.4%, and 35.8%, respectively. When the contrast of the test stimulus is incremented by 0.05 (dark blue), 0.1 (medium blue), or 0.15 (light blue) log contrast steps, the PSE decreases to a baseline test contrast of 23.3%, 20.6%, and 18.5%, respectively.

In Figures 2C and 2D, we plot the PSEs in the increment conditions against the PSEs without increment for each observer in the equality and comparative experiment, respectively. In both experiments, the PSEs in the different increment conditions are arranged in an orderly fashion, consistent with an increase in apparent contrast with contrast increment. The figure also shows that the PSE in the no increment condition is more variable across observers in the equality experiment than in the comparative experiment (F-test, F(8,7) = 0.075, p = 0.002).

We performed a two-way repeated measures ANOVA with factors experiment and increment (2 × 7) on the PSE and found a main effect of experiment (F(1,7) = 13.718, p = 0.008) and of increment (F(6,42) = 552.855, p < 0.001), but no interaction (F(6,42) < 1). The main effect of experiment confirms that the equality method consistently yields a lower estimate for the PSE than the comparative method, which gives PSE estimates closer to the true standard contrast. Figure 3 shows the PSE estimates averaged across observers as a function of the point of objective equality (POE) in each increment condition. The POE in the standard-incremented conditions correspond to the new (incremented) standard contrasts; in the test-incremented conditions, the POE is the contrast of the test stimulus that is (after incrementing) equal to the original standard contrast. For example, if the test values are incremented by 0.05 log contrast step, the 23% contrast test is incremented to 26% and therefore corresponds to the POE. Whereas the PSEs in the comparative experiment fall along the diagonal, the PSEs in the equality experiment are systematically shifted downward, consistent with an underestimation of the true standard contrast. To test the reliability with which the two methods estimate the location of the PSE, for each experiment we performed a t-test comparing the PSE in the baseline condition to the POE. We found no significant difference between PSE and POE in the comparative experiment (t(8)= 1.765, p = 0.115, n = 9), but in the equality experiment, the PSE was significantly lower than the POE (t(7) = 2.795, p = 0.027, n = 8; as indicated above, one observer was excluded from Experiment 1 due to low goodness of fit). This is likely related to the observation that for many observers, the left tail of the distribution of “same” responses was higher than the right tail and does not reach a value lower than ~20%.

PSE estimates Experiments 1 and 2 (Physical increments). Average PSE as a function of POE for each increment condition from the equality (large symbols) and comparative (small symbols) task. Lines mark the linear regression to the average data (dotted: equality, dashed: comparative experiment). Same symbols and color code as in Figure 2. Error bars are standard error of the mean.

A possible explanation for this asymmetry is that observers adopt asymmetric criteria, so that they are more likely to (incorrectly) respond “same” even at large differences between test and standard contrast when the test contrast is lower than when it is higher. This is especially puzzling because in the comparative experiment using the exact same contrast levels, the same observers are perfectly able to correctly report which of the two stimuli has higher contrast: with the lowest test contrast in the no increment condition, observers made on average ~20% errors in the equality experiment, but only ~5% in the comparative experiment.

One-way repeated measures ANOVAs for each experiment revealed a significant effect of increment on the PSE in both experiments (comparative: F(6,48) = 416.269, p < 0.001, n = 9; equality: F(2.135,14.947) = 305.774, p < 0.001, n = 8). The effect of increment shows that both methods are sensitive to shifts of the PSE when the physical contrast is altered. Pairwise comparisons indicated that in both experiments the PSEs for all increments were significantly different from one another (equality experiment: all t(7) ≥ 4.884, p < 0.002, alpha adjusted for 21 comparisons: 0.0024 at an overall significance level of 0.05; comparative experiment: all t(8) ≥ 4.873, p < 0.001).

Standard deviation and amplitude

The standard deviation parameter of the Gaussian fits represents the width of the distribution of “same” responses in the equality experiment and the slope of the psychometric function in the comparative experiment. In both paradigms, the standard deviation is determined by the observer’s ability to discriminate the different contrasts as well as his/her criterion for indicating a difference between the two stimuli. In the equality paradigm, an additional amplitude parameter was necessary. The standard deviation and amplitude parameters are both influenced by the overall proportion of “same” responses independent of actual stimulus contrast, which is dependent on the criterion: If observers are inclined to respond “same,” this is reflected in higher standard deviation and/or higher amplitude. In the equality experiment, the standard deviation and amplitude parameters are clearly interdependent (see below). Note that there is no amplitude parameter in the comparative paradigm. Therefore, the comparison of the standard deviation between the two paradigms should be interpreted cautiously.

A two-way repeated measures ANOVA (2 experiments × 7 increments) on the standard deviation parameter revealed that the standard deviation was marginally lower in the comparative than the equality experiment (F(1,7) = 4.321, p = 0.079). There was a main effect of increment (F(6,42) = 6.406, p < 0.001), but the interaction of experiment and increment was not significant (F(2.799,19.59) = 1.802, p = 0.182). In both experiments, the effect of increment was confirmed in separate one-way repeated measures ANOVAs (equality: F(6,42) = 5.194, p < 0.001; comparative: F(6,42) = 4.569, p = 0.001). Whereas the ANOVA yielded significant effects, none of the pairwise comparisons reached significance after correction for multiple comparisons (all p > 0.004, alpha adjusted for 21 comparisons: 0.0024 at an overall significance level of 0.05).

For the equality experiment, the amplitude parameter also varied significantly with increment (one-way ANOVA; F(6,42) = 6.837, p < 0.001). There is a tendency for the standard incremented—conditions to have a lower amplitude than the test incremented—conditions, although none of the pairwise comparisons reached significance after correction for multiple comparisons (all p > 0.004).

Generally, in the equality experiment, as the distributions shift to the right their amplitudes and their standard deviations become lower. These changes could be related to asymmetric criteria, which might make observers more likely to judge a low contrast test stimulus as equal to the standard than a high contrast test stimulus.

Interdependence of parameter estimates

We analyzed the baseline condition without contrast increment and found not only a strong correlation between standard deviation and amplitude in the equality experiment (Spearman’s rho (6) = 0.95, p = 0.0013), but also a trend for a negative correlation between PSE and standard deviation (Spearman’s rho (6) = −0.64, p = 0.096) as well as for PSE and amplitude (Spearman’s rho (6) = −0.70, p = 0.065). These findings suggests that the parameter independence in the equality judgment, as assumed by Schneider and Komlos (2008), is not justified. In contrast, there was no correlation between PSE and standard deviation in the comparative experiment (Spearman’s rho (7) = −0.12, p = 0.776).

To further investigate influences of criterion settings on the parameters in the equality paradigm, we conducted ideal observer simulations (see Appendix). In the equality experiment, we found that if observers deviate from the ideal observer by adjusting their criteria to the extreme probabilities of the occurrence of “same” trials (Green & Swets, 1966; Maloney, 2002), standard deviations and amplitudes became larger. These values corresponded to the empirical data more closely than the ideal observer simulation. This result suggests that observers develop a bias for responding “same.” Note that in all simulations, symmetric criteria were modeled; thus, unlike the real observers’ PSE, the simulated PSE was not biased and was independent of the other parameters.

Reaction times

We performed a two-way repeated measures ANOVA with the factors experiment and increment (2 × 7) on observer response times for the perceptual judgment. Reaction times were significantly longer in the equality than in the comparative experiment (515 ms ± 23 ms SEM vs. 380 ms ± 29 ms SEM; F(1,8) = 25.993, p = 0.001). However, there was no effect of increment (F(1.607,12.859) = 2.431, p = 0.134) and no interaction (F(1.478,11.821) < 1). The effect of experiment on reaction time is consistent with the idea that observers find the equality task more difficult (Fetterman et al., 1996).

Experiments 3 and 4: Attention

In Experiments 3 and 4, observers judged the contrasts of two simultaneously presented stimuli, one of which appeared on the same side as a transient attentional cue. The only difference between the two experiments was the instruction: In Experiment 3, we asked observers to report if the contrasts were the same or different; in Experiment 4, we asked them to report the stimulus of higher contrast.

To compare our results with those of Schneider and Komlos (2008), we kept the equality task as similar as possible to theirs, but to improve the sensitivity of the equality task, we introduced some critical differences: First, we varied stimulus contrast in finer steps. Second, we tested not only one (attentional) condition, but three conditions in which the cue was either peripheral (standard-cued or test-cued) or central (neutral cue), which is necessary to assess the attention effect.

The comparative task (Experiment 4) was similar to the study by Carrasco et al. (2004; see also Carrasco et al., 2008; Fuller et al., 2008, 2009; Ling & Carrasco, 2007), with the difference that in their study observers had to perform an orientation discrimination task contingent on the contrast discrimination; here we did not include such a contingent task to keep the paradigm comparable to the equality method in which a 2 × 2 AFC design is not possible.