Problems in using d′ measures to assess subjective awareness

Abstract

An accurate method of assessing the awareness to primes is crucial to investigations of subliminal perception. In a recent paper, Vermeiren and Cleeremans (2012) contend that traditional measures of prime detection potentially overestimate awareness to the prime. This can lead to wrongly classifying effects as subliminal, since even when primes are detected at chance level, it could be due to factors other than visibility. Here, I address this and point to another fundamental issue that is inherent to d′ calculations, and has to be considered when using signal detection methods to assess awareness. In subliminal perception studies, unconscious processing of the stimuli is assumed when d′ is not significantly different from zero. However, this is a null finding that leaves open the possibility of there being differences that the statistical test was not sensitive enough to detect. Hence, reported subliminal effects, especially small effects, could have occurred as a consequence of visible trials even when d′ did not significantly differ from chance. Therefore, additional measures such as bootstrapping, which could complement d′ in ensuring that the effects were not a consequence of a small number of trials, should be utilized in future studies.

1. Introduction: Using d´ measures to assess awareness

In the subliminal perception literature, there has been an ongoing debate over whether subjective measures provide an accurate and reliable method to determine awareness, or whether objective measures are necessary (for summaries of these discussions, see Merikle & Reingold, 1990; 1998; Reingold & Merikle, 1990; Snodgrass, Bernat, & Shevrin, 2004). With subjective measures, awareness is assessed on the basis of the observers' self-reports of their conscious experience, so that if people report that they were aware of a stimulus it is assumed that it was consciously perceived, and if they report that they did not perceive the stimulus it is inferred that they were unaware of it. Even though this measure fits well with our intuitive understanding of conscious and unconscious perception as subjective phenomena, it is often difficult to operationalize in a sufficiently controlled manner, and a simple lack of confidence may contribute to someone's reluctance to report having been aware of a particular stimulus, despite having some subjective experience of it (Cleeremans, 2001; Vermeiren & Cleeremans, 2012). Hence, most current studies utilize an objective measure, where awareness is assessed on the basis of the observers' forced-choice decisions regarding different stimulus states (d′ measures). It is assumed that any ability to discriminate between alternative stimulus states at a better than chance level indicates that the stimuli were perceived with awareness, whereas the absence of awareness to the stimuli is determined as an inability to discriminate between them at an above chance level (Merikle, Smilek, & Eastwood, 2001). Subliminal perception is established by showing that “primes exert an indirect influence on participants' behavior, but fail to reach awareness in a direct d′ test” (Vermeiren & Cleeremans, 2012).

2. Problems with d´ measures

In a recent study, Vermeiren and Cleeremans (2012) evaluated the methods used to compute objective measures of awareness, showing that the standard d′ measure is not a straightforward way of assessing visibility. More specifically, it is suggested d′ can be contaminated by other factors besides visibility, and thus may overestimate the visibility of the primes. The main concern of the paper is stated thus:

While we agree that the claim of unconscious perception is established, at least in a functional sense, when d′ is zero, one cannot claim that primes were actually not consciously perceived at the time they were presented. Other factors, such as target interference, might hinder participants' ability to report on primes they were weakly aware of at the time of presentation.

The authors go on to identify three such factors: the distribution of attention, target presence, and the SOA between the prime and the response.

The first point I would like to make is that if it is true that d′ does not reflect visibility alone, the claim that unconscious perception is established when d′ is zero is not always true. If factors other than visibility can reduce d′ to chance level (zero), then primes that exert an influence on behavior for which d′ is zero could nevertheless be subjectively visible. As pointed out by the authors, there is a “clear influence of different variations of the d′ task on the resulting d′ values.” Owing to space limitations, I will not provide a detailed account of their results, but the point can be clarified using a simple example such as the influence of task difficulty on d′: someone can perform at chance level simply due to the task being very difficult (e.g. requiring a very fast response, the correct response is difficult to determine, etc.), while being consciously aware of the stimuli presented.

Second, even if all those factors are accounted for, there is still a fundamental issue that is inherent in d′ calculations and makes it difficult to completely rule out the possibility that all primes were not consciously perceived at the time of their presentation. In subliminal perception studies, unconscious processing of the stimuli is assumed when d′, as is measured across a sample of trials, is not significantly different from zero (chance performance). However, this is a null finding, which still leaves open the possibility of there being differences that the statistical test was not sensitive enough to detect. Practically speaking, even when d′ is not statistically different from zero, and even if we assume that the only influencing factor on d′ is prime visibility, there could still be a number of trials in which the prime was visible which was not large enough to reach statistical significance.

An important corollary is that it is possible that some supposedly subliminal effects that have been reported (d′ was not significantly different from zero) are in reality due to trials in which the subjects were aware of the prime. This potential problem is especially important to consider when analyzing small effects of supposedly subliminal stimuli. For instance, in a recent electrophysiological (ERP) study by Lu, Zhang, Hu, and Luo (2011), a small (∼0.6 μV) but significant difference between the VPP elicited by neutral faces which were primed by masked negative and positive faces was observed. Although there are reasons to believe that unconsciously processed stimuli should elicit similar but weaker stimulus-specific neural activity (see for instance Shevrin, 2001), we nevertheless cannot rule out the possibility that some (although undoubtedly not all) of the small effects that were observed in this study were generated by a small number of visible trials. Citing another example, Morris, Pelphrey, and McCarthy (2007) reported a small (less than 0.08%) but significant difference between the hemodynamic responses elicited by masked faces and masked objects (in that study the hit rate was reported instead of d′, but the two measures are complimentary). Again, while it is certainly not necessarily the case, a contamination by visible trials cannot be completely ruled out.

Hence, the above suggests that it is advisable that in addition to d′, other measures should be taken in order to ensure that stimuli are consistently invisible. Although it is always sensible to complement d′ with subjective measures, this would not necessarily eliminate the problem, since confidence issues would still apply (the visible trials could be reported as invisible due to a lack of confidence in seeing the stimulus). A different approach that could prove useful is to utilize bootstrapping methods in order to test whether supposedly subliminal effects are generated by visible or invisible trials. If the observed effect is a consequence of subliminal trials that consistently generate a similar response, it should be present also after the trials are resampled with replacement. On the contrary, if the effect is generated by only a small proportion of visible trials, it should not occur consistently across samples (because the probability of sampling visible trials would be small in this case).

3. Concluding remarks

In summary, I have claimed that it is possible that even when d′ does not significantly differ from zero, there nevertheless could be trials in which the prime is visible. This is due to both an overestimation of d′ due to other factors that are unrelated to visibility and also because chance is determined as d′ not being statistically different from zero, which does not preclude the possibility that our statistical tests were not sensitive enough to detect a difference. To address this issue in future studies, additional methods such as bootstrapping should be applied, especially when the observed effects are small and a contamination by visible trials is suspected.