Non-Verbal Communicative Signals Modulate Attention to Object Properties

Abstract

We investigated whether the social context in which an object is experienced influences the encoding of its various properties. We hypothesized that when an object is observed in a communicative context, its intrinsic features (such as its shape) would be preferentially encoded at the expense of its extrinsic properties (such as its location). In the three experiments, participants were presented with brief movies, in which an actor either performed a non-communicative action towards one of five different meaningless objects, or communicatively pointed at one of them. A subsequent static image, in which either the location or the identity of an object changed, tested participants’ attention to these two kinds of information. Throughout the three experiments we found that communicative cues tended to facilitate identity change detection and to impede location change detection, while in the non-communicative contexts we did not find such a bidirectional effect of cueing. The results also revealed that the effect of the communicative context was due to the presence of ostensive-communicative signals before the object-directed action, and not to the pointing gesture per se. We propose that such an attentional bias forms an inherent part of human communication, and function to facilitate social learning by communication.

Non-verbal Communicative Cues Modulate Attention to Object Properties

Only a small fraction of the potential information present in the visual environment is encoded by the human brain. In particular, human-made environments are full of various kinds of objects, but their presence and visual features are registered only when they become relevant for actions or other cognitive processes, and attention is directed to them (e.g., Castiello, 1999). Visual attention is controlled by endogenous processes, such as current tasks and goals, and exogenous factors that determine the salience of visual stimuli (Yantis, 1998). Among the external effects on visual attention, a special class involves social stimuli. Humans are a hypersocial species who pay much attention to human-made artefacts, i.e., socially created objects. Furthermore, humans rely on social learning processes to acquire information about the function, the use, the valence, and the social status (including ownership) related to objects and object kinds (e.g., Casler & Kelemen, 2005). Some of these social learning processes involve observing object-directed actions and attitudes of people, while other learning mechanisms extract object-relevant information from verbal and non-verbal communication. Thus, while social information allows learning about others (people's attitudes, intentions, etc.), it also enables us to learn from others, for example about object properties (e.g., the function and the use of human-made artefacts, or the edibility of plants).

Indeed, the social context can have a profound effect on object-directed attention. The best-studied phenomenon of socially modulated attention is gaze cueing, which is sometimes treated as a paradigmatic effect of 'joint attention' (for a review, see Frischen, Bayliss, & Tipper, 2007). If the participant's task is to detect the location of randomly presented objects on a computer screen, a preceding face cue gazing towards the target location (either by head turn or just eye direction) facilitates target detection (Friesen & Kingstone, 1998). This phenomenon shares features with both exogenous and endogenous attentional cueing (Posner, 1980). The gaze cue works like an endogenous cue because it is presented centrally, rather than peripherally. At the same time, it acts as an exogenous cue because its effect is automatic and attention shift is elicited even when the cue is not predictive about the location of the oncoming target (Driver et al., 1999). It has also been shown that 'social' (i.e., viewer-directed) gaze preceding the gaze cue may facilitate target detection (Bristow, Rees, & Frith, 2007), and objects that others pay attention to acquire special properties (Becchio, Bertone, & Castiello, 2008). Beyond gaze, other, non-facial object-directed actions can also direct viewers' attention to objects. For example, pointing can act the same way as a gaze cue (Langton & Bruce, 2000), and pointing to (but not grasping) an object facilitates the detection of target stimuli appearing in their location (Fischer & Szymkowiak, 2004).

Theoretical considerations suggest that, while many kinds of social cues can direct attention to particular objects, they may act differently in terms of preparing the viewer to obtain certain types of object information. It has been proposed that communicative social cues should be distinguished from other, non-communicative cues with respect to their expected effects (Csibra & Gergely, 2006, 2009). Non-communicative cues are object-directed actions, such as gazing, reaching to, or manipulating an object, which do not involve the observer in these behaviors. Such cues have natural meanings (Grice, 1957), derived from the interpretation of the performed action. For example, looking at an object indicates attention devoted to it, while reaching towards it makes it the immediate goal of the actor. By contrast, the meaning of communicative object-directed actions, such as pointing and ostensive gazing, is 'non-natural', and is to be derived from the communicative intention attributed to the actor (Grice, 1957). This non-natural meaning is referential to the object, but the addressee has to infer from the context, from other accompanying cues, or from verbal information, why the actor is highlighting the object by her action (Tomasello, 2008).

As the inclusion of object-directed gaze among both the communicative and the non-communicative cues implies, what makes an action communicative is not its relation to the object but whether or not it is addressed to someone. Making an object-directed action communicative can be achieved by preceding or accompanying it with ostensive signals, such as eye-contact, or calling the addressee's name, which make it manifest to the addressee that the action is performed for her (Csibra, 2010). For example, if a referential pointing action is preceded by eye-contact (an ostensive signal), it can make manifest the communicator's intention to convey some information about the target of the pointing action (the referential signal) to the person who was addressed by the eye-contact. Thus, we make a distinction between ostensive and referential aspects of non-verbal communication. While ostensive signals unambiguously express the communicative intention of the source, referential signals specify the referent about which the communicator is expected to convey some message. In this analysis, what makes an action communicative is that it is performed by the intention to be recognized as such by a specific audience (cf. Grice, 1957; Sperber & Wilson, 1995), and ostensive signals can be used to achieve exactly this effect (Csibra, 2010). In addition, object-directed actions, if accompanied by ostensive signals, may be interpreted as deictic referential signals, making them the vehicles of the referential intention of the actor (Becchio et al., 2008).

A recent theory has developed specific predictions for the attentional effects of communicative-referential signals. The theory of natural pedagogy (Csibra & Gergely, 2009) proposes that ostensive signals automatically generate an expectation of generic content - an assumption that the communicator attempts to transmit generalizable knowledge rather than communicating some episodic information about the here-and-now. This expectation can be implemented in a default bias towards genericity: Unless the context or other cues specify otherwise, possible generic interpretations of the communication are preferred to non-generic ones. When the communication is about an object, such a bias would suggest that the message refers to a property of the object that is (1) not restricted to that particular object but generalizes to an object kind, and/or (2) not restricted to this particular occasion but generalizes across situations. Thus, this bias should direct the addressee's attention towards object properties that allow the utilization of the communicated knowledge by enabling the recognition of the object in a different situation, or objects of the same kind. Such properties are most likely the durable features of an object because transient features may change before re-identification and are unlikely to be kind-relevant. The distinction between durable and transient object properties is similar to the Marc Jeannerod's distinction between “extrinsic” and “intrinsic” object properties (Jeannerod, 1986). Extrinsic object properties are those that become relevant in the context of an object-directed action (such as location, distance and orientation with respect to the body), while intrinsic properties are constituents of the object's identity (such as colour, shape or texture). The theory of natural pedagogy predicts that, when the content of communication is ambiguous, addressees should be biased to pay attention to intrinsic features of communicatively referred objects and to ignore their extrinsic properties.

In the present study, we operationalized this prediction by contrasting attention to kind- and identity-relevant, intrinsic visual features, such as color and shape, with attention to object location (an extrinsic property) in communicative and non-communicative contexts. Since we wanted to avoid potential familiarity effects, such as automatic labelling of objects of known kinds, we used novel, unfamiliar objects. For these objects, both colour and shape were potentially kind-relevant, intrinsic properties, while location information is never informative regarding the object kind. If a genericity bias is operating, communicative referential cues should facilitate the encoding of the kind-relevant information and should impede the encoding of the location of the referred object. This prediction was confirmed in 9-month-old infants (Yoon, Johnson, & Csibra, 2008). In that study, infant participants watched actors who either reached towards, or communicatively pointed to, a novel object. Then the actor was occluded by a curtain and the object was occluded by a screen for 5 s before it was revealed again in one of three different ways: replacing the object by another one, modifying its location, or without any change. Infants' looking times indicated that in the non-communicative (reaching) context they detected the location change but not the object change, while in the communicative (pointing) context they reacted the opposite way, suggesting that communicative reference to the object shifted their attention to the predicted direction.

In theory, it would be possible that such a genericity bias operates only during childhood when learning about culturally determined object properties relies heavily on child-directed communication (Csibra & Gergely, 2006). It was also suggested that the effect was due to the distracting influence of communicative signals on infants (Spencer, Dineva, & Smith, 2009), or to limited cognitive capacities of the developing brain, which prevent infants from encoding all properties of observed objects (Yoon et al., 2008). Furthermore, when infants watched the object-directed actions, they were not given any instruction (Yoon et al., 2008). It is thus possible that the communicative and non-communicative cues exerted their effects on infants' change detection by altering the general cognitive relevance of intrinsic and extrinsic object properties (Sperber & Wilson, 1995), rather than biasing object perception and attention. Alternatively, if the genericity bias reflects a design feature of human communication, it should be present in people of all ages in appropriate tests, even when both types of object properties are explicitly marked as relevant for the task. We therefore developed a paradigm to test whether this phenomenon persists into adulthood and whether it is demonstrable under explicit instructions concerning the relevance of object properties. Affirmative answers to these questions would imply that the modulation of object attention by non-verbal communication is a functional feature of human cognition rather than being one of the transitory phenomena attributed to the immaturity of the infant brain (e.g., Pascalis, de Haan, & Nelson, 2002) to the hyperactivity of a single system (e.g., Jaswal, Croft, Setia & Cole, 2010), or being derivable from general principles of cognitive systems (Sperber & Wilson, 1995).

Our paradigm employs a change detection task (e.g., Luck & Vogel, 1997), in which a set of objects is presented to participants for inspection and, after a blank screen, the set reappears with a single object changed in location or identity. Thus, in this paradigm, location and identity are equally relevant object properties for solving the task. Crucially, in experimental trials, one of the objects is highlighted by communicative or non-communicative cues during the initial presentation, which enables us to measure the effect of these cues on change detection. We hypothesized that, compared to the non-communicative context, the communicative context would facilitate identity change detection and would impair location change detection of the highlighted object.

General Methods

Design

In each experimental trial in all experiments, an array of objects was shown to the participants in one of two social contexts (Communicative or Non-Communicative) defined by the nature of the action that was used to direct attention to one particular object in the array. In both contexts, either the location or the identity of an object changed during a subsequent short blank screen, and this object was either the one cued by the previous object-directed action, or another one. Thus, three orthogonal within-subject factors were employed (Context, Change, and Cue) to test their effects and interactions on change detection. Additionally, in the Experiment 1 a further context (Non-Social Highlighting) was included to control for the effects of non-social exogenous attention cues. In all experiments, we also added Baseline trials in order to test the sensitivity to the two kinds of changes (location vs. identity). In these trials, no exogenous attention-directing cues were present.

Apparatus

Stimuli were presented on a 15” touch screen connected to a Macintosh laptop computer, 50 to 70 cm from the participants’ eyes. The presentation was controlled, and the responses were recorded by scripts written in Matlab Psychtoolbox.

Stimuli

In each trial, participants were presented with a short silent movie clip followed by a blank screen and then a still test picture. Each movie depicted five meaningless objects arranged horizontally in a jagged row on a table covered by a blue and white chequered tablecloth (examples can be seen on Figure 1). These objects were randomly selected from a set of 8 objects assembled of red, white, blue, green and yellow LEGO bricks.

Figure 1.

Representative frames from selected video clips used for stimuli in Experiment 1. In each sequence, one object changed its identity or location by the test phase during the blank screen. In the three examples represented on the figure the cued object changed identity (Non-Communicative and Non-Social Contexts) or location (Communicative Context), but in the experiment changes also occurred on uncued objects. The cue in the Non-Social Context is a bright dot on the second object from the left.

In the experimental trials, an actress sat behind a table wearing a brown colored chemise, and performed an object-directed action (reaching or pointing) towards one of the objects. This action was preceded either by viewer-directed communication signals (direct gaze, waving and smiling), or by non-communicative behaviors (chin rubbing and looking through the objects, as if hesitating) depending on the context (Communicative vs. Non-Communicative). In the Non-Social Highlighting contexts, no human was present but one of the objects was highlighted by a quivering light dot, produced by a laser pointer. During Baseline trials, only the objects were presented without any further cues.

Still test pictures were created for each movie depicting the same arrangement of objects with a change: either the location of an object was modified by shifting it back or forth by 9 cm on the table¹ (location change), or an object was replaced by another one from the remaining objects that were not present during the initial phase. This object was different from the original one in both shape and color (identity change). The position of the change as well as the direction of location change varied randomly and equiprobably across trials. Location and identity changes occurred equally often, and the cued object changed in half of the trials while another one (selected randomly from the uncued set) changed in the other half. The actress, looking down at the table, was present only on the still test pictures paired with movies in either social context.

The duration of all movies was 5 s, and their last frame was frozen on the screen for 2 s. This was followed by a blank screen for 0.5 s, and then the corresponding still picture was presented until a response was produced by the participant.

Procedure

Participants were instructed to ignore the actress and other cues, and attempt to memorize the arrangement of the objects seen in the first phase of each trial. They were explicitly told that both location and identity changes would occur and were instructed to try to detect either. On the still picture, they had to indicate which object had changed during the black screen by touching its location. They did not have to specify what kind of change they detected on the indicated object, and had as much time to respond as they wanted. (We did not ask for speeded responses because the validity of reaction times would have been low when responding to changes at varying locations on a touch screen.) As soon as the participants touched the screen, their response was acknowledged by a small white square at the location of the response for 1 s, then the picture was removed and the next trial started after a 2 s delay.

Data Analysis

The closest object to the location of the participants' first touch was considered as the selected object in each trial. We calculated change detection performance by dividing the number of correct responses by the number of trials in each condition, and converted these figures to a percentage. The conditions were defined by all combinations of Context, Change, and Cue factors. These data were analyzed by repeated measure ANOVAs with the factors above and including the difference between the performance in location and object change trials during the Baseline condition as a covariate. If there are individual differences in sensitivity to location vs. identity change, this method takes that into account and removes such effects from those of other factors.

Beyond an omnibus ANOVA, our analysis focused on two predicted interaction effects derived from our hypotheses. Because communicative reference was expected to shift attention away from location and towards identity-relevant visual features, we predicted an interaction between Context and Change in the sensitivity to the change of cued objects in the two social contexts (when communicative cues are contrasted with non-communicative cues), and an interaction between Change and Cue within the Communicative context (when the effect of a communicative cue is contrasted with its absence). Either or both of these interactions might also produce a three-way interaction among all within-subject factors. We also directly checked by post-hoc LSD tests whether the interactions could be explained by separate simple main effects.

Experiment 1

Methods

Participants

Twenty-four volunteers (14 female; mean age = 23.5 years) participated in Experiment 1 and received five pounds for their time. All of them had normal or corrected to normal visual acuity.

Stimuli

In the Communicative Context, the actress made a pointing gesture towards one of the objects, and finished her action by looking into the camera towards the viewer again. In the Non-Communicative Context, she reached towards the referred object, but her gesture stopped short of touching it when the clip froze. In the Non-Social Highlighting Context, a quivering red dot was projected on one of the objects for 4 s. The other presentation parameters were the same as described in the General Methods section.

Procedure

Eighteen trials were presented in the Baseline context, and 36 trials in each of the Communicative, Non-Communicative, and Non-Social Highlighting contexts. Half of the trials in each context included location change and the other half presented an identity change. Orthogonal to the type of change, the change occurred on the cued object in half of the trials (except in the Baseline condition, in which no object was cued). The 126 trials were presented in random order (different for all participants) in three blocks of 42, allowing the participants to have a break between the blocks. The whole experiment lasted approximately twenty minutes (depending on the speed of the responses).

Results

Change detection performance in all conditions is reported in Table 1 and depicted on Figure 2. We analyzed these data in a 3×2×2 analysis of variance (ANOVA) with Context (Communicative vs. Non-Communicative vs. Non-Social Highlighting), Change (Location vs. Identity), and Cue (Cued vs. Uncued), as within-subject factors, and the difference in performance between the two change types during the Baseline trials was included as a covariate. This analysis revealed a main effect of Cue [F(1, 23) = 5.106, p = .034, η² = .188] due to generally better change detection in cued objects, indicating the social and non-social referential signals did successfully work as attention-guiding cues. The covariate of baseline performance showed a significant interaction only with the Change factor, indicating that some of the variance on this factor was attributable to individual differences in sensitivity to the two types of change [F(1, 23) = 30.386, p < .001, η² = .580]. We also found interactions between Context and Change [F(2, 46) = 4.094, p = .023, η² = .157], and Change and Cue [F(1, 23) = 5.600, p = .027, η² = .203] factors, indicating that sensitivity to the two kinds of change differed across contexts and objects.

Table 1.

Means (and Standard Deviations) of Correct Change Detection (%)

	Identity Change		Location Change
	Cued	Uncued	Cued	Uncued
Experiment 1
Non-Communicative	59.3 (28.5)	52.8 (17.7)	72.2 (23.6)	64.8 (27.0
Communicative	63.0 (22.4)	53.7 (23.3)	56.9 (25.2)	61.6 (20.7)
Non-Social	63.4 (27.7)	47.2 (24.6)	63.0 (22.6)	59.3 (15.6)
Baseline	51.9 (23.8)		63.4 (25.2)
Experiment 2
Non-Communicative	68.4 (19.7)	57.3 (19.1)	66.3 (20.9)	47.2 (23.1)
Communicative	67.0 (18.3)	52.8 (18.8)	51.0 (26.7)	51.0 (21.9)
Baseline	59.0 (21.9)		45.5 (22.0)
Experiment 3
Non-Communicative	60.8 (19.4)	52.1 (18.1)	52.1 (22.7)	37.5 (18.9)
Communicative	64.9 (24.7)	45.5 (22.8)	38.9 (25.0)	49.0 (22.7)
Baseline	49.3 (19.6)		36.1 (17.7)

Figure 2.

Change detection performance in Experiment 1. Error bars represent standard error of means.

To test one of the predicted interactions, a 2×2 (Change × Context) within-subject ANOVA was conducted on performance on the cued objects in the two social contexts. This revealed the predicted interaction [F(1, 23) = 7.087, p = .014, η² = .244] due to relatively better detection of identity change in the Communicative Context and relatively better detection of location change in the Non-Communicative Context for cued objects. However, post-hoc LSD tests indicated that the difference between the detection of the two changes was only approaching significance in the Non-Communicative context (p = .053) but not in the Communicative Context (p = .291). The other prediction was also confirmed by the interaction between Change and Cue factors within the Communicative Context [F(1, 23) = 4.675, p = .041, η² = .169]. This interaction is explained by a facilitatory effect of the communicative referential cue for identity change detection and a lack of effect for location change detection. However, measuring separately, neither of these effects was significant in itself by post-hoc LSD tests (p = .146 and .288, respectively).

A further 3-way ANOVA that included only the Communicative and Non-Social Highlighting conditions showed a significant interaction between Context and Cue [F(1, 23) = 6.163, p = .021, η² = .219] and Change and Cue [F(1, 23) = 10.310, p = .004, η² = .319]. These interactions demonstrate that communicative reference had a different effect from non-communicative attention cueing.

Discussion

The primary question of this study was whether different attention cues facilitate the detection of different types of information of cued and uncued objects. This was confirmed by the Context × Change and Change × Cue interactions in the omnibus ANOVA. More specifically, when we compared change detection performance in the two social contexts, we found that location change detection was easier than identity change detection on cued objects in the Non-Communicative context, while we found the opposite pattern in the Communicative Context. Thus, the communicative cue had its effect not on the amount of visual attention (the average performance was similar in the two contexts) but on what participants paid attention to: Compared to non-communicative reaching, communicative-referential pointing shifted participants' attention away from location and towards the identity-relevant visual features of the indicated object. This result is in line with our prediction, according to which communication facilitates referent encoding in terms of permanent properties at the expense of ignoring accidental object features, such as location. Note, however, that while communicative cues, compared to non-communicative cues, shifted attention away from object location and toward object identity as shown by the above interaction, the simple main effects within conditions were not significant. This suggests a slight modulatory effect, rather than a dramatic change, on object attention by communicative cues.

Note also that the effect of the non-communicative reaching action was not neutral either. This cue facilitated the detection of object change and did so for both the cued and the uncued objects (Figure 2). A plausible explanation of this effect is that the goal of a reaching event is grasping the target object, and target selection for this action requires location encoding. It has been shown that the presence of non-target objects influences the kinematic parameters of grasping actions (Tipper, Howard, & Jackson, 1997), probably because their location is also taken into account. If observed actions are, at least partially, encoded in motor activation, a motor integration process could account for the generalized facilitation of location change detection in the Non-Communicative Context (Sartori, Xompero, Bucchioni, & Castiello, 2011).

While the facilitatory effect of the reaching action seemed to spread to all objects in the Non-Communicative Context, this was not the case in the Communicative Context. The strong main effect of Cue in the omnibus ANOVA showed that the attention-directing cues generally worked as expected: they facilitated the detection of change in the cued objects compared to the uncued objects. However, there was an exception to this general rule: When attention was directed to the object by communicative pointing, sensitivity to location change of this object was not better that that of other, uncued objects (see Figure 2). Note that the absence of cueing effect was restricted to the communicative cue and to location change, and is evidenced by the predicted significant Change × Cue interaction in the Communicative context. This effect is, in fact, a paradoxical case of spatial attention, as it suggests an inhibitory influence on encoding a certain kind of information (here, location) about the attended object. This effect indicates that communicative-referential cues, such as pointing, do more than directing the attention of the viewer to an object, and may generate expectations about the types of relevant information to be encoded about the object.

Importantly, we also found that communicative reference differs not just from other social cues, but also from non-social exogenous attention directing mechanisms. While a quivering red dot was as effective in attracting visual attention as the social signals, it resulted in equal sensitivity to the two kinds of changes and a different pattern of performance from the other two contexts. We emphasize that our findings represent involuntary shifts and tuning of attention in all conditions. Our participants were told to ignore the actress and the quivering dot but did not seem to be able to do so. In fact, some of them mentioned after the experiment that they had attempted to resist the distracting impact of the contextual elements, like the actions of the actress, and focus on the object array.

Our primary interest here is the special effect that communicative reference exerts on object perception. Experiment 1 demonstrated such an effect, but did not specify which elements of the communicative-referential action sequence contributed to its effect. Experiments 2 and 3 addressed this question.

Experiment 2

The attention-guiding element of referential cueing in the communicative context of Experiment 1 was the pointing gesture that the actress performed toward a specific object. Index-finger pointing is a primarily communicative act, though it is sometimes also used for aiding memory or reasoning processes in solitary contexts (Kita, 2003). In our movies, further ostensive signals (Csibra, 2010), such as direct eye gaze and waving towards the viewer, clarified that this gesture was meant to be performed for the participant, i.e., that it was a communicative act. In Experiment 2, we aimed at testing whether the pointing gesture by itself, without any additional communication signals would produce the same effect on change detection as it did with them. In other words, Experiment 2 addressed the question whether referential gestures (such as pointing) can elicit the effect, or ostensive signals (such as eye contact) are also needed in order to shift the attention away from the location and towards the permanent features of cued objects. Thus, in this experiment, we replaced the reaching action of the Non-Communicative context by a pointing gesture in order to test whether it had the same effect without the support of the accompanying ostensive signals.

Method

Participants

Twenty-four volunteers (17 female; mean age = 21.6 years) participated in Experiment 3 and received five pounds for their time. All of them had normal or corrected to normal visual acuity.

Stimuli

The Communicative Context presented the same movies as in Experiment 1. However, in the Non-Communicative Context, the reaching gesture was replaced by pointing. Thus, in this context, the actress rubbed her chin while looking through the objects, and then made a pointing gesture (with extended index finger) towards one of the objects without ever looking to the viewer or using any other ostensive signals.

Procedure

This experiment did not include the Non-Social Highlighting Context. Because of this, the number of trials in the remaining conditions was increased from 9 to 12. There were altogether 120 trials.

Results

Table 1 and Figure 3 show the average proportion of correct change detection in each condition. A 2×2×2 ANOVA was conducted with the Context, Change and Cue factors and Baseline performance difference as a covariate. This analysis yielded a main effect of Context [F(1, 23) = 7.097, p = .014, η² = .244] due to better performance following Non-Communicative pointing than after Communicative pointing, and a main effect of Cue [F(1, 23) = 13.792, p = .001, η² = .386], indicating superior detection of changes on pointed compared to other objects. The interaction between Change and Baseline covariate was also significant, suggesting consistent biases towards certain types of changes by participants [F(1, 23) = 54.480, p < .001, η² = .712]. Finally, a significant three-way interaction between Context, Change and Cue [F(1, 23) = 5.380, p = .030, η² = .196] suggested that the detection of the two types of changes was modulated differentially by cueing in the two contexts.

Figure 3.

Change detection performance in Experiment 2. Error bars represent standard error of means.

We also confirmed the presence of the two predicted interactions. Considering the performance on the cued objects, we found a Context by Change interaction [F(1, 23) = 5.715, p = .026, η² = .206] because communicative pointing reduced the detection performance only for location changes. In fact, while post-hoc LSD tests indicated no difference in change detection in the Non-Communicative context (p = .674), the location change of the cued object was less likely detected than its identity change in the Communicative context (p = .001). Within the Communicative context, the interaction between Change and Cue was also significant [F(1, 23) = 4.321, p = .050, η² = .164] because cueing by communicative pointing did not increase location change detection while it helped identity change detection. This explanation is also supported by post-hoc LSD comparisons: cueing increased identity change detection (p = .004) but did not have an effect on location change detection (p = 1.000) in the Communicative context.

Discussion

We found that when the pointing gesture was not preceded by other communicative cues, this referential gesture did not have the same effect on object perception as the fully communicative pointing act. Changing reaching to non-communicative pointing might have had an effect though, as the comparison of Figures 2 and 3 suggests. While reaching positively facilitated the detection of location (compared to identity) changes of the cued object in Experiment 1, the non-communicative pointing gesture did not have such an effect in Experiment 2. However, non-communicative pointing did not produce the same pattern as communicative pointing. While in the non-communicative context, this gesture resulted in the same performance for change detection of identity and location, the same gesture accompanied by ostensive signals generated more correct detections of identity change than of location change. Thus, the predicted interaction between Context and Change for cued objects was confirmed.

The pattern of results in the Communicative context essentially replicated that of Experiment 1. Although location detection rate was not better for uncued than for cued objects, it was not worse either. Since in all the other conditions cueing increased change detection performance by at least 15 %, the absence of cueing effect on location change of communicatively pointed objects is peculiar and suggests the suppression of encoding of the current location of the referent.

Because the participants observed the same gesture (hand shape) in the two contexts, the distinct patterns of change detection performance must have been due to the further communicative signals present in one but not in the other context. These ostensive signals, which included direct gaze before and after the pointing gesture, and smiling and waving at the beginning of the trial, let the viewer know that the pointing action was a communicative act performed for her benefit. It is also possible though that the kinematics of the pointing actions differed between the two conditions. It is known that communicative intention can modulate how an action is performed (Sartori et al., 2009). Thus, it is plausible that the kinematics of the gesture contributed to its interpretation. Nevertheless, whether the lack of the accompanying ostensive signals, or the subtle kinematic differences are responsible for the differential effects of pointing on attention in the Non-Communicative Context, it is the communicative intention of the actor that influenced the participants' performance.

However, if the effects of the Communicative Context were due to the presence of ostensive signals, then they might also have distracted the participants from observing the object array and contributed to the difference in change detection between the contexts. In particular, if the actress' direct gaze towards the viewer after finishing the pointing action caught the attention of the participants in the Communicative context, it might have impeded their ability to keep the exact locations of the objects in short-term visual memory. Since the Non-Communicative context did not include such a distractive event at the end of the trials, this could explain the difference across contexts. This alternative account was tested in Experiment 3.

Experiment 3

In order to test whether attentional distraction explains poor location change detection on communicatively cued objects, we repeated Experiment 2 with the final direct gaze edited out from the stimuli in the Communicative context. If change detection shows the same pattern as in Experiment 1 and 2, then the assumption that weak encoding of the location of objects in the Communicative context was only due to the distracting effect of the final direct gaze can be rejected.

Method

Participants

Twenty-four people participated in Experiment 3 (12 female; mean age = 23.5 years) and received a five pounds for their time. All of them had normal or corrected to normal visual acuity.

Stimuli

We used the same stimuli as in Experiment 2, but we cut the final direct gaze, which followed the pointing gesture, from the movies in the Communicative context (about the last 0.5 s). To equalize the length of the movies, we slowed down the presentation of these trials to make their duration the same (5 s) as the ones in the Non-Communicative context.

Results

Table 1 and Figure 4 shows the proportion of change detection in each condition. To analyze the results, a 2×2×2 ANOVA was conducted with Context, Change and Cue variables and the differences of Baseline performance were included as a covariate. Beyond the expected interaction between Change and the Baseline covariate [F(1, 23) = 12.785, p = .002, η² = .368], this analysis revealed a significant main effect of the Cue [F(1, 23) = 5.859, p = . 025, η² = .209], an interaction between Change and Cue [F(1, 23) = 5.824, p = .025, η² = . 209], and a three-way interaction [F(1, 23) = 9.551, p = .005, η² = .303]. The latter effect suggests that cueing modulated change detection differently in the two contexts.

Figure 4.

Change detection performance in Experiment 3. Error bars represent standard error of means.

Further analyses tested the predicted interactions. Considering only the changes of the cued objects, we found a significant interaction between Context and Change [F(1, 23) = 5.782, p = .025, η² = .208], which was due to a non-significant difference between the detection of the two types of changes in the Non-Communicative context (LSD p = .180), and a significant one in the Communicative context (LSD p < .001). Confirming the other prediction, we found a significant interaction between Change and Cue [F(1, 23) = 16.307, p = .001, η² = .426] within the Communicative context, arising both from a significant cueing effect on identity change (LSD p = .006) and a marginally significant reverse cueing effect on location change (LSD p = .051). These interactions confirm that communicative pointing modulates change detection in comparison to both non-communicative pointing and non-pointed objects, and indicate that this effect does not depend on the eye contact after the performance of the gesture.

Discussion

The results of Experiment 3 replicated those of Experiments 1 and 2, showing even stronger patterns of the predicted effects. Thus, we conclude that the poor location change detection of communicatively cued objects was not the result of attentional distraction by direct gaze in the earlier experiments. The findings of Experiment 2 and 3 suggest that neither the gesture performed by the communicator, nor how this action terminates, but the presence of the initial ostensive signals (and possibly some additional subtle kinematic cues) determines the effect of communication on change detection. These ostensive signals set the scene for communication and are supposed to generate the expectation for further, contentful information from the same source (Csibra, 2010). One interpretation of our result is that this expectation also triggers the assumption that certain kinds information are more relevant for encoding than others and tune the addressee's object-directed attention accordingly.

Comparison Across Experiments

While the general pattern of results was the same across experiments, participants' change detection performance and the magnitude of the measured effects varied considerably. We compared the results across experiments using the same strategy of analyses we applied to each of them. A 2×2×2×3 ANOVA with Context, Change, and Cue as within-subject factors, Experiment as a between-subject factor, and Baseline difference as a covariate revealed main effects of Context [F(1, 69) = 8.201, p = .006, η² = .108] and Cue [F(1, 69) = 18.589, p < .001, η² = .215]. The Context effect was due to generally better performance in the Non-Communicative than in the Communicative Context. Furthermore, both the 2-way interactions of Context and Cue, and Change and Cue were significant [F(1, 69) = 7.951, p = .006, η² = .105; F(1, 69) = 7.003, p = .011, η² = .093], whereas the interaction of Context and Change approached significance [F(1, 69) = 3.713, p = .058, η² = .052]. Finally, the 3-way interaction of Context, Change and Cue was highly significant [F(1, 69) = 18.822, p < . 001, η² = .217]. The Experiment factor did not yield significant interactions with other factors.

We then collapsed the data across the three experiments to test the predicted interactions in the separate analyses. Within Cued objects, change detection differed between the two contexts [F(1,71) = 22.851, p < .001], because location and identity change detection performance was similar in the Non-Communicative context (LSD p = .826), but location change detection was worse than identity change detection in the Communicative context (LSD p < .001). Within the Communicative context, cueing effects differed across change types [F(1,71) = 25.499, p < .001], because cueing facilitated identity change detection (LSD p < .001), while it marginally significantly impeded on location detection (LSD p = .069).

General Discussion

In three experiments, we found that different kinds of cues that direct attention to a particular object in an array modulated the detection of a change on that object in different ways. In particular, communicative reference, when compared to human-delivered, superficially similar, but non-communicative cues, impeded the detection of location change of the cued object. Such an effect cannot be accounted for by a general modulation of the amount of attention paid to the cued object in the two situations. Although in one out of three experiments we found a Context main effect due to lower performance in the communicative than in the non-communicative context, this does not explain the significant interactions. Thus, we conclude that the communicative nature of the referential cue had an effect on the quality, rather than just the quantity, of attention paid to the cued object.

The same conclusion can also be drawn from comparing the encoding of the visual features and locations of communicatively cued to that of uncued objects. While communicative pointing facilitated the encoding of the identity of cued objects, it did not have an effect (Experiment 2), or had the opposite effect (Experiments 1 and 3) on location change detection, as evidenced by the Change by Cue interactions in the Communicative Context of all, and across, experiments. Our study did not address the question of how object location is encoded in the type of situation we presented to the participants. They might have registered object location in screen coordinates (e.g., relative to the table) or relative to the surrounding objects. Either way, the effect of ostensive pointing on the encoding of object location represents an anomalous case of spatial attention: focusing on the cued object seems to have suppressed the encoding of its location in the array. This effect cannot be explained by paying less attention to the cued object than to the uncued ones because the same cue helped the encoding of object features. Communicative reference thus modulated not only which object received preferential processing but also which properties of it were selected to be encoded in visual memory.

Our paradigm employed a visual working memory task with a recognition test, which cannot determine which phase of the memory process was modulated by the communicative signals. It can be that ostensive signals exert their effect on the encoding of object information, determining which type of information enters into the working memory. It is also possible that both the location and the identity of the objects were initially encoded, and ostensive signals influenced later their maintenance selectively, by facilitating the retainment of identity-relevant information and the discardment of location information. However, since the ostensive signals were not present during the storage phase (in fact, they were only presented before the referential cue in Experiment 3), we find it more likely that they exerted their effects on the encoding than on the maintenance of object information.

These effects are consistent with, and were predicted by, the proposal that the processing of incomplete communicative acts, such as the ones with which we presented our participants, is subject to a bias towards intrinsic, and away from extrinsic, properties of referents. One can re-identify an object as the same object as a previously experienced one either by spatio-temporal criteria (it occupies the same location, it continues the previous pathway, etc.), or by visual features (it looks the same). In short temporal spans, these two methods are equally applicable, and spatio-temporal identification may even be preferred, because it requires only the maintenance of an object index (Pylyshyn et al., 1994) or an object file (Kahneman & Treisman, 1984) without storing the detailed visual attributes of the object. However, for identifying an object after a longer time delay, its visual features are more useful because they are less likely to change than the object’s location. In fact, if the task is to re-identify a movable object later, information about current location is irrelevant and it is better not to be stored in memory.

It is even more important to suppress location information if the communicative cue is interpreted as referring to the object kind rather than to the particular object, because object kinds are abstract concepts that cannot be individuated by spatial location. Indeed, interpreting a communicative-referential action, such as ostensive pointing, as picking out the object kind, rather than a particular object, as its referent requires the addressee to ignore object properties, such as location, that vary across members of the kind. In everyday communication, unlike in our experiments, referential signals are accompanied by further communicative acts that specify some predicate. These can be words (such as the name of the object), facial expressions (to provide affective evaluation), or actions performed on the objects (for example, to reveal a hidden and/or functional property). The attentional bias that we demonstrated in these studies will facilitate the binding of these predicate not to the particular object present in the situation but to the object kind represented by it. Thus, such a bias could support learning of object labels, object valence, and object functions – i.e., properties that do indeed belong to the whole category of objects exemplified by the referent. This is how, perhaps paradoxically, ignoring location information by communicative signals could facilitate social learning from communicators.

In sum, our findings are consistent with the hypothesis that referential signals (in our case, pointing), performed in a communicative manner induce a genericity bias in addressees, and, as a result, they selectively ignore extrinsic object properties, like location, which are not relevant for generalization across occasions of encountering the same object or to members of a kind. However, since we employed only one type of non-verbal referential action, pointing, our conclusion may not necessarily extend to other signals. Further experiments are needed to test whether other communicative-referential actions, such as ostensive gazing or showing up objects, would induce the same effects.

We predicted that communicative reference would have both a positive (increasing attention to object identity) and a negative (decreasing attention to object location) effect on the encoding of the features of the cued object. We found that the negative effect of communication was stronger than the positive one when it was compared to the effect of non-communicative cues. We speculate that the reason for this asymmetry is that we employed novel objects, which did not belong to any object kind known to the participants. In the absence of such background knowledge, they could not assess which visual features of the objects were kind-relevant and worthy of attention. Alternatively, since the non-communicative cues, like reaching and non-ostensive pointing, facilitated the encoding of both extrinsic and intrinsic properties of the highlighted objects, they may not have been the best comparison stimuli to assess the relative benefit of the two cues separately for the two kinds of changes. Nevertheless, as we predicted, the pattern of change detection was different in the two contexts. Note also that, compared to uncued objects (rather than compared to non-communicative cues), communicative cues elicited both facilitatory and inhibitory effects on encoding object identity and object location, respectively, though in different magnitude.

It is also noteworthy that these effects of communicative signals emerged automatically. Participants were told to ignore the actress, but our results suggest that they were unable to do so. One side of this automatic attention modulation, namely the better performance on cued than on uncued objects, can be explained by known phenomena. Both the social and non-social cues may have acted as exogenous attention cues, which are known to elicit spatial attention at the location where they appear (Posner, 1980). In addition, these cues carried valid information, because the cued objects were more likely to change in any given trial than any other object on the scene (0.5 vs. 0.125), which could also have contributed to the overall cueing effect. However, the type of change (identity vs. location) was perfectly counterbalanced with the type of cue (communicative vs. non-communicative). Still, communicative cues managed to influence the encoding of these object properties and the subsequent detection of their change, and this automatic modulation of attention did not provide any advantage to the participants in performing their task. Many studies that demonstrate the influence of social stimuli on attention are discussed as examples of 'joint attention' (e.g., Bristow et al., 2007; Fischer & Szymkowiak, 2004; Frieschen et al., 2007). This term comes from developmental psychology, where it refers to episodes of adult-child interaction, which focus both parties' attention to a particular object. However, it is clear that such a construct alone would not be sufficient to explain the difference that we found between Communicative and Non-Communicative contexts. In both of these situations, the actor and the observer allocated attention to the very same object, but which object properties were preferentially encoded by the observer depended on whether he or she was addressed by the actor. Infants' memory of objects has also been shown to be influenced by the communicative signals of adult interactors (Striano, Chen, Cleveland, & Bradshow, 2006; Cleveland & Striano, 2007). Note, however, that, just like in our study, the crucial factor modulating infants' encoding of visual features of objects was not whether the adult attended the object but whether she was communicating to the infant. For this reason, it would be more appropriate to characterize these situations as establishing 'joint reference' rather than 'joint attention' (cf. Baldwin, 1991).

How did communication signals exert their effect on object perception and attention? Equalizing gesture type (Experiment 2) and removing direct gaze from the stimuli after the referential gesture (Experiment 3) did not change the pattern of results, suggesting that it was the presence of the initial ostensive signals (eye contact and waving to the viewer) or subtle kinematic cues that biased the utilization of the referential cue (i.e., pointing). Our findings do not allow us to pinpoint the locus within the visual system where this bias occurred. One possibility is that the ostensive signals had a differential tuning effect on the dorsal and ventral visual streams (Mishkin, Ungerleider, & Macko, 1983). Since the dorsal stream primarily processes extrinsic object attributes (including location) and the ventral stream deals with intrinsic visual features that make object recognition possible, inhibiting the former and/or facilitating the latter would produce effects similar to our results. Findings in infants (Mareschal & Johnson, 2003) and adults (Shmuelof & Zohary, 2005) have shown that the two visual streams could be independently modulated by contextual factors, and ostensive signals may operate the same way. Alternatively, the communicative context might have had its effect on a higher level of processing. If there is a bias to interpret referential pointing as indicating the object kind rather than the individual, this may result in selective retainment of object attributes. These explanations are not mutually exclusive, and further behavioral and neuroimaging studies could clarify the relative contribution of lower and higher level processes to the effect.

Another question that awaits further research is whether the effect of ostensive signals on attention modulation is restricted to the addressee of these signals or it elicits the same effect in third parties as well. Human infants are especially sensitive to communication addressed to them (Csibra, 2010), and it is only later that they become sensitive to communicative signals in observed interactions (Beier & Spelke, 2012). Adults, however, pay special attention to communication between others. For example, while 6-month-olds only follow the head turns after they have been addressed by ostensive signals (Senju & Csibra, 2008), adults also follow gaze after observing eye contact among third parties (Böckler, Knoblich, & Sebanz, 2012). It is thus possible that the presence of communicative signals, and not necessarily communicative signals addressed to the viewer, is sufficient to elicit the modulatory effect on object perception.

In sum, we have demonstrated that, just like in human infants (Yoon et al., 2008), communicative signals modulate the attention to, and encoding of, properties of the referent object in adults as well. In infants, a similar finding was partly attributed to limited representational and/or memory capacities, which could have explained why they failed to detect location changes in communicative contexts. Our results suggest that modulatory effects of communication to object attention are not due to limited resources but may play a role in comprehension, which implies that this attentional bias is an inherent part of human communication rather than specific to certain age groups, and may function to facilitate the acquisition of generic knowledge from others (Prasada, 2000). This conclusion is compatible with the view that the evolutionary origin and function of human communication cannot be exclusively derived from, or restricted to, the needs to support cooperative collaboration (Tomasello, 2008), to track and maintain social coalitions (Dunbar, 1998), or to manipulate the mental states of others for one's own interests (Sperber, 2001). Thus, the potential for inter- and intra-generational transfer of generic knowledge may not be a by-product but one of the functions of the unique system of ostensive communication in humans (Csibra & Gergely, 2011).