Skip to main content
PMC home page
iScience logoLink to iScience
. 2020 Apr 30;23(5):101112. doi: 10.1016/j.isci.2020.101112

The Active Sensing of Control Difference

Wen Wen 1,5,6,∗, Hiroshi Shibata 2,5, Ryu Ohata 2,3, Atsushi Yamashita 1, Hajime Asama 1,4, Hiroshi Imamizu 2,3,4
PMCID: PMC7225729  PMID: 32408176

Summary

In everyday life, people control objects in the world around them to varying degrees. The processes people actively use to establish their control, while interacting with an environment containing large ambiguity, remain unknown. This study examines how people explore their control over the environment and how they detect small differences in control among objects. In the experimental task, participants moved three dots on a screen and identified one dot over which the level of control is different from that of the other two. The results support a two-step behavior mechanism underlying the sensing of control difference: People first explore their overall control in the environment, and then the results of the initial exploration are used to selectively tune the direction (i.e., either more or less) of the detected control difference, ensuring efficient and rapid detection of the type of control difference that is potentially important for further action selections.

Subject Areas: Neuroscience, Behavioral Neuroscience, Cognitive Neuroscience

Graphical Abstract

graphic file with name fx1.jpg

Open in a new tab

Highlights

  • •

    This study examines how people explore their control over multiple objects

  • •

    The results support a two-step behavior mechanism of active sensing of control

  • •

    People first sense their overall control over the environment at an early stage

  • •

    The initial exploration tunes the sensitivity of detecting control difference

Neuroscience; Behavioral Neuroscience; Cognitive Neuroscience

Introduction

When people move around in an environment and interact with external objects, some objects may respond well to people's action, whereas others may not. The relation between one's voluntary actions and actual feedback causes the subjective feeling of controlling external events. This subjective feeling is called the “sense of control,” which is also called the “sense of agency” in the literature. The sense of control helps people to efficiently allocate attention to the objects they can better control (Kumar et al., 2015, Salomon et al., 2013, Wen and Haggard, 2018) and to select among alternative actions to optimize outcomes (Eitam et al., 2013, Karsh et al., 2016).

In daily life, aside from use of well-learned tools, people generally need to find out the extent of their potential control over novel objects. How do they do this? People might confirm their control by comparing the sensory predictions based on their motor commands with the actual sensory feedbacks, as the comparator model suggests (Blakemore et al., 2002, Frith et al., 2000). They might also sense their control via retrospective processes, such as inference (Synofzik et al., 2013, Wegner, 2003) and regularity detection (Wen and Haggard, 2020). Nevertheless, most of the prior studies assumed that the sense of control was a type of feeling or judgment based on sensorimotor signals sensed from both the body and the environment.

The processes underlying the sense of control over a single object have been extensively studied over the past several decades (Haggard, 2017, Haggard, 2019, Haggard and Chambon, 2012, Moore, 2016, Moore and Fletcher, 2012). However, in daily life, people usually simultaneously interact with multiple objects, some of which they may be able to control. Sensing control among multiple objects, particularly the differences among them, is important for human decision making and action selection. For example, in the case of driving a vehicle, the driver has effective control over both the wheels (controlling the direction) and the engine (controlling the speed). Usually, the driver does not consciously think about her or his control over the multiple parts of the car, since maintaining good control is highly probabilistic knowledge from daily use. In this case, any unpredicted sensory feedback due to a sudden loss of control (e.g., a malfunctioning part) is usually salient. As another example, when one presses a button on a remote control device when the associated machine is unknown, our perceptual system is tuned to detecting expected (based on one's action) sensory feedback. Empirical studies show that both unpredicted action consequences due to lack of control (Blakemore et al., 1998) and predicted action consequences (Kumar et al., 2015, Wen and Haggard, 2018) could capture bottom-up attention. How does the cognitive system know when to attend to control and when to attend to the lack of control when top-down attention is not involved? The mechanisms underlying such efficient sensing of control among multiple objects remain poorly understood. The simplest model for sensing control difference would be directly based on the same model of the sense of control for a single object: People can sense their levels of control over different objects one by one and then compare the obtained results, subsequently giving attention to the “odd” one. However, such sequential sampling of control is not cognitively efficient. The research on human information processing proposed a two-process theory, suggesting that detection, search, and attention allocation involve both parallel automatic detection processes and sequential controlled search processes (Schneider and Shiffrin, 1977). Automatic detection is linked to bottom-up attentional allocation, resulting in pop-out phenomena in extreme cases (Salomon et al., 2013). On the other hand, controlled search is linked to top-down attentional allocation. Previous studies showed that humans can rapidly allocate their attention depending on control in a bottom-up fashion (Salomon et al., 2013, Wen and Haggard, 2018). Such phenomena reflect automatic detection rather than controlled search and cannot be explained by sequential sampling of control.

Here, we suggest that the control context plays a critical role in the cognitive system allocating cognitive resources for control sensing. Control context refers to the level of overall control over multiple interacting objects. It determines the current state of control. For the above example of vehicle control, the control context is high because people usually have a high level of control over many of the vehicle's components (e.g., wheels, engine, and brakes) in daily use. In such cases, people do not need to pay so much attention to confirm their control. On the other hand, control context is low in the condition of interacting with many uncontrollable objects. We suggest that control context tunes the perceptual sensitivity in detecting different directions of control differences. Importantly, control context is sensed via the initial exploration behavior before people detect control difference in the environment. Once the ambiguity of control over the majority of objects is solved, orientating cognitive resources in a selected direction of difference is more efficient for humans than sequential sensing of control. Moreover, as described in the examples of a vehicle and a remote control, such selective attention is also important for a human's adaptive behavior. This two-step control sensing hypothesis is summarized in Figure 1. First, the two-step hypothesis assumes an initial exploration of control context followed by a later usage of control difference. In the first step of initial exploration of control, sensorimotor information can be processed in parallel to quickly solve the ambiguity of control context (Friston et al., 2016). In the second step, sensory feedback is then associated with each object to compute the sense of control and allocate cognitive resources in only one direction of control difference. The sensing of control is not necessarily a matter of cognitive judgment. It can also be implicitly conducted, automatically influencing human behavior and perceptions. The hypothesis of two-step control sensing suggests that the initial sensing is critical. For example, sensing a low level of control context (e.g., level 3, assuming there are 10 levels from 1 to 10 with 5 being the medium level) enhances the sensitivity of detecting a higher level of control than 3 (e.g., level 4) but decreases the sensitivity of detecting a lower level of control than 3 (e.g., level 2). On the other hand, if the initial sensed control is level 2 (i.e., a low level), both levels 3 and 4 should be easy to detect. Alternatively, from the viewpoint of sequential sampling of control, the magnitude of difference between level 2 and level 3 should determine the sensing sensitivity, regardless of the initial level of control context.

Figure 1.

Figure 1

Open in a new tab

Hypothesis of Two-Step Control Sensing

In order to examine the two-step control sensing hypothesis, we designed a novel control difference detection task (Figure 2). In this task, participants freely explored their control over three visual objects by moving their finger on a touchpad. Two objects among three were under the same level of control (30%, 50%, 70%, or 90%). The remaining one object was under at a level 10%, 20%, or 30% more or less than the other two objects. We focused on whether the level of control over the majority (i.e., two of three stimuli, which is considered the control context) affects the detection accuracy. Participants freely explored their control over the three objects and detected the target over which the control was slightly different from the others. The velocity, onset, and offset of the dots' movements were normalized to match the participants' finger movements, while the directions of the dots were mixed based on the participants' movements and randomly selected sections from 10,000 pre-recorded motions (Wen et al., 2018) for each dot. The three dots moved at the same speed but in very different directions from each other because a different section of pre-recorded motions was applied to each dot. Consequently, the only cue that could be used to identify the target dot was the sensorimotor correlation between one's real-time movements and the moving direction of the dot (i.e., the levels of control). The level of control over the two distractors was the same but slightly differed from the target, and it was defined as the control context, since the distractors were the majority of the stimuli. In the control difference detection task, theoretically, one can solve it by sampling the level of control over the three objects and then comparing the sampling results among them (i.e., the alternative hypothesis of sequential sampling of control). In such a case, the magnitude of difference in control between the target and control context should determine the detection accuracy, regardless of whether the target was under more or less control than the control context. In other words, the hypothesis of sequential sampling of control predicts a main effect of the magnitude of difference on detection accuracy, and this effect should not interact with control context. In contrast, our two-step control sensing hypothesis predicts that the difficulty of detecting the target does not only depend on the extent of actual difference in control between the target and the distractors but also heavily relies on the control context illustrated in Figure 1. In short, the proposed theory predicts an interaction between the control context and the effect of the magnitude of difference on detection accuracy. After identification of the target dot, participants also made an explicit judgment on whether they felt that the selected dot was under more or less control than the other two dots. Furthermore, we examined the two-step control sensing hypothesis by kinematic analyses of the free-exploration movements (see Results for details). In addition to the main task, participants made binary judgments (i.e., yes/no) on their control of a single dot, over which the actual level of control was between 0% and 100% at 10% steps, after freely moving it for 3 s in each trial. This binary control judgment task was designed to measure the psychological function of the sense of control in the present control paradigm.

Figure 2.

Figure 2

Open in a new tab

Timeline of Control Difference Detection Task

Yellow text shows levels of control over each dot in an example of one experimental condition, but this was not shown on the screen during the experiment.

Results

Detection Accuracy: Effect of Control Context

Figure 3 shows the detection accuracy for each condition in the control difference detection task. We focused on whether the control context selectively affected the detection accuracy depending on the direction difference between the target and the control context. The detection accuracy for each condition except for 0% difference was compared with the chance level (33.3%) using one-sample t tests. The significance level was set to .002 (= .05/22) according to Bonferroni's correction for multiple comparison. Shapiro-Wilk's tests confirmed no significant deviation from normality for all the conditions (the same significance level as the t tests was used). The detection accuracy was significantly better than chance in the +20% and +30% conditions when the control context was at 30% (t(15) = 4.89 and 8.84, respectively, ps < .001; see red asterisks), in the +10%, +20%, and +30% conditions when the control context was at 50% (t(15) = 8.11, 7.68, and 5.35, respectively, ps < .001), in the −10%, −20%, and −30% conditions when the control context was at 70% (t(15) = 4.17, 16.86, and 19.89, p = .001, p < .001, p < .001, respectively), and in the −10%, −20%, and −30% conditions when the control context was at 90% (t(15) = 7.39, 10.48, and 25.49, respectively, ps < .001). The detection accuracy did not significantly differ from the chance level in the remaining conditions. In short, the results show that the participants were only able to detect the target when the control over it was more than the others in the low-control context (30% and 50%) or when the control over it was less than the others in the high-control context (70% and 90%).

Figure 3.

Figure 3

Open in a new tab

Detection Accuracy for a Dot under a Different Level of Control (10%–30% more or less) than the Other Two Dots

Red asterisks represent significant differences from chance level (i.e., dashed lines) (a Bonferroni-corrected significance level of .002 was used). Error bars represent 95% confidence intervals. ∗p < .002, one-sample t tests against 0.33.

One may argue that the above results are due to poor resolution of difference at extremes. In other words, perceiving a difference between two levels of control may be difficult if the two levels are both very high or both very low, but it may be easy if the two levels are both intermediate. Therefore, we chose three pairs of control difference from the experimental conditions, and each of the participants was required to identify either the lower level or the higher level of each pair as the target. We compared the detection accuracy for each pair between the condition when people need to detect poor control from relatively good control and that when people need to detect good control from relatively poor control. For example, if one can tell the difference between 30% control and 50% control, theoretically the difficulty of detecting 30% control from a 50% control context should be the same as that of the reverse condition. The three pairs were 30% versus 50%, 50% versus 70%, and 70% versus 90%, where either side was the control context or the target. Figure 4 shows the identification accuracies obtained for these pairs. A repeated-measures ANOVA (3 × 2, type of comparison [30% versus 50%, 50% versus 70%, and 70% versus 90%) × direction [more versus less]) revealed a significant main effect of the type of comparison (F(2, 30) = 35.01, p < .001, partial η2 = .700), a significant main effect of direction (F(1, 15) = 6.31, p = .024, partial η2 = .296, better performance in the condition of less control than in that of more control), and a significant interaction (F(2, 30) = 35.79, p < .001, partial η2 = .705). Mauchly's test of sphericity indicated that the assumption of sphericity had not been violated (type of comparison: χ2(2) = 5.591, p = .061; interaction: χ2(2) = 4.077, p = .130). Bonferroni correction was used for post hoc comparison (i.e., multiplying p value by the number of repeated comparisons). Regarding the main effect of type of comparison, the detection accuracy was higher in the comparison between 50% and 70% than in the other two comparisons (ps < .001, original p values multiplied by 3). Regarding interaction, we compared detection accuracy between the more and the less conditions for each type of comparison. The task performance was better in the more condition for the comparison between 30% and 50% control (p = .003, original p value multiplied by 3), and it was better in the less condition for the comparison between 70% and 90% control (p < .001, original p value multiplied by 3). There was no significant difference between the more and less conditions for the comparison between 50% and 70% control (p = .303, original p value multiplied by 3).

Figure 4.

Figure 4

Open in a new tab

Detection Accuracy in the Selected Experimental Conditions

Red bars represent identification accuracies when the target was under more control than the distractors; blue bars represent identification accuracies when the target was under less control than the distractors. For example, in the 30% versus 50% condition, the red bar shows the detection accuracy when the target was 50% control and the distractors were under 30% control, whereas the blue bar shows the detection accuracy when the target was 30% control and the distractors were under 50% control. Error bars represent 95% confidence intervals.

In summary, the results clearly show that control context modulated the perceptual sensitivity to control difference. This perceptual sensitivity does not directly reflect the differences between sensorimotor signals at different control levels. For example, the detection accuracies were significantly different between detecting 30% target control from a 50% control context and detecting 50% target control from a 30% control context, despite the fact that the differences in sensorimotor signals in the two cases were identical. Moreover, our results also showed that the perceptual sensitivity of control difference was higher when the control was at an intermediate level (50%–70%) than when the control was at an unambiguously high or low level.

Explicit Judgments of Control

In the control difference detection task, participants made the explicit judgment of whether the dot they selected was under more or less control than the other two dots. This question was designed to clarify the basic sense of control over the selected stimulus: Did they have a correct sense of control over the different objects when they correctly identified the target dot? Figure 5A shows the proportion of correct direction responses to the number of correct detection trials when the detection performance was better than chance level (i.e., in the 11 experimental conditions marked with red asterisks in Figure 3). The proportion of correct judgments on the direction of control difference was high (above 77%). Furthermore, we found a positive correlation between the proportion of correct control judgments and the detection accuracy in the 11 conditions at the individual level (Figure 5B). The average correlation coefficient was .34 (SD = .28), which was significantly larger than zero (t(15) = 4.88, p < .001, Cohen's d = 1.22, the normality was confirmed by a Shapiro-Wilk's test, W = .967, p = .782). In short, when participants correctly detected the target dot, they certainly felt a clear sense of control over it.

Figure 5.

Figure 5

Open in a new tab

Response on Direction of Control Difference

(A) Proportion of correct responses on direction to the number of correct detection trials in the conditions when the participants' detection performance was better than chance level. Results show that, when participants correctly identified the target, they also had an explicit and correct sense of control over it. Error bars represent 95% confidence intervals.

(B) Per-individual plot showing proportion of correct judgment against detection accuracy in the conditions when the participants' detection performance was better than chance level. Different colors represent individuals. Solid lines represent the results of linear regression for each individual.

Kinematic Analyses: Timeline of Control Sensing

We first examined the total moving distance in each trial among conditions to confirm the following two predictions: (1) more movements are linked to better control; (2) fewer movements are linked to correct detection of the target. Then, we examined the above two effects along the timeline to determine when people perceived the difference in control context and when they detected the target.

Figure 6 shows the kinematics of the participants' motion. Figure 6A shows the averaged moving distance of the finger in each condition. Previous developmental search reported that the detection of control (i.e., detection of the contingency between one's action and sensory feedback) has a reinforcement effect on the frequency of behavior (Rochat and Morgan, 1995, Rovee and Rovee, 1969). Furthermore, Karsh and Eitam suggested that control is an internal reward for humans, enhancing the speed of action execution and increasing the frequency of action selection (Karsh and Eitam, 2015, Karsh et al., 2016, Penton et al., 2018). We also observed this phenomenon in the moving distance among different conditions of control context (Figure 6A, see below for details of statistics). Moreover, our experimental task was strongly goal oriented. After the target is identified, i.e., once the control difference is successfully sensed, the main motivation of movement diminished, and thus movements should decrease. Figure 6A showed a trend of less moving distance in the conditions with better detection accuracy (e.g., more conditions in 30% and 50% control context, less condition in 70% and 90% control context), supporting the prediction of a decline in movement when the target is detected. The behaviors in this task provide proxies for the timeline in the sensing of control: when people find their control over the environment and when they find the target. The details of the statistics are given below.

Figure 6.

Figure 6

Open in a new tab

Moving Distance of the Finger on the Touchpad

(A) Total moving distance in each condition. Error bars represent 95% confidence intervals.

(B) Moving distance in each sliding time window of 1 s at 0.2-s steps, depending on the control difference condition (no difference, more, or less) and response (correct or incorrect). Results show that a higher control context is linked to more movements, whereas correct detection of the target is linked to reduced movements. Vertical gray broken lines show the time window from which the main effect of control context emerged, indicating the sensing of control context. Vertical black broken lines show the time window from which the interaction between control context and trial type emerged, indicating the identification of the target (i.e., control difference).

Normal distribution of the results from all the 12 conditions was confirmed with the Shapiro-Wilk's tests (ps > .05). The mean moving distance was analyzed with a 3 (direction of difference between target and context: more, less, and no difference) × 4 (control context: 30%, 50%, 70%, and 90%) repeated-measures ANOVA. The Greenhouse-Geisser correction was used because Mauchly's test of sphericity was violated (direction of difference: χ2(2) = 4.899, p = .086; control context: χ2(5) = 20.069, p = .001; interaction: χ2(20) = 43.483, p = .002). The ANOVA revealed a significant main effect of direction of difference (F(1.544, 23.162) = 12.69, p < .001, partial η2 = .458), a significant main effect of control context (F(1.606, 24.093) = 14.03, p < .001, partial η2 = .483), and a significant interaction (F(3.135, 47.018) = 5.51, p = .002, partial η2 = .269). Bonferroni correction was used for post hoc comparison. First, we examined the main effect of control context. Participants moved their fingers for a significantly longer distance when the control context was highest than under the three other control context conditions (90% control context versus 30% control context: p = .004; 90% control context versus 50% control context: p = .006; 90% control context versus 70% control context: p < .001, original p values multiplied by 6). There was no significant difference in moving distance among the 30%, 50%, and 70% control contexts. The longer moving distance in the 90% control context reflects control-motivated movements: people were more willing to move their fingers when they felt better control over the stimuli. Importantly, this effect holds even when we exclude the possible influence of task difficulty by only considering the no difference condition (gray bars in Figure 6A). A repeated-measures ANOVA revealed the significant main effect of control context on the moving distance in the no difference condition (F(3, 45) = 6.07, p = .001, partial η2 = .288; Mauchly's test of sphericity: χ2(5) = 9.014, p = .109). Bonferroni-corrected comparisons showed that people moved significantly more in the condition of 90% control context than in the conditions of 30% control context (p = .031, original p value multiplied by 6) and 50% control context (p = .022, original p value multiplied by 6).

Furthermore, the significant interaction confirmed our prediction that declines in movements were associated with correct detections (i.e., less control condition in 30% and 50% control contexts and more control condition in 70% and 90% control contexts). We compared the moving distance between each control difference condition in each control context. For the 30% control context, movements did not differ among control difference conditions (F(2, 30) = 0.71, p = .498, partial η2 = .045). For the 50% control context, the main effect of control difference was significant (F(2, 30) = 7.64, p = .002, partial η2 = .337). Averaged moving distance was shorter in the more control condition than in the other two conditions (more versus less: p = .048; more versus no difference: p = .010, original p values multiplied by 3). For the 70% control context, the main effect of the control difference condition was significant (F(2, 30) = 8.77, p = .001, partial η2 = .369). Movements were fewer in the less control condition than in the other two conditions (more versus less: p = .003; less versus no difference: p = .017, original p values multiplied by 3). For the 90% control context, the main effect of control difference was also significant (F(2, 30) = 7.00, p = .003, partial η2 = .318). Movements were fewer in the less control condition than in the other two conditions (more versus less: p = .038; less versus no difference: p = .013, original p values multiplied by 6). In addition, a simpler comparison of moving distance between correct detection trials and incorrect detection trials (excluding the no-difference trials) showed that movement distance was significantly shorter in correct trials than in incorrect trials (Mcorrect = 5,003, SDcorrect = 1,165, Mincorrect = 5,187, SDincorrect = 1,169, where the unit was pixels of mouse movement; t(15) = 2.53, p = .023, Cohen's d = 0.63). In summary, as we predicted, a higher control context was linked to more movements (shown by the main effect of control context on moving distance), and successful detection of the target was linked to a reduction in movements.

Moreover, in order to examine the temporal feature of active sensing of control, we used a sliding window of 1 s with 0.2-s steps and calculated the moving distance accumulated within each window. The moving distance in each sliding window is depicted in Figure 6B. The drop in averaged moving distance in each window along the timeline again confirmed the predicted effect of goal achievement (i.e., a decrease in motion when the target is successfully detected). According to the analyses of moving distance from all trials (Figure 6A), the main effect of control context on moving distance reflects control-motivated behaviors. In other words, when sliding the time window, the emergence of the main effect of control context indicates that people understand the control context. On the other hand, the analyses from all trials also revealed that the moving distance was shorter for the trials in which the target was more successfully identified. Therefore, the decreased moving distance in the trials indicates the identification of the target. Here, we divided the trials into five control difference & response conditions: no difference, more control & correct response, more control & incorrect response, less control & correct response, and less control & incorrect response. Because participants were more successful in identifying a more controllable target in the poor control context but more successful in identifying a less controllable target in the good control context, an interaction between control context and control difference & response would be a sign of sensing the control difference. This interaction better reflects the influence of goal achievement than the decreased movement in correct detection trials, since a gradual decrease in movement was observed in all conditions, even when the target was not detected. In order to avoid false significance due to multiple comparisons, an effect in a time window can be confirmed as significant only when its significance holds in all of the subsequent time windows.

We conducted 4 × 5 (control context × control difference & response) repeated-measures ANOVA while sliding the time window. Greenhouse-Geisser correction was used. We found that the main effect of control context became significant from the sixth time window (F(1.578, 23.672) = 4.56, p = .028, partial η2 = .233; main effect remaining significant in all of the subsequent windows), and the interaction became significant from the 16th time window (F(3.371, 50.562) = 2.89, p = .039, partial η2 = .161; interaction remaining significant in all of the subsequent windows). The results show that during the sensing of control difference among objects, people explored and understood their control over the environment within the first 2 s (i.e., the end of the sixth time window). Good control maintained the movement frequency, whereas poor control reduced it. Then, about 4 s after the onset of movement (i.e., the end of the 16th time window), detection of the target resulted in decreased movement. Taken together, the analysis of a sliding window supports our hypothesis by showing that people first explore their control over the environment and then, under the resulting tuned perceptual sensitivity, find the control difference.

Binary Judgment of Control: Actual Control Mapping to Subjective Judgment of Control

Finally, Figure 7 shows the binary judgment and the logistical fitting curves in the control judgment task. The fitting curves show how the objective level of control was mapped to subjective judgment of control (Yon et al., 2019). Binomial logistic regressions were carried out with MATLAB R2017a (The MathWorks, Inc.) using the generalized linear model function in the Statistics and Machine Learning Toolbox. The point of subjective equality (PSE) was calculated to predict 50% of Yes-response from the logistic functions. The mean average of PSE was 58.4% of actual control, ranging from 35.6% to 76.2%, with a standard deviation of 9.5%. According to these results, we concluded that the participants felt a reliable sense of control over the majority of stimuli in the conditions of 70% and 90% control context but felt weak or no sense of agency over the majority of stimuli in the conditions of 30% and 50% control context. Furthermore, the results of binary agency judgment also confirmed that the selection of the four control contexts (30%, 50%, 70%, and 90%) had roughly balanced distance from the midpoint of subjective feeling of control rather than the midpoint of actual control. In addition, there was no significant correlation between the individual PSE and task performance.

Figure 7.

Figure 7

Open in a new tab

Response and Logistic Fitted Curves in the Binary Control Judgment Task

Colors represent individuals. Dots connected by lines represent the participants' responses, and curves represent the logistic fitting results. Black broken lines and black dots represent the mean proportion of Yes-responses (self-control response).

Discussion

This study examined the sensing of control difference among objects. Previous studies suggest that the sense of control over an object is based on the relationship computed from one's action and the sensory feedback from that object. In this study, we extended the present theories by proposing a two-step model of the sensing of control for multiple objects, stressing the importance of control context in control sensing. We suggest that the early sensing of control affordances from the environment is critical for the subsequent precise sensing of control difference. Our two-step control sensing model was supported by the following results. First, the sensing of control difference between two levels of control was greatly affected by control context, suggesting the existence of an initial step that explores the level of overall control over multiple interacting subjects. For example, as shown in Figure 4, the accuracy of detecting a 50% control target from a 30% control context was significantly higher than the accuracy of detecting a 30% control target from a 50% control context, regardless of the fact that the comparison was between 30% and 50% control in both cases. The significant interaction between control context and the effect of magnitude of control difference on detection accuracy provides evidence against the alternative hypothesis of sequential sampling of control. Second, the results of the kinematic analyses in Figure 6 show that the effect of control context on the participants' motion emerged much earlier than the effect of goal achievement (i.e., detection of control difference) on participants' motion. This result supports our two-step model by showing that people first sense the control context and then orient their attention to the target that is under a different level of control from the others.

The results from this study show that a high control context selectively enhances the sensitivity of detecting the lack of control but weakens the sensitivity of detecting more control. By contrast, a low control context shows the opposite effects, enhancing the detection of more control but weakening the detection of less control (Figure 3). Importantly, these results cannot be simply explained by the poor resolution of sensorimotor signals when they are at extremes. For example, it is difficult to discriminate the brightness between two stimuli if they are both very bright or very dark, but it is easier to discriminate them if their brightness levels are at intermediate values, despite the fact that the physical difference values between pairs are identical (Goldstone, 1994). However, we argue that our findings indicate something more than this psychological characteristic. We found that, even for the same comparison of control difference, the discrimination of control was dramatically influenced by the control context (Figure 4). For instance, for the comparison between 30% and 50% control, people were able to perceive the difference between the two levels when the control context was 30% and the target was under 50% control (detection accuracy = 49.4%), but they were unable to identify the target when the control context was 50% and the target was under 30% control (detection accuracy = 26.9%, nonsignificant difference from chance level). The results clearly showed that the sensing of control difference is not a simple projection of the comparison between sensorimotor signals. In addition, our results also show that more moderate levels of control are easier to detect when among more extreme control contexts. The comparison between moderate levels of control was more accurate than those between extreme levels (Figure 4). This is in line with a well-known phenomenon called categorical perception, which has been reported in many types of perception such as color, language, and face (Goldstone, 1994). The initial exploration of control context can be considered a fast categorizing behavior, and the result of categorization shapes the following behaviors and perceptions related to further exploitative sensing of control. However, the sensing of control is usually an interplay between voluntary actions and perception, whereas in the traditional research on categorical perception only passive sensory input is involved. In our experimental task, the three stimuli moved in very different directions, so one could not perform the task with only visual input. The only cue to solve it was the sensorimotor correlations among one's motion and the dots' moving directions. Our findings shed light on how people solve the ambiguity of control through their actions and how this initial sensing behavior influences the perception of action consequences.

Moreover, the movement kinematics revealed temporal features of the sensing of control difference. The analysis of moving distance in sliding windows showed that people perceive the difference between control contexts at a very early stage, within 2 s after the onset of movement (Figure 6). Such a sense of control over the control contexts influenced the explorative movements among conditions of control context: There was more motivation for sensing motion in a higher control context. Next, once people detect the target, their movements decrease following goal achievement. This effect arose 4 s after the onset of movement, showing that the sensing of control context and the sensing of control difference are not simultaneous. Instead, the latter occurs after the former. The results support our hypothesis of a two-step control exploration: People first sense their control over the environment, and then this initial sense of control tunes the perceptual sensitivity for the subsequent precise sensing of control. In other words, people do not sense their control over the three dots one by one and then simply sum them up or compare them.

The findings of the present study provide insight relevant to the debate over the role played by prediction of action outcomes. The classic account, such as the comparator model, suggests that people are very sensitive to deviation from the prediction of action consequences (Blakemore et al., 1998), as revealed by the phenomenon of sensory attenuation. In sensory attenuation, sensory input that does not match the prediction based on one's motor commands is perceived more intensely than that matching the prediction, since the former is “canceled” from perception (Blakemore et al., 1999, Wolpert and Flanagan, 2001). Furthermore, recent studies have suggested that the perception of predicted action outcomes is actually initially “facilitated” (Roussel et al., 2013), and the deviation from the prediction becomes salient while the evidence of “cancellation” is revealed at a later point in time (Yon and Press, 2017). In addition, Press et al. (2019) suggested a two-process model, in which the perceptual system initially upweights sensory input that is likely to occur and deviant events become surprising when they are perceived as errors at a later point in time (Press et al., 2019). Our results support the latter theory of a later sensitivity to “errors.” We found that people were sensitive to errors only after sensing a high control context. The initial sensing of control context resembles the acquisition of probabilistic knowledge regarding action outcomes in the two-process model of Press et al. (2019).

In summary, this study's results support our hypothesis of two-step sensing of control over the environment. People first sense their overall control over the environment at an early stage. Such an early sense of control tunes the perceptual sensitivity of detecting more or less control in the subsequent more precise sensing of control. However, it remains unknown whether the sensing of more control or less control in the second stage shares the same processing mechanism of sensorimotor signals. Nevertheless, our results show that control over the environment is critical. When people do not have much control over the environment, the cognitive system is selectively sensitive to more control. Once control has been acquired, any small lack of control is highly salient.

Limitations of Study

There are various issues with the task that need to be discussed. Although in most conditions the three dots moved in different trajectories, the greater the level of contextual control, the more that the finger movement would globally dominate the stimuli's coherent motion. In such cases, detection of a less coherent motion may be easier than that in a condition with lower control context. For example, the detection accuracy of 10% less control was indeed higher in the 90% control context than the 70% control context (60% versus 48%). However, we argue that our main findings stand even after considering the influence of the coherent motion in a high control context. First, this effect cannot account for the asymmetrical detection accuracy for more and less controlled targets in the same control context. Second, it cannot account for the higher detection accuracy of a 50% controlled target from 30% control context compared with that of a 30% controlled target from 50% control context.

Another issue with the task is that it allows free exploration. The participants may have used very different motion or control sensing strategies, which perhaps resulted in the large individual difference in binary control judgments shown in Figure 7. However, we believe that the free-exploration process is critical to studying the active sensing of control. Any regulation of motion may bring an unexpected effect to the sense of control, such as comparison between goal and goal achievement (Wen et al., 2015b, Wen et al., 2015a). Moreover, although free exploration may cause a large deviation in motion, active motion itself indeed provided important and useful implicit proxies for the sensing of control, as shown in Figure 6 and the kinematic analyses.

In the end, one may argue that the effect of control context is due to top-down selective attention. However, we believe this was unlikely, because both directions of control difference were equally presented for each control context, and this fact was clearly explained to the participants.

Methods

All methods can be found in the accompanying Transparent Methods supplemental file.

Acknowledgments

This work was supported by the Japan Society for the Promotion of Science KAKENHI grant numbers 26120002 and 26120005. W.W. was additionally supported by KAKENHI Early-Career Scientists grant number 19K20642. H.I. was supported by KAKENHI grants (numbers 18H01098, 19H01777, and 19H05725), and “Research and development of technology for enhancing functional recovery of elderly and disabled people based on non-invasive brain imaging and robotic assistive devices” by the Commissioned Research of National Institute of Information and Communications Technology.

Author Contributions

W.W., H.S., and H.I. initiated and designed the research. All authors reviewed the experimental design. W.W. and H.S. performed the experiments and analyzed the data. W.W. wrote the manuscript with the help from R.O. and H.I. All authors read and approved the final manuscript.

Declaration of Interests

We declare no competing interests.

Published: May 22, 2020

Footnotes

Supplemental Information can be found online at https://doi.org/10.1016/j.isci.2020.101112.

Supplemental Information

Document S1. Transparent Methods
mmc1.pdf (139.1KB, pdf)

References

  1. Blakemore S.-J., Wolpert D.M., Frith C.D. Central cancellation of self-produced tickle sensation. Nat. Neurosci. 1998;1:635–640. doi: 10.1038/2870. [DOI] [PubMed] [Google Scholar]
  2. Blakemore S.-J., Frith C.D., Wolpert D.M. Spatio-temporal prediction modulates the perception of self-produced stimuli. J. Cogn. Neurosci. 1999;11:551–559. doi: 10.1162/089892999563607. [DOI] [PubMed] [Google Scholar]
  3. Blakemore S.-J., Wolpert D.M., Frith C.D. Abnormalities in the awareness of action. Trends Cogn. Sci. 2002;6:237–242. doi: 10.1016/s1364-6613(02)01907-1. [DOI] [PubMed] [Google Scholar]
  4. Eitam B., Kennedy P.M., Higgins E.T. Motivation from control. Exp. Brain Res. 2013;229:475–484. doi: 10.1007/s00221-012-3370-7. [DOI] [PubMed] [Google Scholar]
  5. Friston K., FitzGerald T., Rigoli F., Schwartenbeck P., O’Doherty J., Pezzulo G. Active inference and learning. Neurosci. Biobehav. Rev. 2016;68:862–879. doi: 10.1016/j.neubiorev.2016.06.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Frith C.D., Blakemore S.-J., Wolpert D.M. Abnormalities in the awareness and control of action. Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci. 2000;355:1771–1788. doi: 10.1098/rstb.2000.0734. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Goldstone R.L. Influences of categorization on perceptual discrimination. J. Exp. Psychol. Gen. 1994;123:178–200. doi: 10.1037//0096-3445.123.2.178. [DOI] [PubMed] [Google Scholar]
  8. Haggard P. Sense of agency in the human brain. Nat. Rev. Neurosci. 2017;18:197–208. doi: 10.1038/nrn.2017.14. [DOI] [PubMed] [Google Scholar]
  9. Haggard P. The neurocognitive bases of human volition. Annu. Rev. Psychol. 2019;70 doi: 10.1146/annurev-psych-010418-103348. [DOI] [PubMed] [Google Scholar]
  10. Haggard P., Chambon V. Sense of agency. Curr. Biol. 2012;22:R390–R392. doi: 10.1016/j.cub.2012.02.040. [DOI] [PubMed] [Google Scholar]
  11. Karsh N., Eitam B. I control therefore I do: judgments of agency influence action selection. Cognition. 2015;138:122–131. doi: 10.1016/j.cognition.2015.02.002. [DOI] [PubMed] [Google Scholar]
  12. Karsh N., Eitam B., Mark I., Higgins E.T. Bootstrapping agency: how control-relevant information affects motivation. J. Exp. Psychol. Gen. 2016;145:1333–1350. doi: 10.1037/xge0000212. [DOI] [PubMed] [Google Scholar]
  13. Kumar N., Manjaly J.A., Sunny M.M. The relationship between action-effect monitoring and attention capture. J. Exp. Psychol. Gen. 2015;144:18–23. doi: 10.1037/xge0000032. [DOI] [PubMed] [Google Scholar]
  14. Moore J.W. What is the sense of agency and why does it matter. Front. Psychol. 2016;00:00. doi: 10.3389/fpsyg.2016.01272. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Moore J.W., Fletcher P.C. Sense of agency in health and disease: a review of cue integration approaches. Conscious. Cogn. 2012;21:59–68. doi: 10.1016/j.concog.2011.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Penton T., Wang X., Pierre M., Catmur C., Bird G. The influence of action – outcome contingency on motivation from control. Exp. Brain Res. 2018;236:3239–3249. doi: 10.1007/s00221-018-5374-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Press C., Kok P., Yon D. The perceptual prediction paradox. Trends Cogn. Sci. 2019;24:13–24. doi: 10.1016/j.tics.2019.11.003. [DOI] [PubMed] [Google Scholar]
  18. Rochat P., Morgan R. Spatial determinants in the perception of self-produced leg movements in 3- to 5-month-old infants. Dev. Psychol. 1995;31:626–636. [Google Scholar]
  19. Roussel C., Hughes G., Waszak F. A preactivation account of sensory attenuation. Neuropsychologia. 2013;51:922–929. doi: 10.1016/j.neuropsychologia.2013.02.005. [DOI] [PubMed] [Google Scholar]
  20. Rovee C.K., Rovee D.T. Conjugate reinforcement of infant exploratory behavior. J. Exp. Child Psychol. 1969;8:33–39. doi: 10.1016/0022-0965(69)90025-3. [DOI] [PubMed] [Google Scholar]
  21. Salomon R., Lim M., Kannape O., Llobera J., Blanke O. “Self pop-out”: agency enhances self-recognition in visual search. Exp. Brain Res. 2013;228:173–181. doi: 10.1007/s00221-013-3549-6. [DOI] [PubMed] [Google Scholar]
  22. Schneider W., Shiffrin R.M. Controlled and automatic human information processing: I. Detection, search, and attention. Psychol. Rev. 1977;84:1–66. [Google Scholar]
  23. Synofzik M., Vosgerau G., Voss M. The experience of agency: an interplay between prediction and postdiction. Front. Psychol. 2013;4:1–8. doi: 10.3389/fpsyg.2013.00127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Wegner D.M. The mind’s best trick: how we experience conscious will. Trends Cogn. Sci. 2003;7:65–69. doi: 10.1016/s1364-6613(03)00002-0. [DOI] [PubMed] [Google Scholar]
  25. Wen W., Haggard P. Control changes the way we look at the world. J. Cogn. Neurosci. 2018;30:603–619. doi: 10.1162/jocn_a_01226. [DOI] [PubMed] [Google Scholar]
  26. Wen W., Haggard P. Prediction error and regularity detection underlie two dissociable mechanisms for computing the sense of agency. Cognition. 2020;195:104074. doi: 10.1016/j.cognition.2019.104074. [DOI] [PubMed] [Google Scholar]
  27. Wen W., Yamashita A., Asama H. The sense of agency during continuous action: performance is more important than action-feedback association. PLoS One. 2015;10:e0125226. doi: 10.1371/journal.pone.0125226. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Wen W., Yamashita A., Asama H. The influence of goals on sense of control. Conscious. Cogn. 2015;37:83–90. doi: 10.1016/j.concog.2015.08.012. [DOI] [PubMed] [Google Scholar]
  29. Wen W., Brann E., Di Costa S., Haggard P. Enhanced perceptual processing of self-generated motion: evidence from steady-state visual evoked potentials. Neuroimage. 2018;175:438–448. doi: 10.1016/j.neuroimage.2018.04.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Wolpert D.M., Flanagan J.R. Motor prediction. Curr. Biol. 2001;11:R729–R732. doi: 10.1016/s0960-9822(01)00432-8. [DOI] [PubMed] [Google Scholar]
  31. Yon D., Press C. Predicted action consequences are perceptually facilitated before cancellation. J. Exp. Psychol. Hum. Percept. Perform. 2017;43:1073–1083. doi: 10.1037/xhp0000385. [DOI] [PubMed] [Google Scholar]
  32. Yon D., Bunce C., Press C. PsyArXiv; 2019. Illusions of Control without Delusions of Grandeur. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Document S1. Transparent Methods
mmc1.pdf (139.1KB, pdf)

Articles from iScience are provided here courtesy of Elsevier

RESOURCES