Body and visual instabilities functionally modulate implicit reaching corrections

Summary

Hierarchical brain-information-processing schemes have frequently assumed that the flexible but slow voluntary action modulates a direct sensorimotor process that can quickly generate a reaction in dynamical interaction. Here we show that the quick visuomotor process for manual movement is modulated by postural and visual instability contexts that are related but remote and prior states to manual movements. A preceding unstable postural context significantly enhanced the reflexive manual response induced by a large-field visual motion during hand reaching while the response was evidently weakened by imposing a preceding random-visual-motion context. These modulations are successfully explained by the Bayesian optimal formulation in which the manual response elicited by visual motion is ascribed to the compensatory response to the estimated self-motion affected by the preceding contextual situations. Our findings suggest an implicit and functional mechanism that links the variability and uncertainty of remote states to the quick sensorimotor transformation.

Graphical abstract

Highlights

• •Interactions between hierarchical sensorimotor mechanisms remain enigmatic
• •Lower sensorimotor mechanism has been thought to be regulated by the voluntary mechanism
• •We show a novel modulation of visuomotor response without voluntary action guidance
• •This is caused by a mechanism to consider body instability in implicit arm reactions

Neuroscience; Sensory neuroscience; Cognitive neuroscience

Introduction

Skilled and dynamic motor actions are realized by interwoven mechanisms consisting of voluntary and involuntary components.^1,2 Reflexive sensorimotor control, a particular class of involuntary sensorimotor process, plays a significant role in complex, dynamic situations, especially in sport, where drastic changes in body state and the external environment (e.g. a ball and opponents in a football game, or water and wind in sailing) occur rapidly. This is because the reflexive sensorimotor process can generate motor commands more quickly, and with less computational cost, than the voluntary action process, by bypassing the conscious decision-making process.

Although reflexes are often thought of as stereotyped and inflexible reactions to particular stimuli, a number of empirical and theoretical studies^{2,3,4,5,6,7,8,9,10,11} emphasized that stretch reflex responses can be functionally modulated by task goals and environmental context. Such quick reactions are also observed for visuomotor responses in visually guided dynamic motor tasks. For instance, we can make rapid online reaching corrections to visual shifts of a target^12,13,14,15 or a hand cursor,^16,17,18 or to the surrounding visual motion^19,20,21,22 before having intention to adjust the movement. Task goals and environmental context are also important in modulating these visuomotor responses.^{14,15,23,24,25,26}

Despite those investigations, less is clear about the mechanism of reflex modulation with respect to its interaction with voluntary motor adjustment. It has been widely considered that the voluntary motor system generates appropriate motor commands based on information derived from the consciously perceived task goal and environment, although voluntary reaction processing takes longer than reflexive sensorimotor processing.^1,2 Actually, in a high level cognitive-motor process such as decision making and perception, the resultant action is flexibly formed with the optimal integration of multisensory information^27,28,29 or of prior information and sensory evidence.^30,31,32 Such flexibility and optimality relative to the task and environment therefore led to the hypothesis that reflexive response modulation is guided by subsequent voluntary motor activities in repetitive trials^23,33,34 or is observed in superposition with the initial part of the voluntary reaction.^35,36 Since most of these previous studies have examined the modulation of reflexes accompanied by subsequent voluntary reactions, it is hard to consider from these studies whether the modulation of reflex response requires assistance of voluntary reaction or not.

To discover additional important aspects of reflex modulation mechanisms for reaching movements, here we investigated the contextual effect related to remote internal states on the reflexive visuomotor response, called the manual following response (MFR). The MFR evoked in a short latency by a large-field visual motion during reaching movement is known to be modulated by visual stimulus features (motion coherence, contrast, stimulus area, spatiotemporal frequency of the visual stimulus, and so forth)^21,22,37 and by the arm reaching states.^21,38,39 However, the effect of actual/estimated postural states on the MFR was not well examined while the function of MFR has been suggested as a compensatory response to the postural fluctuation estimated from visual motion.^20,21,40 We, therefore, conducted a series of experiments to examine the MFR modulation to the preceding contexts of postural and/or visual instabilities. Our results in the current study clearly show that the reflexive manual response was modulated according to the imposed prior contexts, that related to the postural and visual instabilities, without direct assistance from voluntary reactions. Additionally, these reflex modulations were well explained by the Bayesian inference model in estimating self-motion, supporting the hypothesis that the MFR is driven by illusory self-motion rather than by an object motion. Intriguingly, while both the MFR and voluntary reaction are similarly susceptible to the change of visual stimuli,^41,42 the modulation of MFR and voluntary reaction were not consistent for the imposed prior contexts. This dissociation further supports that the obtained modulation of reflexive visuomotor response is not guided by voluntary reactions. Our findings extend the current understanding of the hierarchical sensorimotor mechanisms by exhibiting an implicit and functional linking of low-level processing for quick reactions, with remote state variability and uncertainty.

Results

Modulation of reflexive visuomotor response depending on postural stability context

In our daily actions, we frequently make hand-reaching movements to an external object, during either active or passive self-motion. To realize successful reaching, hand movement should be adjusted in the opposite direction to self-motion because the shoulder is attached to the body. The self-motion, on the other hand, causes the retinal motion in the opposite direction to the self-motion. Therefore, one could imagine that the quick manual response induced by visual motion, is tightly linked to the self-motion representation. This phenomenon, however, could be also interpreted as a manual adjustment caused by a shift of the target position representation affected by the retinal motion.^19,20

To explore whether the reflexive manual following response (MFR) is modulated according to the postural environment, in each trial of the first experiment, participants were first asked to stand on a motion platform or a wooden platform in the UP (unstable posture) or SP (stable posture) context phase in front of the screen, on which a contrast grating pattern was drawn, as illustrated in Figure 1A. The motion platform undulated so as to make participant’s posture unstable in the context phase (5 s) while the posture was almost stable on the wooden platform (See experiment 1 in method details). The posture and hand movements were captured by a 3D-motion capture system. Importantly, the participants were instructed to keep standing without any reaching movement during the context phase to exclude the possibility of the experience of voluntary reaching correction in each context phase.

Figure 1.

Experimental configuration and procedure

(A) Cartoon of the experimental setup in which participant stood on the motion platform in front of the screen.

(B) The sequence of a single trial in Experiment 1. Participant stands on a wooden (stable) or motion (unstable) platform while watching a screen on which a contrast grating pattern is back-projected. After applying one of the two contexts (SP: Stable posture context or UP: Unstable posture context), a reaching target marker is briefly shown on the center of the screen, and then the participant performs a reaching movement initiated by a beeping sound. During reaching movement, a visual motion (rightward or leftward) is applied in two-third of the trials to induce reflexive visuomotor response (MFR).

After the postural context phase (SP or UP) in each trial, the participant was asked to perform arm-reaching movements toward the remembered target on the screen, which was briefly shown before reaching start, in the test phase, as described in the trial sequence in Figure 1B. Note that the motion platform was stabilized in the test phase. When the grating pattern on the screen suddenly started to move rightward or leftward during reaching movements, the hand trajectory automatically deviated in the direction of the visual motion, as shown in Figure 2A (orange curve for the leftward motion, green curve for the rightward motion). As shown in an example of the temporal profiles of hand acceleration along the x axis (Figure 2B), the acceleration started to change significantly 104 ms after the visual motion onset, time 0 s (See Data analysis for details). This quick manual response, i.e., MFR, was observed in all participants (mean latency ± SD across participants: 113 ± 13.9 ms).

Figure 2.

Manual and body responses induced by sudden visual motions

(A) Averaged reaching trajectories in the x-y plane (green curve: rightward visual motion; orange curve: leftward visual motion; black dashed curve: no visual motion) after experiencing the stable platform. Orange and green arrows indicate the visual motion direction, and open square on the trajectory indicates the mean hand position at the visual motion onset.

(B) Temporal acceleration patterns of manual response in the x-direction in three visual motion (rightward: green; leftward: orange; no motion: dashed black) conditions. Time zero denotes the onset of visual motion. The upward arrow indicates the statistically detected initiation of the MFR response.

(C) Temporal patterns of MFR induced by the visual motion (acceleration difference for the rightward visual motion and leftward visual motion) in the test phase after applying the UP (unstable posture: magenta) and SP (stable posture: blue) contexts. (Solid curve: mean across participants; Shaded area: SE). The horizontal black line just above the horizontal axis denotes the duration for calculating the mean response amplitude. The right bar graph shows the mean MFR amplitudes across participants in the SP and UP contexts. Error bars denote SE. Double asterisks: significance at p < 0.01. Each connected thin line represents the data of an individual participant.

(D) Temporal patterns of body sway (difference of body accelerations for the rightward and leftward visual motion) induced by visual motion in the test phases after the SP and UP contexts. The right bar graph shows acceleration amplitudes of body sway. Notations are the same as in panel c. There was no statistical difference between the response amplitudes for the SP and UP contexts.

By taking the difference between the hand accelerations for the leftward and rightward visual motion, the temporal patterns of MFR for the SP and UP contexts were illustrated in the left panel of Figure 2C. The onset latency of the response was 112 ± 15.8 ms for the SP context and 113 ± 12.2 ms for the UP context, and no significant difference was found in the latency (Paired t-test: t(17) = 0.38, p = 0.71). On the other hand, the peak of the response appeared to be greater for the UP than for the SP context. The response amplitudes (quantified by temporal averaging over a period of 120-160 ms from the onset of visual motion) (right panel in Figure 2C) for the stable (2.68 ± 0.84 m/s²) and unstable (3.21 ± 0.84 m/s²) contexts were significantly different (Paired t-test: t(17) = 3.2, p = 0.005, Effect size dz = 0.76). Importantly, since the reaching movements were performed after the context phase, postural stability and the visual motion signal during the reaching movement were identical in the test phases after all contexts.

Since sudden visual motion also induces a rapid postural sway,⁴³ the context-dependent modulation of MFR could be ascribed to the change in postural response that could also be altered by the two contexts. However, the body-sway responses induced by the visual motion shown in Figure 2D were not different for these two contexts (Paired t-test: t(17) = 0.043, p = 0.97). We further examined whether the postural contexts altered any aspect of performance in reaching movements (i.e., endpoint variability, movement duration, or tangential velocity), which could modulate MFR. However, for the paired t-test comparisons between the reaching movements in the no-visual-motion trials after applying the SP and UP contexts, no significant differences emerged in the movement duration [SP: 718 ± 66 ms, UP: 709 ± 62 ms, t(17) = 0.97, p = 0.34], in the endpoint position on the x axis [SP: 7.3 ± 1.2 cm, UP: 7.2 ± 1.1 cm, t(17) = 0.81, p = 0.43] and on the z axis [SP: 41.0 ± 2.1 cm, UP: 41.0 ± 2.0 cm, t(17) = 0.72, p = 0.48], in the endpoint variability on the x axis [SP: 6.4 ± 2.1, UP: 6.9 ± 1.9, t(17) = 1.01, p = 0.33] and on the z axis [SP: 6.4 ± 4.3, UP: 7.4 ± 7.5, t(17) = 0.98, p = 0.34], and the averaged tangential velocity between the stimulus onset and the end of the MFR calculation [SP: 1.47 ± 0.10 m/s, UP: 1.48 ± 0.075 m/s, t(17) = 0.50, p = 0.63]. These results additionally rule out the possibility that the context-dependent modulation of MFR is due to any changes in the performance of the reaching movements. These analyses, therefore, support the account that the MFR modulation was due to the postural contexts given just before the reaching movements.

Manual following response modulation depending on postural and visual contexts

To explore the modulation mechanism underpinning the implicit visuomotor response, we next examined what factors of the postural context contribute to the MFR modulation. Here we considered two major factors: intrinsic sensory (vestibular and somatosensory) and visual signals, both related to the postural control. The first possible account is that the MFR is simply modulated only by fluctuations of the intrinsic sensory signals that may represent the degree of postural instability experienced during the context phase and that MFR is irrelevant to the visual instability context realized by random visual motion, as summarized in the context effects on MFR by (A1) in Table 1. If this account is true, we could expect the greatest MFR after a context in which posture fluctuated the most. The second possible account (A2) is that the MFR modulation is mediated by the illusory postural instability estimated from visual fluctuation only. If this second account is true, we could expect that the MFR will increase by imposing a visual fluctuation context (random visual motion context), as summarized in the context effects on MFR by (A2) in Table 1. Since the participant receives the visual motion even in the postural fluctuation, this account could also explain the MFR increase by the unstable posture (UP) contexts, demonstrated in experiment 1. An additional third possible account (A3) is that the MFR modulation is governed by an integrated postural instability, considering the consistency between the visual motion and intrinsic sensory information. In this account, MFR will increase when visual motion is consistent with the intrinsic sensory signals both are actually caused by the postural fluctuation, but decrease with the visual motion unrelated to self-motion, as summarized in the context effects on MFR by (A3) in Table 1. Namely, MFR modulation would be affected by both the postural instability (MFR increasing effect) and the visual motion which is independent of self-motion (MFR decreasing effect).

Table 1.

Summary of expected MFR modulations induced by different postural and visual contexts under the three possible accounts

	Context effect on MFR
Context Possible account	UP, SV	SP, RV	UP, RV
(A1) MFR increase as intrinsic sensory (vestibular and somatosensory) signal increase during the context phase	+, 0	0, 0	+, 0
(A2) MFR increase as the visual motion signal increase during the context phase	+, 0	0, +	+, +
(A3) MFR modulation considering the consistency between visual motion and intrinsic sensory signals during the context phase	+, 0	0, -	+, -

“+,” “0,” and “-,” denotes the possible effects (increase, no change, and decrease) of the postural (left side) or visual (right side) context on the MFR amplitude, relative to the base context (stable posture and stable vision) in each account. SP: stable posture context; UP: unstable posture context; SV: static visual pattern (stable vision) context; RV: random visual motion (unstable vision) context.

To test these possibilities, we designed factorial combinations of visual and postural contexts (Figure 3A) in the second experiment and examined these contextual effects on the amplitude of MFR. As in the first experiment, MFR was induced by the visual motion during reaching movement in the test phase as depicted in Figure 3A (Procedure was identical to that in experiment 1) after applying one of the four context pairs: SP (Stable posture)+SV (Static visual pattern), SP + RV (Random visual motion), UP (Unstable posture)+SV, and UP + RV. In a single context pair, one of the postural contexts (SP or UP) and one of the visual contexts (SV or RV) were simultaneously applied. Note that the random visual motion applied in the context phase was generated using the phase-randomized head motion recorded in experiment 1 (See experiment 2 in method details).

Figure 3.

Effect of postural and visual contexts on MFR

(A) Four context conditions and sequence of one trial in Experiment 2. Two-by-two factorial design with one factor representing postural context (SP: stable posture; UP: unstable posture) and the other representing visual context (SV: static visual pattern; RV: random visual motion). After one of these contexts, the participant performs a reaching movement in the test phase of each trial as in Experiment 1.

(B) Body motion (root-mean-square ) during four context phases (SP + SV, UP + SV, SP + RV, UP + RV).

(C) MFR temporal patterns (mean across participants) in test phases after the four context phases.

(D) Mean amplitudes of MFR for the four contexts.

(E) Body sway induced by visual motion in test phases after the four contexts. In panels B, D, and E, error bars denote SE and thin gray lines denote the data of individual participants. Bar-edge colors denote green: SP + SV, red: UP + SV, blue: SP + RV, and cyan: UP + RV. Triple asterisks: significance at p < 0.001.

We first assessed the postural fluctuation during the context phase (See Data Analysis, postural stability for details). The postural fluctuation indexes (RMS: root-mean-square of lateral body movement) during the four context pairs (SP + SV, SP + RV, UP + SV, and UP + RV were 3.56 ± 1.60, 3.32 ± 1.56, 12.58 ± 2.01, and 16.47 ± 2.82 cm, respectively (See Figure 3B). A two-way ANOVA(postural x visual contexts) showed significant main effects of postural context (F(1,14) = 467.1, p < 0.0001, Partial η² = 0.97, Effect size f = 5.69), visual context (F(1,14) = 61.4, p < 0.0001, Partial η² = 0.81, Effect size f = 2.06), and interaction factor (F(1,14) = 75.4, p < 0.0001, Partial η² = 0.84, Effect size f = 2.29). The postural fluctuation was therefore the greatest when the platform was moved while providing random visual motion (UP + RV). If MFR regulation is simply mediated by the instability coded by the amplitude of the actual postural fluctuation during the context phase (account A1 mentioned above), we should expect that the MFR would be greatest for the UP + RV context. This was however not in the case as shown below.

Figure 3C shows the temporal profiles of MFR after each of the four contexts averaged across the participants. As seen in this graph, peak of the response looks greatest after the UP + SV context and least after the SP + RV context. The MFR amplitudes for the four contexts (bars and error-bar: mean and SE among participants) were drawn in Figure 3D. As in Experiment 1, the amplitude of the MFR was greater for the UP (pinkish-filled bars) than for the SP (blueish-filled bars) context in each visual context (SV or RV). On the other hand, the MFR amplitude was smaller for the RV than for the SV context in each postural context (SP or UP). A two-way ANOVA(postural x visual contexts) revealed a significant main effect of postural context (F(1,15) = 15.7, p = 0.001, Partial η² = 0.51, Effect size f = 1.02) and visual context (F(1,15) = 23.9, p = 0.0002, Partial η² = 0.61, Effect size f = 1.26). No significant interaction emerged (F(1,15) = 0.34, p = 0.57). Namely, the MFR amplitude for the UP + RV context was not maximal among the four conditions, whereas the actual postural fluctuation was maximal in the UP + RV context phase shown above (Figure 3B). This result, therefore, refutes the first account (A1) that the MFR modulation is simply associated with the degree of postural fluctuation.

Furthermore, since the MFR decreased by the random visual motion (RV) context respectively with SP and UP contexts as shown in Figure 3D, the second account (A2, mentioned above), which expected a greater MFR by the random visual motion context described above, does not hold either. Interestingly, in addition to the context-dependent MFR changes rigorously compared within Experiment 2, the MFR amplitudes for the SP + SV and UP + SV contexts (Figure 3D) look small, respectively compared with those for the same contexts (SP and UP) of Experiment 1. This could be ascribed to the random visual motion context, included in two of the four contexts in Experiment 2. Namely, it is possible that the RV context, which decreases MFR, had a longer effect than the UP context, which increases MFR, and that the residual effect of RV may have influenced the MFR over all blocks, because the RV context blocks were randomly ordered among the different context blocks (See experiment 2 in method details). Although details of the temporal histories of the contextual effects could not be known from the current results, effects of the four context conditions on MFR have been clearly and fairly characterized and the reduction of MFR by the context of random visual motion would be critically important toward elucidating the implicit mechanisms of visuomotor control during reaching movements.

Why did the context of random visual motion reduce the MFR amplitude? One possibility is that repeated exposure to random visual motion globally attenuated visual sensitivity in the early stages of visual processing, through a mechanism similar to the one underlying short-term visual adaptation.⁴⁴ However, this is not likely because the direction and speed of the visual motion were randomly changed during the context phase. Additionally, in the test phase, there was a duration without visual motion (∼2 s) before the test motion stimulus. Further, the rapid lateral body sway induced by the visual motion during reaching (See the Data analysis section) did not exhibit a similar modulation to the modulation of the MFR (Figures 3D and 3E). A two-way ANOVA of the body-sway amplitudes did not show significant interaction between postural and visual contexts (F(1,15) = 2.9, p = 0.11), or significant main effect of postural context (F(1,15) = 0.47, p = 0.51) or visual context (F(1,15) = 0.42, p = 0.53). Therefore, the MFR reduction caused by applying the random visual motion context cannot be explained by a change in sensitivity during the early stages of visual processing.

The above examinations indicate that the MFR does not simply increase with actual postural fluctuations detected only by intrinsic sensory (vestibular and somatosensory) signals, or with illusory postural instability induced only by random visual motion in the context phase. If the MFR is considered as a compensatory response against the estimated postural sway for correctly interacting with environments,⁴⁰ the MFR decrease by the random visual motion context would be reasonable since the exposure to random visual motion reduces the association between visual motion and actual body motion (the third account mentioned above). Namely, MFR would not be directly functioned to adjust arm movement to the moving visual object, but rather to the self-motion estimated from visual motion. In supporting this idea, when visual motion is properly associated with the postural fluctuation in the context (i.e., UP + SV), sensitivity of the MFR increased most among the four contexts. This would be beneficial for producing quick reaching adjustments against self-motion.

Changes in likelihood variance of Bayesian optimal formulation

To further explore the computational mechanism of context-dependent MFR modulation, we employed a Bayesian optimal inference model in which the MFR is generated as a compensatory response to the self-motion estimated from visual motion (see Data Analysis for details). The MFR modulation results obtained in the experiments only tested the hypothesis with qualitative causal assumptions, but by using the model, the linkage between self-motion and visual motion (i.e., likelihood of self-motion given a visual motion) was inversely and quantitatively estimated. Figure 4A explains the possible effects of the postural- and visual-instability contexts on the prior and likelihood functions of self-motion representation, all of which are modeled by Gaussian probability distributions. We hypothesized that the unstable postural (UP) context would induce a variance increase in the postural prior with zero mean (dashed red curve), as examined in the previous study.⁴⁵ This leads to a rightward shift in the posterior estimate of self-motion from retinal motion signal, $P\left(V_s|V_r\right)$, (solid red curve from the solid green curve). But such a shift could be also caused by decreasing variance of the likelihood function, ${\sigma }_{r|s}^2$, (dotted red curve). On the other hand, it was hypothesized that the visual-instability context could increase the variance of the likelihood of self-motion given observation of visual motion, ${\sigma }_{r|s}^2$, (dotted blue curve) because the retinal motion caused by the random visual motion is irrelevant to the self-motion, resulting in a leftward shift of the posterior estimate of self-motion (solid blue curve from the solid green curve), but this could also happen through a decrease in the self-motion prior variance, ${\sigma }_s^2$, (dashed blue curve).

Figure 4.

Bayesian inference model of self-motion estimated from sudden visual motion applied in the test phase, and estimated likelihood-variance ratios using empirically obtained MFR

(A) Illustration explaining possible effects of the unstable posture (UP: red) and random visual motion (RV: blue) contexts on the prior and likelihood probability distribution functions. Green represents the control context (SP + SV). The UP context possibly increases the variance of the self-motion prior function (dashed curve) and/or decreases the variance of the likelihood function (dotted curve), both of which make the posterior function (solid curve) shift rightward (estimated velocity increase). The RV context possibly increases the variance of the likelihood function and/or decreases the variance of the prior function.

(B) Upper panels: Trial frequency histograms of acceleration deviations caused by applying rightward (pink bar) and leftward (yellow bar) visual motions. Triangles indicate the means of the corresponding acceleration distributions. Lower panels: The distribution functions of MFR obtained by a resampling method (See Data Analysis for details) for four context conditions (SP + SV, UP + SV, SP + RV, and UP + RV). Red curve in each graph represents the fitted Gaussian distribution. Mean and SD of the distribution are shown in each graph.

(C) Likelihood variance ratios in log-scale for the three contexts (UP + SV: red, SP + RV: blue, UP + RV: cyan) relative to the control (SP + SV) context. Bars show median values across participants, and error bars represent the confidence intervals (95%). Each connected thin line represents the data of an individual participant. Single and double asterisks: significance at p < 0.05 and p < 0.01, respectively.

To disambiguate the source of the MFR modulation, we quantitatively examined changes in the likelihood function on the basis of a Bayesian inference model (See Equations (Equation 1), (Equation 2), (Equation 3), (Equation 4) in Data Analysis). The lower panels of Figure 4B show an example of distributions of the MFR amplitudes for the four context conditions, obtained by randomly resampling of empirical observation of acceleration deviations (shown as histograms in the upper panels of Figure 4B) in the rightward visual motion trials (pink bars) and in the leftward visual motion trials (yellow bars) of a particular participant (See Data Analysis in Method details). The mean and SD of the distribution varied across the conditions: mean increased for the UP + SV context, mean decreased for the SP + RV context, and SD increased for the SP + RV, all compared to those for the SP + SV context, in this participant. By using the mean and variance of MFR estimates of all participants with Equation 4 which was derived by the Bayesian inference model of self-motion (See Data Analysis in Method details), we quantified the ratio of the likelihood variance of self-motion, ${\sigma }_{r|s\_B}^2/{\sigma }_{r|s\_A}^2$, for each context condition relative to the control condition (SP + SV context). As shown in Figure 4C, the likelihood variance ratio significantly (t(15) = 2.55, p = 0.022, Effect size dz = 0.637) increased (i.e., over zero in log-scale) for the SP + RV context, while it was not significantly different for the UP + SV context (t(15) = −2.047, p = 0.058) from the control (SP + SV) condition. Interestingly, the likelihood variance ratio did not increase (t(15) = −1.035, p = 0.317) for the UP + RV context even though the visual motion experienced in the context phase might have been the same with or even stronger than that in the SP + RV context, because of the greater body fluctuation (See Figure 3B). Considering a part of retinal motion was actually induced by the body fluctuation even in the UP + RV context, the result suggests that a partial consistency between the actual visual motion and postural fluctuation prevents to increase in the variance of likelihood of self-motion. A one-way ANOVA showed a significant difference (F(2,15) = 13.31, p = 0.0000725, Partial η² = 0.4702, Effect size f = 0.942), and a multiple comparison revealed significant differences in the log-likelihood variance ratio between the UP + SV and SP + RV contexts (p = 0.0028) and between the SP + RV and UP + RV contexts (p = 0.0085). These differences were also confirmed by the permutation tests between the UP + SV and SP + RV contexts (permutation p = 0.00006) and between the SP + RV and UP + RV contexts (permutation p = 0.0014). The Bayesian model using the experimentally observed changes in the response amplitudes and variances, therefore, successfully estimated the increase in the variance of the likelihood probability for the SP + RV contexts. Additionally, considering the increase in the MFR amplitude for the UP + SV context (see Figure 3D), the insignificant change in the likelihood-variance for the UP + SV contexts (red bar in Figure 4C) suggests an increase in the variance of the prior probability of self-motion, by which the hypothesis mentioned above has been justified from the data. These examinations also support the idea that the MFR is generated as a compensatory response to the self-motion estimated from visual motion.

Modulation of voluntary reaction time depending on the postural context

Since, as mentioned in Introduction, both the MFR and voluntary reaction have been shown to exhibit similar susceptibility changes to the various visual stimuli,^41,42 we could expect consistent changes in the reaction time of button press and in the MFR amplitude if changes in signal efficacy inducing response modulation occurred in the common information stream in the brain. In addition, according to the prevailing idea that voluntary action guides the modulation of reflexive action,^23,33,34 it is expected that visual motion sensitivity of voluntary reaction increases after applying the UP context, since the MFR amplitude increased for that context. Although our experimental paradigms for investigating the MFR modulation were already designed to exclude the possible assistive modulation of MFR by the voluntary reaction, it would be interesting to examine whether the contextual modulation in voluntary and reflexive reactions is consistent. To examine this, in the third experiment we investigated the effect of the postural context on the voluntary button press reaction of motion-direction discrimination. If the stimulus sensitivity increased for the UP context, we should expect that the reaction time (RT) would decrease in the voluntary reaction because of faster evidence accumulation for decision making.⁴⁶ To quantify the sensitivity change in this voluntary response, RT was measured during the test phase after applying each of the SP and UP contexts, by means of a task in which participants were asked to press one of the buttons as quickly as possible to determine the direction of a given visual motion. (See Experiment 3 in Method details).

Figure 5A illustrates the cumulative probability functions of the reaction times (RT) of all participants and shows the means of the reaction times (bar graphs in the inset) for the two postural contexts (SP: bluish-filled bar, UP: pinkish-filled bar). Interestingly, we found a significant effect of postural context on the RT (t(15) = 4.12, p = 0.001, Effect size dz = 1.03). Note that the discrimination error rates were very low and were not significantly different between these contexts (SP: 1.56 ± 1.8%, UP: 1.62 ± 1.70%, t(15) = 0.162, p = 0.873). The RT was prolonged for the UP context compared with the SP context, suggesting that voluntary reaction sensitivity to visual motion actually degraded for the UP context. In contrast, MFR amplitude was significantly greater for the UP context than for the stable one (Paired t-test: t(15) = 2.93, p = 0.01, Effect size dz = 0.73) (figure is not shown), indicating increased sensitivity of the reflexive visuomotor processing, as observed in Experiments 1 and 2. In addition to the opposite directional sensitivity changes in the voluntary and reflexive reactions, we did not find a significant relationship (r = −0.07, p = 0.8) between the changes of MFR (UP – SP contexts) and of RT (UP – SP contexts) among participants as shown in Figure 5B. These results therefore clearly rule out the possibility that the MFR modulation associated with postural context is mediated by the regulation of the voluntary reaction.

Figure 5.

Voluntary reaction performances after postural and visual contexts

(A) Cumulative probability functions of reaction times (RTs) in the button-press task for motion-direction discrimination after imposing contexts of stable posture (SP) and unstable posture (UP) in Experiment 3. The inset of the bar graph compares mean RTs of all participants for the SP and UP contexts. Error bars indicate SE and thin gray lines indicate each participant.

(B) Relationship between changes of MFR and of RT by the context difference (UP – SP) across participants. Each circle represents each participant. Many participants exhibited a susceptibility increase in MFR (amplitude increase) while exhibiting a susceptibility decrease in voluntary reaction task (RT increase), but a significant correlation was not found among participants.

(C) Cumulative probability functions of RTs in the button-press task after imposing contexts of the static visual pattern (SV) and random visual motion (RV) in Experiment 4. The inset of the bar graph compares mean RTs of all participants for the SV and RV contexts. Error bars indicate standard error, and thin gray lines indicate each participant. Double asterisks indicate significance at p < 0.01.

However, one might question why the voluntary reaction sensitivity was degraded by imposing the UP context. As known well and explained by a sensory integration theory,^29,45 visual motion can be attributed to self-motion or to external motion. Since the participants were required to indicate the direction of external visual motion on the screen in the reaction task and the UP context could have strengthened the attribution of visual motion to self-motion, the voluntary-reaction sensitivity may have decreased for the UP context. If this is the case, the voluntary reaction would be conversely improved by the RV context. We therefore additionally investigated the effect of the RV context on the performance of the direction discrimination task. The experimental paradigm of the discrimination task was identical to the task in the third experiment except for the applied context (SV and RV in Experiment 4). As shown in Figure 5C, the cumulative probability function of the reaction times (RT) (all participants) clearly shifted rightward for the RV context (pinkish curve) and the RT for the RV context (pinkish-filled bar in the inset graph) was significantly longer than that for the SV context (t(15) = 3.89, p = 0.001, Effect size dz = 0.94), without any difference in error rate for these two contexts (SV: 1.41 ± 1.26%, RV: 1.63 ± 1.52%, t(15) = 0.51, p = 0.61). Thus, the voluntary reactions were similarly modulated (RT increased) by imposing the RV and UP contexts while the MFR was differently modulated by the RV and UP contexts. These results, therefore, indicate that the brain processing does not link the sensory information and contextual information in terms of self-motion and external motion as a way to improve the performance of voluntary reactions while it does so specifically for the reflexive manual response, MFR.

Discussion

Dexterous limb movements are realized by hierarchical motor control mechanisms which incorporate apparently distinct forms of information processing: e.g., voluntary and reflex mechanisms. A series of experiments in this study demonstrated a context-dependent modulation of reflexive reactions of the manual movement (MFR), which was affected by postural and visual instability, without any assistance by voluntary adjustment. Importantly, the result also suggests that the prior (memorized) context information can be used to modulate the MFR while previous studies^47,48 examined the effect of current postural contexts (sitting and standing). Further, in addition to the experimental paradigm for excluding the chance for voluntary reaction to guide the MFR modulation, voluntary reaction did not show a similar context-dependent modulation, indicating a dissociation between reflexive and voluntary reactions with respect to the visuomotor sensitivity change. Here we discuss the computational mechanism behind the modulation of the reflexive visuomotor reaction and the possible neural substrates for realizing the modulation.

As many examples mentioned in Introduction, task- and environment-dependent modulations of reflexive responses are widely believed to be guided by or associated with subsequent voluntary motor actions. This can be regarded as a shared modulation mechanism for generating motor commands for voluntary and reflexive reactions, as schematically shown in Figure 6A. Note that the difference in adaptation rates of voluntary and reflexive reactions is omitted here for simplicity. In this model, context estimation governed by the voluntary reaction mechanism modulates the shared stimulus sensitivity, w, to fulfill the task goal. If this were the case for the MFR, consistent modulations should be observed for the voluntary and reflexive reactions to the visual motion, as have been observed in many previous studies.^33,34,35,36 Indeed, considering the previous studies^41,42 demonstrating consistent susceptibility changes in voluntary reaction time, in MFR, and in magnetoencephalography to various visual stimuli, one would expect shorter reaction time when greater MFR amplitude is produced. However, we did not observe such consistent modulations in voluntary reaction and MFR for the unstable postural context, as shown in the experiments. The MFR modulation, therefore, cannot be explained by this shared sensitivity model.

Figure 6.

Possible schematic models for explaining modulation mechanisms for reflexive reaction (MFR) and voluntary reaction (direction discrimination) to visual motion applied in the test phase

(A) Shared sensitivity model. Shared sensitivity, $w$, from visual motion to voluntary and reflexive (MFR) reactions, is regulated by the voluntary action system (marked by pink color).

(B) Antagonistic motion coding model. Visual sensitivity, $w_s$, is adjusted by the context estimation so as to code the self-motion component, ${\hat{V}}_s$, which generates reflexive reaction (MFR). In this model, this adjustment process is governed by the voluntary action system (marked by pink color) because the external motion component, ${\hat{V}}_e$, is calculated by using the estimated self-motion and the visual motion, for generating voluntary reaction.

(C) Independent context estimation model. While the context estimation for the MFR modulates the sensitivity, $w_s$, depending on contexts of self-motion estimated using visual, vestibular, and somatosensory signals, the context estimation for the voluntary reaction modulates the sensitivity, $w_d$, using only visual information without considering vestibular and somatosensory information.

In line with the computational model of the optimal sensory integration in the brain,^{27,28,31,45,49} one could, instead, imagine that the dissociation of the modulations of the MFR and of voluntary reactions is due to differences in the interpretation of visual motion signals in these forms of visuomotor processing, as depicted in the schematic diagram of Figure 6B. As is well known, the visual motion signal does not solely represent external (object) motion, but is also caused by self-motion. External motion (given by the grating pattern on the screen in this experiment), ${\hat{V}}_e$, therefore, should be estimated by subtracting the self-motion component, ${\hat{V}}_s$, from the visual (retinal) motion, $V_r$.

Previous studies showed that visual motion induced by postural fluctuation greatly improves postural stability^50,51 and that visual motion itself induces self-motion perception.^49,52,53 In addition to those contributions of visual motion to controlling posture and to self-motion perception shown in previous studies, the current study has clarified the remote effect of self-motion estimates on reflexive manual movements, using the Bayesian optimal estimation model. This model suggests that the contribution of visual motion to estimating self-motion, which drives the MFR, is enhanced by the unstable postural context. Indeed, it has been known that an increased postural variability enhances the dependence of postural control on visual information.^45,54 On the other hand, the sensitivity decrease in the MFR caused by applying a random visual motion context can also be predicted by the optimal estimation. The random visual motion which is inconsistent with postural fluctuations would decrease the contribution of sensory evidence to the self-motion estimation. This corresponds to the increased likelihood-variance of self-motion in the model, which has been confirmed in our analysis (SP + RV in Figure 4C).

If this optimal estimation is also governed by the voluntary reaction mechanism for rationally interpreting the visual motion signal, as reported in successful multimodal integration in self-motion perception,^29,49,55,56 one could further expect a decrease in reaction time (i.e., increased sensitivity) in the direction discrimination task by applying the random visual motion (RV) context. This is because this context could decrease the likelihood-variance in estimating external motion, as an antagonistic process against the increased likelihood-variance in estimating self-motion that was confirmed in Experiment 2. However, as observed in experiment 4, the latency of voluntary reactions was prolonged by imposing a random visual motion context. Therefore, the context estimation for determining the sensitivity of the voluntary reaction should be dissociated from the one for determining the sensitivity of the reflexive reaction, MFR, as depicted in the schematic diagram of Figure 6C. The decreased sensitivity of the voluntary reaction, observed in both of the contexts of unstable posture and random visual motion respectively in Experiments 3 and 4, could be ascribed to the process of making a reliable reaction to the visual motion direction without considering different forms of prior contextual information (i.e., visual motion caused by self-motion and visual motion caused by external motion). In contrast, the reflexive reaction mechanism considers such prior contexts by accessing a functional representation of self-motion that is estimated from multisensory and prior information, as depicted in Figure 6C.

The computational mechanism of the quick manual response to the background motion has remained controversial. One view is that visual motion induces an illusory shift of the internal position representation of the reaching target, which leads to rapid corrective movements.^19,20,57 However, recent evidence⁵⁸ has shown that the effect of visual motion on manual reaching via an illusory shift of the visual target appears slower than the direct visual motion effect, suggesting that this account does not fit to explain the MFR generation process. An alternative view is that the MFR makes compensatory reactions against self-body movements which frequently induce visual motion.^21,22,38,40 Since the surrounding visual background, whose spatial distance is frequently closer than the gaze distance, moves on the retina in the opposite direction to self-motion, an automatic and rapid visuomotor mechanism using surrounding motion would have great advantages for quickly adjusting reaching movements.⁴⁰ The latter account is supported by the current study showing the modulation of the MFR by the contexts of postural and visual instability. Namely, the increased MFR caused by imposing unstable posture contexts in Experiments 1, 2, and 3 can be rationally explained by the estimation of faster self-motion under the Bayesian inference framework, while it cannot be explained by the effect of the estimated external motion because a posterior estimate of external motion would decrease with postural instability. Since the manual response is greater for visual than for vestibular stimuli both of which produced similar postural responses,⁵⁹ it is likely that the visual signal is more weighted than the vestibular signal in self-motion estimation for controlling hand movements.

Determining the neural substrates driving and modulating the MFR remains a challenge. Nevertheless, behavioral,^22,37 imaging,⁴¹ and physiological studies^60,61 have suggested the involvement of the middle temporal (MT) and medial superior temporal (MST) areas in the visual motion analysis for the MFR. While the MST, which also receives vestibular inputs,^62,63,64,65 is widely known to be an extrastriate visual area sensitive to large-field optic flow related to self-motion^{66,67,68,69,70,71,72} in addition to the occipital area V3 and occipito-parietal areas V6,^73,74,75 recent studies have demonstrated that it contributes to self-motion perception by combining visual and vestibular signals.^29,49,55,56 Since our model explaining the context effect on MFR modulation is consistent with the optimal cue integration theory for self-motion, the MST and related cortical area are possibly involved in modulating the MFR by representing self-motion from visual motion. To clarify the neural substrates for the MFR modulation, it would be needed to explore how the postural and visual instability contexts alter the neural representation related to self-motion for manual control.

In contrast to the prevailing hypothesis of reflex modulation guided by voluntary action, this study clearly showed that the visuomotor reflexes during manual movements modulated by unstable postural context are not explained by a subsequent goal-directed voluntary reaction because no voluntary reaction during reaching movement in our experiment acted to increase the MFR. Theoretical consideration suggests that the reflex modulation is explained by a change in the self-motion estimation affected by the body and visual instability contexts. Even more surprisingly, it transpires that a reflexive reaction can respond more smartly to the contextual information of the body-environment interaction than a voluntary reaction. While high-level processing in the sensorimotor hierarchy, such as perception, cognition, and decision making, has frequently been explained through Bayesian inference,^{24,27,28,31,32,63} reflexive sensorimotor control has been rarely explained by such a mechanism without a contribution from the voluntary reaction. The experimental evidence shown here implies that the reflex mechanism in the hierarchical sensorimotor control is not always subservient to the subsequent voluntary reaction. This class of reflex modulation mechanisms may implicitly elicit useful knowledge of remote states, including information relating to the uncertainty or variation in those states, by various forms of interaction between our body and dynamic environments, even without the explicit guidance of voluntary action.

Limitations of the study

Our findings indicate that the MFR is modulated by the preceding contextual situations of postural and visual instabilities without any assistance of voluntary reaching adjustments to the visual stimuli. Additionally, the postural-dependent MFR modulation, which was inconsistent with the voluntary reactions (button press) to the visual motion direction, suggests its unique information processing. However, we do not know yet whether and how the postural contexts affect the additional component specifically in reaching adjustment.^14,76,77,78 Apart from the MFR modulation, it would be worth examining in the future study whether such component has similar modulation specificity with that of MFR or with that of slow voluntary reaction.

To further understand how the MFR modulation is formed, its underlying hierarchical computational and neural mechanisms accessing the information of postural and surrounding-visual stabilities should be investigated. By examining similarities and differences among the voluntary and reflexive reactions to the various stimuli, we would be able to examine the processing and interaction between those information streams. In addition, while the modulation of quick manual response to a large-field visual motion has been focused in the current study, future studies should address the modulations of manual responses to the various visual stimuli, in order to disentangle the interwoven functions in hierarchical and parallel brain processing for interacting with dynamic environments.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Software and algorithms
MATLAB (R2017, R2019a)	MathWorks	RRID: SCR_001622
Cogent Graphics	Wellcome Department of Imaging Neuroscience, University College London, London, UK
Nexus 1.8 with VICON MX13	Peak Performance Technologies	PRID: SCR_015001

Resource availability

Lead contact

Further information and requests for resources should be directed to the Lead Contact, Hiroaki Gomi (hiroaki.gomi.ga@hco.ntt.co.jp).

Materials availability

This study did not generate new unique reagents.

Experimental model and subject details

Four experiments were conducted in this study. Eighteen (3 males, mean age ± SD: 40 ± 7.2), sixteen (5 males, mean age ± SD: 37.8 ± 8.7), sixteen (7 males, mean age ± SD: 39.6 ± 6.8) and sixteen (10 males, mean age ± SD: 27.8 ± 9.1) healthy right-handed naive participants took part in the first to fourth experiments, respectively. The sample sizes in Experiments 2, 3, and 4 were a priori decided on the basis of a power analysis of Experiment 1 with α = 0.05 and power = 0.80. All gave informed consent regarding participation in the study. Experiments were undertaken with the understanding and written consent of each participant in accordance with the Code of Ethics of the Declaration of Helsinki, and with the NTT Communication Science Laboratories Research Ethics Committee approval.

Method details

Experimental apparatus

Each participant performed reaching movements while standing on a wooden stable platform or a motion platform (Thrive FD-003, Daito Electric) in front of a vertical screen (eye-monitor distance: 50 cm). Visual stimuli were generated using MATLAB (Mathworks) on a computer (OS: Microsoft Windows7) with a graphic toolbox, Cogent Graphics (Wellcome Department of Imaging Neuroscience, University College London, London, UK), and were back-projected by a projector (NEC WT610; NEC Viewtechnology) onto the screen (RHY-130; Kikuchi Science Laboratory) with a vertical refresh rate of 60 Hz (Mean luminance at the center of the screen: 57 cd/m²). As illustrated in Figure 1A, vertical sine-wave gratings (spatial frequency: 0.1 cycle/degree, contrast: 50%) were presented as a visual stimulus (width: 84 cm, height: 65 cm). A pair of grating patterns (anti-phase) was shown on the top and bottom sides of the screen, which were separated by a gray zone (vertical width: 4.5 cm). The participant’s eye height was approximately aligned to the center of the gray zone (i.e. center of screen). Two photo-diodes (S1223-1; Hamamatsu Photonics, Shizuoka, Japan) were attached to the screen to detect the exact onset timing of the visual stimuli, the signals of which were recorded at 2 kHz. A button switch connected to the computer’s parallel port was used to detect the initiation of reaching movements. The button was placed approximately 10 cm in front of the participant’s body, and that position was at [x, y, z] = [0, −36, −44.5] cm with respect to the center of the screen. Here, x was the horizontal direction parallel to the screen, y was the direction perpendicular to the screen, and z was the vertical direction. In the discrimination reaction task (Experiments 3 and 4), a two-button box, which was connected to the computer, was placed at the same position.

Reflective markers for a motion capture system (VICON MX13; Peak Performance Technologies) were attached to the back of the head, the right and left shoulder, the bottom of the right index finger, and the back of the body (around the upper parts of the thoracic vertebrae). The head marker was used to confirm that the head posture was roughly kept constant during the experiments. In addition to these body segments, a marker was attached to the motion platform. The position data of these markers were recorded at a sampling rate of 250 Hz.

Experiment 1

This experiment investigated the effects of postural context on MFR modulation. Participants performed a reaching task in the test phase after standing on a wooden stable platform (SP: stable posture context) or on a motion platform (UP: unstable posture context) in the context phase of each trial. Each trial started with the presentation of static visual gratings. In the SP-context trial, participant stood on a wooden platform while looking at the center of screen for 5 s without touching the button. Subsequently, 300 ms after the participant started to press the button with right index finger, a reaching target (red marker) was flashed (for 700 ms) at the center of the screen. After 300 ms from the target elimination, a beeping sound was given to instruct participants to start the reaching movement toward the remembered target location at a relatively moderate speed. Shortly after button release (∼50 ms), in visual motion trials, the background gratings started to move either rightward or leftward at a constant speed (105 deg/s). Note that the two gratings (located on top and bottom on the screen) always started to move simultaneously in the same direction. In non-visual motion trials, gratings were stationary. Regardless of the application of visual motion, participants were asked to maintain the reaching toward the original target location.

In the UP-context trials, participants stood on the motion platform (Figure 1A). At the beginning of each trial (context phase), the platform continuously moved in an undulating manner in the x, y, and z directions (Amplitude ∼3 cm with 1.4 Hz in x-dir, ∼2 cm with 2.8 Hz in y-dir, and ∼3 cm with 2.8 Hz in z-dir) for 5 s to destabilize the participant’s posture. Participants were asked to maintain their posture without touching anywhere while looking at the static grating patterns during the context phase. After the context phase, the platform stopped moving, followed by the beep sound to start reaching. The subsequent reaching task in the test phase was identical to the stable condition.

Three types of visual stimuli (rightward motion, leftward motion, or stationary), each with 9 trials, were randomly presented as one block (27 trials). In each block, postural context (SP or UP) was fixed, and the block of each context was repeated four times with the order counterbalanced within and across participants. Each participant performed 36 trials for each of 6 combinations of postural context and visual motion (total 216 trials).

Experiment 2

To investigate the prior context contributions of visual and other sensory information related to the postural stabilities to the MFR modulation observed in the test phase, we combined visual context with postural context in a factorial design as shown in Figure 3A. Postural context (SP or UP) was identical as Experiment 1. As for the visual condition, background gratings were kept stationary in the static visual pattern (SV) context (top two panels in Figure 3A) or horizontally moved while switching the direction of motion randomly to the right or left in the random visual motion (RV) context (bottom two panels in Figure 3A). The random visual motion was generated using the velocity profile of the phase-randomized head-motion acquired in the motion-platform condition of Experiment 1. More specifically, we applied Fast Fourier Transformation (FFT) to the nine patterns of head-velocity in x-direction obtained in Exp. 1, and then reconstruct the velocity patterns by the inverse FFT after phase randomization. Those velocity patterns were magnified 10-fold so that the visual motion velocity given in the test phase was in the range of the random visual motion velocity given in the context phase. Mean and SD of the velocity of random visual motion was 0.1 ± 60.2 deg/s (min: −137 deg/s; max 143 deg/s). As a result, random visual motion did not synchronize with the body motion caused by the motion platform in this experiment, and it disturbed postural stability during the context phase. During the context phase (5 s), participants were required to maintain their posture. The motion platform was stabilized after the UP + SV and UP + RV context phases. After all context phases, the static grating pattern was shown on the screen. Subsequently, a beeping sound was given to initiate the reaching, and then visual motion was applied to induce MFR (See Figure 3A). Task and visual stimulus in the test phase were identical to those in Experiment 1. Context was fixed throughout a given block (27 trials), and each context was repeated four times (with the order randomized and counterbalanced within and across participants), resulting in a total of 432 trials.

Experiment 3

As previous studies demonstrated^34,36,79 that the reflex response amplitude increases as the voluntary reaction time (RT) decreases, RT is an important measure to examine the reflex modulation mechanism. In addition, the MFR amplitude is well correlated with the brain response measured by magnetoencephalography⁴¹ which is inversely correlated with RT.⁴² Based on these experimental observations, one could expect that RT decreases as MFR increases if same information originating from sensory signals is used for generating MFR and voluntary response. Since significant interaction between the effects of visual and postural contexts on the MFR was not found in Experiment 2, we examined the effect of postural context on voluntary reactions of motion-direction discrimination, using the identical postural contexts with Experiment 1. In each trial, after applying the SP or UP context, a fixation marker was presented at the center of screen. Participants were asked to gaze at the marker. Then, the fixation marker disappeared with a beep sound. After a random interval (0.5–1.5 s) from the beep, background gratings started to move rightward or leftward in two-thirds of the trials. In the remaining one-third of the trials, the gratings remained stationary. Participants were asked to judge the direction of visual motion by pressing buttons on their left or right side as quickly and accurately as possible. In the no-visual-motion trials, they were asked not to press the button. If the response was greatly delayed (>700 ms), the trial was rejected and repeated later. Reaction time (RT) was quantified as the duration between stimulus onset (detected by the photo-cell) and button press onset. One block consisted of 27 trials, and the condition of visual motion (rightward, leftward, or stationary; 9 trials for each) was randomly selected in each trial. Postural context (SP or UP) was fixed during a block, and context was repeated four times with the order counterbalanced (total of 216 trials) in each participant. To compare the sensitivity changes in voluntary and reflexive reactions of the same participants, each participant also performed the reaching task after the postural context (SP or UP) to evaluate MFR, whose procedure was identical to Experiment 1. The order of these two tasks (voluntary reaction task and reaching task) was counterbalanced across participants.

Experiment 4

The effect of visual context (SV or RV) with the SP context on voluntary reactions of motion-direction discrimination was examined. The experimental paradigm was identical with the discrimination reaction task in Experiment 3, except for the context. Participants conducted the discrimination reaction task with the order of context (SV or RV) counterbalanced as done in Experiment 3.

Quantification and statistical analysis

Motion-induced manual following response (MFR)

All captured position data were filtered (fourth-order Butterworth low-pass filter with a cutoff of 30 Hz), and the velocity and acceleration patterns were calculated using three- and five-point numerical time differentiations of the filtered position data, respectively. Data were temporally aligned with respect to the initiation of visual motion detected by the photo-diodes. For each condition, outlier trials in which the hand motion profile largely differed from the corresponding median profile were excluded. In detail, outlier trials were detected by using the velocity deviation (±2SD) from the median pattern on each axis and the movement-time deviation detected by a standard interquartile method. The actual outlier percentages were 5.6 and 5.3% in the two conditions in Exp. 1, and 5.3–7.1% in the four conditions in Exp. 2. In the reaching experiment in Exp.3, outlier percentages were 5.1 and 4.9% in the two conditions. In all experiments, any difference of outliner percentages among conditions were not found. Those outlier trials were removed from subsequent analyses.

To evaluate the rapid manual responses induced by visual motion (MFR), we took the difference between the mean x-directional acceleration patterns of the hand movements in response to rightward and leftward visual motion. To quantify the amplitude of rapid (reflex) responses, we computed the temporal average of the differential pattern during a period of 120–160 ms after the visual motion onset, as done previously.³⁷ The latencies of the MFR, relative to the visual motion onset, were determined statistically from hand-acceleration patterns. We performed a running independent t-test between the responses to rightward and leftward visual motion under the assumption of random trial-by-trial fluctuation, and the response onset was defined by the first time point when the p value of the test was continuously less than 0.01 for 40 ms. We also analyzed the lateral body sway induced by visual motion during the reaching movements. An identical analysis as for the MFR was performed on the acceleration patterns of the back of the body. The amplitude of the body response was quantified in exactly the same manner as for the MFR.

Postural stability

In Experiment 2, postural stability was assessed in the context phase, during which there was platform motion and/or random visual motion (Figure 1B). The root-mean-square (RMS) of the lateral position of the marker attached to the back of the body (body marker) was computed for each participant in order to examine the relationship between the body fluctuation in the context phase and the amplitude of the manual response in the test phase. Note that the RMS of the body marker was evaluated for 15 out of 16 participants because of detection errors (marker occlusion by hair shaking due to platform motion) of the body marker of one participant.

Reaction time in voluntary task

To examine the context dependency of the reaction time in the direction discrimination tasks (Experiments 3 and 4), the mean reaction time for each participant and the cumulative distribution function of the reaction times of all participants were computed for each context condition. Note that the mean reaction time of one of the participants in the random visual condition was obtained from 54 trials (3 out of 4 blocks) because of the loss of one block of data, while the mean of the other participants was obtained from 72 trials.

Estimation of likelihood variance in Bayesian optimal formulation

In examining the account of the information processing for generating MFR, we assumed that the MFR is produced in proportion of the self-motion estimated from the visual motion stimulus applied in the test phase. We consider an observation model acquired during the context phase, $P\left(V_r|V_s\right)$, to represent the uncertainty of the visual (retinal) motion velocity $V_r$ depending on the self-motion velocity $V_s$, and used the following Bayesian formulation to estimate the self-motion velocity from the visual motion velocity.

$$P\left(V_s|V_r\right)=\frac{P\left(V_r|V_s\right)\cdot P\left(V_s\right)}{P\left(V_r\right)}\text{.}$$ (Equation 1)

Here, $P\left(V_s|V_r\right)$ denotes the posterior probability of self-motion velocity, $V_s$, given the visual (retinal) motion velocity, $V_r$. $P\left(V_s\right)$ denotes the prior probability of self-motion velocity representing postural fluctuation, and $P\left(V_r|V_s\right)$ gives the likelihood of self-motion velocity, $V_s$, given observation of visual motion velocity, $V_r$. Representing the means and variances of the prior, likelihood, and posterior functions as $E\left[P\left(V_s\right)\right]={\mu }_s$, $Var\left[P\left(V_s\right)\right]={\sigma }_s^2$, $E\left[P\left(V_r|V_s\right)\right]={\mu }_{r|s}$, $Var\left[P\left(V_r|V_s\right)\right]={\sigma }_{r|s}^2$, $E\left[P\left(V_s|V_r\right)\right]={\mu }_{s|r}$, and $Var\left[P\left(V_s|V_r\right)\right]={\sigma }_{s|r}^2$ and supposing an optimal estimation with a Gaussian distribution for each probability and zero mean for the prior distribution (${\mu }_s=0$), the mean of the self-motion estimate and its variance are expressed as follows:

$${\mu }_{s|r}=\frac{{\sigma }_s^2}{{\sigma }_s^2+{\sigma }_{r|s}^2}{\mu }_{r|s},{\sigma }_{s|r}^2=\frac{{\sigma }_s^2\cdot {\sigma }_{r|s}^2}{{\sigma }_s^2+{\sigma }_{r|s}^2}\text{.}$$ (Equation 2)

Therefore, the mean of self-motion estimated from the visual motion, ${\mu }_{s|r}$, varies according to the internal representation (prior knowledge) of the postural fluctuation ${\sigma }_s^2$ and the uncertainties in the visual motion given self-motion ${\sigma }_{r|s}^2$.

By assuming that the MFR is proportionally driven by the self-motion estimated from the visual motion in the test phase. Namely, the mean and trial variance of MFR are expressed as

$$M=g{\mu }_{s|r},{\sigma }_M^2=g^2{\sigma }_{s|r}^2\text{.}$$ (Equation 3)

For two different contexts, A and B, with the same visual motion stimulus in the test phase, the variance ratio of the visual observation functions, ${\sigma }_{r|s\_B}^2/{\sigma }_{r|s\_A}^2$, can be expressed using Equations 2 and 3 as follows.

$$\frac{{\sigma }_{r|s\_B}^2}{{\sigma }_{r|s\_A}^2}=\frac{M_A}{M_B}\frac{{\sigma }_{M\_B}^2}{{\sigma }_{M\_A}^2}\text{.}$$ (Equation 4)

Therefore, the ratio of the variances of the visual observation model, ${\sigma }_{r|s\_B}^2/{\sigma }_{r|s\_A}^2$, can be calculated from the experimentally observed manual responses.

The trial variance, ${\sigma }_M$, and mean, $M$, of the MFR amplitudes of each participant for each context were calculated using MFR datasets created by resampling (10000 samples) the acceleration profiles in random trial pairs with the rightward and leftward visual-motion stimuli for the corresponding context. In details, we first prepared acceleration deviations (mean for 120–160 ms after the visual motion onset) in the visual motion trials by subtracting the mean acceleration of all no-visual motion trials. The trial histograms of the acceleration deviations of the rightward visual motion trials (pink bars) and of the leftward visual motion trials (yellow bars) for the four different contexts of the particular participant are shown in the upper panels of Figure 4B. Then, acceleration difference, which corresponds to a putative MFR of a single trial, was obtained using acceleration deviation of one trial randomly selected from the rightward-visual-motion trials and acceleration deviation of one trial randomly selected from the leftward-visual-motion trials. This random trial selection was repeated 10000 times for each context in order to estimate the trial variability of MFR. Note that, since MFR could not be experimentally quantified using a single trial, this resampling method was hired to obtain the MFR variance across trials. The lower panels of Figure 4B show the estimated empirical distribution functions of MFR for the four different contexts of the corresponding participant. These distribution functions of MFR were obtained for each participant. Then, the median of the variance ratio, ${\sigma }_{r|s\_B}^2/{\sigma }_{r|s\_A}^2$, was obtained from the empirical distribution functions for each participant. By applying this calculation, we compared log likelihood-variance ratios of the all participants (n = 16) for the three contexts (UP + SV, SP + RV, UP + RV) relative to the context of SP + SV, as shown in Figure 4C.

Statistical analysis

Paired t-tests were used to statistically compare data for different contexts in Experiments 1, 3, and 4. In Experiment 2, a two-way repeated measures ANOVA was conducted on the postural context (stable or unstable) and visual context (static or random visual motion). A power analysis was conducted for each significant test result by using a G∗Power 3.1.⁸⁰ A one-sample t-test and one-way ANOVA with post-hoc multiple comparison (Scheffe’s method) were used to evaluate variance ratios of the log likelihood in Experiment 2. We also conducted permutation tests to examine the differences between the variance rations among conditions by randomly swapping the ratio data among conditions (100000 times).

Published: January 20, 2023

Data and code availability

• •Data is available from the lead contact author upon request.
• •For providing the visual stimuli, we used Cogent Graphics library in MATLAB (version R2017). For data analysis, we used MATLAB (R2017, R2019a).
• •Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon reasonable request.