No enhancement of vestibular stimulation on visual working memory for actions

Abstract

Holding and processing actions in visual working memory (VWM) is crucial for daily functioning, but it is capacity-limited. The vestibular system, which is activated by both vestibular signals and visual gravitational motion, may be involved in this cognitive process. Past research indicates that galvanic vestibular stimulation (GVS) is a non-invasive technique that can enhance motor functions and various cognitive functions. However, the impact of GVS on VWM for actions has not been explored. This study addresses this gap by investigating whether GVS can improve VWM capacity for both upright and inverted actions. Using a virtual reality platform, we assessed VWM capacity in a change detection task, applying either STIM-current (0.8 mA) or SHAM-current (0 mA) GVS. The results showed no significant effect of GVS on VWM for either upright or inverted actions, at least for non-noisy GVS with stimulation intensity below cutaneous threshold. Despite the absence of significant effects, this study provides valuable insights into the interactions between the vestibular system and cognitive processes, laying the groundwork for future research on GVS applications in cognitive enhancement.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-024-80678-7.

Introduction

Visual working memory (VWM) for actions, the cognitive ability to temporarily store and manipulate information about the movements of living beings, is crucial for our daily activities¹, including social interactions, motor learning, and other dynamic tasks^2,3. The capacity of VWM for actions is not only correlated with core cognitive abilities, such as fluid intelligence⁴, but also closely related to critical social traits, including theory of mind and empathy^5–7. However, VWM has a limited capacity, retaining up to six actions in real-world scenarios⁸. Therefore, it is essential to explore methods that can enhance the VWM capacity of actions.

The vestibular system, traditionally recognized for monitoring linear and rotational head accelerations and gravitational forces^9,10, plays a crucial role in processing visual actions inherently constrained by the physical laws of gravity¹¹. It estimates the effects of gravity on visual motion by integrating multisensory information from vestibular organs and vision, which assists in the processing of visual gravitational motion^12–16. This processing is influenced by vestibular input signals, as the core regions of the vestibular cortex responding to visual gravitational motion^12–14 overlap with the brain regions activated by direct vestibular stimuli^12,17–21. For instance, changes in vestibular inputs under microgravity conditions modulate visual action processing through the reweighting of sensory inputs¹⁴. Moreover, gravity cues from visual signals are instrumental for the interpretation of actions¹³. When actions are inverted, they mismatch with gravity-constrained motion patterns, leading to severely impaired processing^14,22. Such visual motion that are unrelated to natural gravity result in less involvement of the vestibular network^12–14. These findings collectively underscore the critical role of the vestibular system in processing actions that are coherent with natural gravity. Consequently, it is posited that stimulating the vestibular system might modulate VWM for actions.

Galvanic vestibular stimulation (GVS), a non-invasive brain stimulation technique used to enhance cognitive functions through stimulating relevant brain regions (for reviews see^23–25), might be effective for enhancing VWM for actions. This technique delivers small currents to the vestibular organs via electrodes placed over the mastoid processes (for reviews see^18,19,26). Previous studies have demonstrated that GVS can enhance various cognitive functions, including face perception²⁷, spatial attention^28–33, action recognition³⁴, spatial memory^35–38, and visual object memory^39,40, by activating the associated brain regions. For example, GVS has been shown to enhance spatial memory by activating regions involved in spatial navigation, including the hippocampus and striatal regions. Human neuroimaging studies using GVS have shown that vestibular input activates the vestibular system^17–21, which also responds to visual gravitational motion^12–15. Given the vestibular system’s role in action processing, it is plausible that GVS could also enhance memory for actions.

This study aimed to investigate the potential of GVS to enhance VWM storage for actions. We selected a sinusoidal alternating current with an amplitude of 0.8 mA for GVS, as previous studies have suggested that low currents below 1 mA do not induce vestibular dysfunction but have the potential to improve cognition^38,39,41,42. A change detection task was utilized via virtual reality (VR) equipment to assess the VWM capacity in a high ecological-validity environment, with either STIM-current (0.8 mA) or SHAM-current (0 mA) GVS applied to healthy participants during the task. It was initially hypothesized that STIM-current GVS would improve the VWM performance for actions but not for inverted actions. However, the findings contradicted this hypothesis, showing no significant effects of STIM-current GVS on VWM for either upright or inverted actions. These findings highlight the importance of considering various factors when exploring the effects of GVS on cognitive functions, and provide a foundation for further research on GVS in cognitive enhancement.

Results

No effect of STIM-current GVS on the VWM capacity for actions

We initially explored the impact of STIM-current GVS on the VWM storage of actions. In Experiment 1, we examined the VWM capacity of actions with or without GVS using a change detection paradigm via VR equipment (see Fig. 1a,d). Our recent study revealed that at most five to six actions could be retained in VWM⁸. Accordingly, participants were required to memorize either five or seven actions, with one load condition falling within the VWM capacity and the other exceeding it. These two load conditions were randomly presented at the center of the VR screen. A probe was displayed after 1-s presentation of the memory array and participants had to judge whether it appeared in the memory array. Participants completed the task under two GVS conditions (see Fig. 1b,c): without stimulation (SHAM block) or while simultaneously receiving vestibular stimulation (STIM block). During the STIM block, a bilateral bipolar stimulation with a sinusoidal current was used with an amplitude of 0.8 mA and frequency of 0.5 Hz. During the SHAM block, the stimulation current was set to 0 mA. The number of actions stored in VWM was estimated by calculating Cowan’s K⁴³.

Fig. 1.

(a) Schematic illustrations of the trials in blocks with different GVS states in Experiments 1–3. (b) Placement of stimulating electrodes over the left and right mastoid processes. (c) The photograph of the experiment scene. (d) The 3-D human model with different actions is used as stimuli.

A two-way repeated-measures analysis of variance (ANOVA) was conducted on K with set size (five vs. seven) and GVS state (SHAM vs. STIM) as within-subject factors. As shown in Fig. 2a, we found that the K value significantly increased from set size five to seven (F (1, 24) = 11.278, p = 0.003, = 0.320, BF₁₀ = 11.190), suggesting that more actions were stored when retaining seven actions in VWM. However, neither the main effect of GVS state (F (1, 24) = 0.297, p = 0.591, = 0.012, BF₁₀ = 0.318) nor the interaction between set size and GVS state (F (1, 24) = 0.286, p = 0.598, = 0.012, BF₁₀ = 0.328) was significant. These findings suggest that the STIM-current GVS did not influence the VWM storage of actions.

Fig. 2.

The results for Experiment 1. The VWM capacity of actions for each set size of each participant (light-colored dots) and their average results (dark-colored dots) under STIM-current GVS and SHAM-current stimulation for the entire duration of two blocks (a), and for the early and late parts of the two blocks (b). Error bars indicate standard errors of the mean.

Past research has found that stimulation duration could influence the GVS effects^31,44. Considering that the duration of our blocks was relatively long (approximately 14 min per block) and might fall within the influential range, it was necessary to analyze the impact of GVS under shorter stimulation duration. Consequently, we divided the data from each block into two halves and calculated the K values for each participant in the first and second halves respectively. We then conducted the ANOVA on the first and second halves of the two blocks, respectively (see Fig. 2b). For the first half of the two blocks, the K value significantly increased from set size five to seven (F (1, 24) = 7.545, p = 0.011, = 0.239, BF₁₀ = 2.750). Neither the main effect of GVS state (F (1, 24) = 2.221, p = 0.149, = 0.085, BF₁₀ = 0.542) nor the interaction between set size and GVS state (F (1, 24) = 0.115, p = 0.737, = 0.005, BF₁₀ = 0.311) was significant. Similarly, in the second half, the K value significantly increased from set size five to seven (F (1, 24) = 5.251, p = 0.031, = 0.180, BF₁₀ = 1.898). Neither the main effect of GVS state (F (1, 24) = 0.555, p = 0.463, = 0.023, BF₁₀ = 0.378) nor the interaction between set size and GVS state (F (1, 24) = 0.701, p = 0.411, = 0.028, BF₁₀ = 0.424) was significant. These results align with the previous findings, suggesting that STIM-current GVS had no significant effect on the VWM capacity of actions, irrespective of the stimulation duration. To further investigate whether the initial effect of GVS on VWM capacity is transient and dissipates as the duration of stimulation extends, or whether GVS has no impact on VWM capacity at all, we conducted the follow-up experiment.

No effect of STIM-current GVS on the VWM capacity for actions with a shorter stimulation duration

We subsequently investigated whether the STIM-current GVS affects the VWM capacity of actions when applied over a shorter stimulation duration. In Experiment 2, we halved the number of trials in each condition, reducing the duration of each block to half that of Experiment 1. Each block contained 32 trials, lasting approximately 7 min.

As shown in Fig. 3a, we found that the K value significantly increased from set size five to seven (F (1, 19) = 7.012, p = 0.016, = 0.270, BF₁₀ = 3.562). Neither the main effect of GVS state (F (1, 19) = 0.003, p = 0.960, < 0.001, BF₁₀ = 0.327) nor the interaction between set size and GVS state (F (1, 19) = 0.338, p = 0.568, = 0.017, BF₁₀ = 0.320) was significant. These results replicate the findings in Experiment 1, supporting the view that STIM-current GVS did not influence the VWM storage of actions regardless of the stimulation duration.

Fig. 3.

The results for Experiment 2 (a) and Experiment 3 (b). Error bars indicate standard errors of the mean.

No effect of STIM-current GVS on the VWM capacity for inverted actions

Despite the lack of significant effects on VWM for actions in Experiments 1 and 2, we proceeded to Experiment 3 to investigate whether the GVS might influence VWM for inverted actions. This experiment tested the hypothesis that while GVS does not affect memory for upright actions, it might differentially impact the memory of inverted actions, which could pose additional cognitive challenges. Experiment 3 aimed to determine whether the results from the previous experiments would be consistent across different types of action stimuli.

In Experiment 3, we inverted the action stimuli in Experiment 2. As shown in Fig. 3b, we found that neither the main effect of set size (F (1, 19) = 0.518, p = 0.481, = 0.027, BF₁₀ = 0.364), nor the main effect of GVS state (F (1, 19) = 0.086, p = 0.773, = 0.004, BF₁₀ = 0.310) was significant. The interaction between them was not significant (F (1, 19) = 0.159, p = 0.695, = 0.008, BF₁₀ = 0.326). These findings suggest that the STIM-current GVS did not affect the VWM capacity of inverted actions.

Discussion

This study is the first to explore the potential of GVS technology to enhance VWM storage for actions. To assess its feasibility, we conducted experiments in a VR environment, simulating real-world VWM scenarios to increase the ecological validity of our experiments. We applied either STIM-current (0.8 mA) or SHAM-current (0 mA) GVS to healthy participants and assessed their VWM capacity. Contrary to our hypothesis that STIM-current GVS would enhance VWM for actions, the results showed no significant impact on the VWM capacity for either upright (Experiments 1 and 2) or inverted actions (Experiment 3). These findings suggest that galvanic stimulation of the vestibular system via mastoid currents does not influence VWM storage for actions, regardless of the stimulation duration (Experiment 2).

The absence of enhancement in VWM capacity for actions, despite the vestibular system’s known role in processing gravity-constrained actions and the documented cognitive benefits of GVS, suggests several possible explanations. One possibility is that visual gravitational information is not central to VWM for actions. On one hand, gravitational cues may only be important for perception but insufficient for the gain of VWM. VWM involves multiple stages, including encoding and maintenance, each supported by specific neural processes⁴⁵. While vestibular system utilizes gravitational constraints facilitate action perception^13,14, potentially improving VWM encoding, the impact on VWM maintenance remains unclear. Gains in encoding may not persist through later stages of VWM processing, thus not impacting the overall VWM capacity. On the other hand, gravitational information may not be stored in VWM. Existing research suggests that we process actions through two distinct systems: one that rapidly processes the local foot motion which is constrained by gravity; and another that slowly processes the fine global structure-from-motion⁴⁶. It is possible that VWM primarily stores fine configural information without gravitational information, meaning that the vestibular system’s influence on action perception does not necessarily translate to enhanced VWM capacity.

Another potential explanation lies in the GVS stimulation protocol used in our study. While non-noisy GVS has been shown to enhance certain cognitive functions^{27,29,31–33,36,37}, research suggests that noise-enhanced GVS might be more effective^{24,30,35,38,39}. Studies have found that noisy GVS can improve inhibition performance⁴⁷ and accelerate visual memory recall³⁹, whereas non-noisy GVS does not produce similar results^39,42. These findings indicate that the effectiveness of GVS may depend on the inclusion of noise, which operates on the principle of stochastic resonance to enhance sensory processing and perception⁴⁸, thereby positively influencing related cognitive processes. Additionally, individual variability in response to GVS should be considered. We selected a 0.8 mA sinusoidal alternating current based on previous studies^39,42, which generally regard low currents below 1 mA as potentially beneficial for motor and cognitive performance^{38,39,47,49–51}. However, some participants in other studies have shown enhanced cognition with stimulation currents above 1 mA^27–29,31, suggesting that the optimal intensity may vary between individuals⁵². Although the protocol in the current study was effectively administered, with participants reporting a slight sensation of head rotation during the current stimulation without any tingling at the electrode sites, future research could consider enhancing the effects of GVS by adding noise or personalizing the current intensity.

This study has several other limitations that may have contributed to the null results observed. First, we did not control for participants’ motor familiarity with the action stimuli. Previous studies indicate that familiarity with observed actions can influence recognition, discrimination, and memory performance due to the activation of motor representations^34,53–56. Second, we did not control for the vestibular demands of the action stimuli. GVS has been found to selectively affect recognition of actions with high vestibular engagement³⁴; however, our stimuli ranged from actions involving significant vestibular activity (e.g., jumping, dancing) to those requiring minimal vestibular engagement (e.g., hand raising, saluting). Third, our study lacked requirements for response speed, resulting in an absence of constraints on response times. Previous studies suggest that GVS may differentially affect accuracy and response times in cognitive tasks^34,39. To address these limitations, future research should measure participants’ familiarity with each action stimulus, categorize stimuli based on vestibular demands, and incorporate response time measurements under speeded conditions to more accurately assess the impact of GVS on VWM for actions.

In summary, this study is the first to investigate the effects of GVS on VWM for actions, though no significant enhancements were observed. Several factors may explain these null results, including the absence of noise-enhanced GVS, variability in individual responses to GVS, and the lack of control for motor familiarity and vestibular demands. Future studies should consider incorporating noise-enhanced GVS, assessing participants’ sensory thresholds for personalized GVS intensity settings, and addressing the factors of motor familiarity and vestibular demands of stimuli. Given the observed complexities and variations in response, further investigation into GVS’s potential to enhance VWM for actions is warranted. Such research would help clarify its effectiveness under different conditions, address individual differences, unlock potential applications for cognitive enhancement, and provide deeper insights into the neural mechanisms underlying VWM and the broader role of the vestibular system in cognitive functions.

Methods

Participants

We ran three experiments and recruited participants separately for each experiment at Zhejiang University (Experiment 1: 25 effective participants, mean age = 22.5 years, age range = 18–27 years, 12 males, all right-handed; Experiment 2: 20 effective participants and 2 excluded participants, mean age = 22 years, age range = 18–28 years, 6 males, all right-handed; Experiment 3: 20 effective participants and 4 excluded participants, mean age = 21.0 years, age range = 18–24 years, 8 males, all right-handed). All participants had normal or corrected-to-normal vision acuity, and no one reported a history of motion sickness, vestibular dysfunction or hearing loss. A Sequential Bayes Factors (BFs) design was adopted to determine the sample size⁵⁷. In the Sequential BF design, BFs are computed repeatedly during data collection until the BF exceeds a priori-defined grade of evidence. The minimum sample size was set to 20, and the critical BF₁₀ for stopping the sequential sampling was set to 3 (resp. 1/3)⁵⁸. The BF₁₀ greater than 3 indicates moderate to strong evidence supporting the alternative hypothesis (H₁), whereas the BF₁₀ less than 1/3 indicates moderate to strong evidence supporting the null hypothesis (H₀)⁵⁸. We calculated the BFs via JASP (version 0.18.3.0) software (https://jasp-stats.org/) with the default prior width. If a participant’s accuracy did not exceed 50%, they were excluded from the analysis. Finally, effective participants were included in the analysis. The participants had no prior experience with GVS, and received payment or course credits for their participation. Written informed consents from all participants were obtained prior to study participation. Participants involved in the images (Fig. 1b,c) consented to the publication of these images in an online open-access publication. Ethic approval in accordance with the Helsinki declaration for this study was granted by the Research Ethics Board of Zhejiang University. All methods were performed in accordance with the relevant guidelines and regulations.

Stimuli and apparatus

Fifteen actions were selected as stimuli in Experiments 1 and 2. They were composed of a 3-D human model exported from Poser (version 11.0) software (https://www.poser software.com/) and 15 animations downloaded from the Mixamo (https://www.mixamo.com/). The action stimuli were labeled in Mixamo as “golf drive”, “guitar playing”, “hand raising”, “jazz dancing”, “jogging”, “jump”, “jumping jacks”, “right turn”, “salsa dancing”, “salute”, “walk”, “waving”, “boxing”, “martelo”, and “air squat bent arms”. All stimuli were created and presented in the Unity 3D engine (version 2019.3.11f1) and displayed in VR environments supported by the Unity 3D and SteamVR plugins. For each action, the human model was resized to the appropriate size for observation (1.37 m width, 3.64 m height in Unity 3D), and each animation consisted of 60 distinct frames, leading to a 1-s action (refresh rate 60 Hz). We inverted these actions as stimuli in Experiment 3.

During each experiment, participants sat on a chair and completed the memory task in a VR environment with a horizon-like background. They wore an HTC Vive Pro headset, held two controllers in both hands, and kept their heads still. Participants could see all the displayed stimuli simultaneously in their visual field and used the controllers to start the experiment and send their judgments.

At the beginning of the experiment, two square rubber electrodes (5 cm in length) were attached to the left and right mastoids of the participants. GVS was delivered by the neuroConn DC-STIMULATOR MC. A bilateral bipolar stimulation was used, employing a sinusoidal signal with an amplitude of 0.8 mA and a frequency of 0.5 Hz. The current was delivered until the participant reported a slight perception of head swaying from side to side without any tingling sensation at the electrode sites, confirming that the current effectively stimulated the vestibular system and was below the cutaneous threshold. In the SHAM condition, the current intensity was set to 0 mA by turning off the GVS stimulator after the confirmation procedure. In the STIM condition, 0.8 mA-current GVS was applied continuously until the end of the block.

Procedure and design

Each trial began with two white digits at the view center for 500 ms, and participants were required to verbally repeat the digits aloud throughout the trial. The concurrent articulatory suppression task was used to inhibit verbal coding of the to-be-memorized stimuli^8,59. Next, a white fixation cross appeared for 300 ms to inform the participants of the upcoming memory task. After a blank interval of 150 to 350 ms, the memory array (Experiments 1 and 2: either five or seven actions; Experiment 3: either five or seven inverted actions) was presented for 7 s to ensure that participants had enough encoding time. The to-be-memorized stimuli were randomly displayed at nine fixed positions in VR. The view distances of the stimuli were 13 m and 17 m, respectively, for the two depths of stimuli. After a 1,000 ms blank interval, one probed stimulus (Experiments 1 and 2: probed action; Experiment 3: probed inverted action) was presented at the view center for 1,000 ms and then replaced by a red “?”. Participants had to judge whether the probe appeared in the memory array within 3,000 ms. They pressed the controller trigger in the left hand for “not appear”, and in the right hand for “appear”. The probed stimulus was set to change independently with a probability of 50%. When a change occurred, a new stimulus, which was not used in the memory array, was adopted as a probe.

In each experiment, a within-subject design was used, in which both the set size (five vs. seven) of the memory array and the GVS state (SHAM vs. STIM) were within-subject factors. The experiment was divided into two blocks according to the GVS state: participants completed the VWM task without GVS (SHAM block) or with continuous GVS (STIM block). In Experiment 1, there were 30 trials for each factor combination, resulting in 60 randomly presented trials for each block. Participants took a 2-min rest after 30 trials each in the block and had a 2-min break between the two blocks. The current stimulus was turned off during the break. In Experiments 2 and 3, there were 16 trials for each factor combination, resulting in 32 randomly presented trials for each block. Participants rested for at least 10 min between the two blocks to ensure they did not experience any discomfort before starting the next block.

For each experiment, the order of the two blocks was counterbalanced via reverse counterbalancing. Before the formal trials, participants practiced at least 12 trials to ensure they understood the instructions. After the practice session of the STIM block, they were confirmed to feel the sensation of head rotation induced by GVS without experiencing discomfort. If participants experienced any unbearable discomfort in any area of their body (such as a burning sensation at the electrode site, dizziness, or other uncomfortable symptoms), they could withdraw from the experiment at any time. The whole experiment lasted about 40 min.

Data analysis

To estimate the VWM capacity, we used Cowan’s formula⁴³: K = S × (H − F), in which K is the VWM capacity, S is the set size, H is the hit rate referring to the successful detection of a new probe, and F is the false alarm rate of misjudging an unchanged probe as a new one. We calculated K for each set size in each GVS state of each participant. The task and calculated capacity (K value) allow for sufficient sensitivity and granularity to observe changes in VWM^60–62. For each experiment, two-way repeated-measures ANOVAs were conducted on K, H, F and response times, with set size (five vs. seven) and GVS state (SHAM vs. STIM) as within-subject factors. Since the separate ANOVAs for H, F and response times showed GVS effects similar to those found for K, we only reported the analysis results of K in the Results section to avoid redundancy. The detailed statistical analysis results for H, F and response times are shown in the supplementary materials. To gain an accurate estimate of the VWM capacity of actions or inverted actions in each GVS state condition, we considered the maximum K (K-max) among the two load conditions as one’s VWM capacity^8,59.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Author contributions

Luyao Hu designed research, provided the working memory task codes, conducted the experiments, conducted the data analyses, and wrote the first version of the manuscript; Qi Gao and Zaifeng Gao assisted in the experiment design and provided suggestions for the writing of the manuscript; Zaifeng Gao and Rui Wang provided equipment and financial supports; all authors reviewed and edited further versions of the manuscript.

Data availability

The datasets generated during and analyzed during the current study are available in the Open Science Framework repository, https://osf.io/nv62z/?view_only=f8566c5bdc784be1bd296bac6e15ea8b

Declarations

Competing interests

The authors declare no competing interests.