Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2017 Oct 4;37(40):9771–9777. doi: 10.1523/JNEUROSCI.1167-17.2017

Content-Specific Codes of Parametric Vibrotactile Working Memory in Humans

Timo Torsten Schmidt 1,2,✉, Yuan-hao Wu 1,3, Felix Blankenburg 1,3
PMCID: PMC6596612  PMID: 28893928

Abstract

To understand how the brain handles mentally represented information flexibly in the absence of sensory stimulation, working memory (WM) studies have been essential. A seminal finding in monkey research is that neurons in the prefrontal cortex (PFC) retain stimulus-specific information when vibrotactile frequencies were memorized. A direct mapping between monkey studies and human research is still controversial. Although oscillatory signatures, in terms of frequency-dependent parametric beta-band modulation, have been observed recently in human EEG studies, the content specificity of these representations in terms of multivariate pattern analysis has not yet been shown. Here, we used fMRI in combination with multivariate classification techniques to determine which brain regions retain information during WM. In a retro-cue delayed-match-to-sample task, human subjects memorized the frequency of vibrotactile stimulation over a 12 s delay phase. Using an assumption-free whole-brain searchlight approach, we tested with support vector regression which brain regions exhibited multivariate parametric WM codes of the maintained frequencies during the WM delay. Interestingly, our analysis revealed an overlap with regions previously identified in monkeys composed of bilateral premotor cortices, supplementary motor area, and the right inferior frontal gyrus as part of the PFC. Therefore, our results establish a link between the WM codes found in monkeys and those in humans and emphasize the importance of the PFC for information maintenance during WM also in humans.

SIGNIFICANCE STATEMENT Working memory (WM) research in monkeys has identified a network of regions, including prefrontal regions, to code stimulus-specific information when vibrotactile frequencies are memorized. Here, we performed an fMRI study during which human subjects had to memorize vibratory frequencies in parallel to previous monkey research. Using an assumption-free, whole-brain searchlight decoding approach, we identified for the first time regions in the human brain that exhibit multivariate patterns of activity to code the vibratory frequency parametrically during WM. Our results parallel previous monkey findings and show that the supplementary motor area, premotor, and the right prefrontal cortex are involved in vibrotactile WM coding in humans.

Keywords: mental representation, prefrontal cortex, support vector regression, vibrotactile, working memory

Introduction

Flexible handling of mentally represented information is the basis for any higher cognitive process. To investigate how the brain represents mental material in the absence of sensory stimulation, working memory (WM) studies have been essential. For a long time, the dominant view was that major aspects of WM content are maintained in prefrontal regions (Goldman-Rakic, 1995). This view was supported by electrophysiological recordings in monkeys finding that neurons in the prefrontal cortex (PFC) exhibited stimulus-specific firing (Romo et al., 1999). Studies on sensory WM by Romo and colleagues, who explored the retention of vibration frequency of tactile stimuli in nonhuman primates, were particularly influential (Hernández et al., 2000; Romo and Salinas, 2001, 2003a, Romo et al., 2002, 2004). These seminal studies have established direct links between the observed neuronal signals and the actual content that was memorized (Pasternak and Greenlee, 2005), namely, that firing of neurons in the PFC was modulated parametrically depending on the maintained vibration frequency (Romo et al., 1999). Consecutive studies depicted a network of regions exhibiting similar WM codes while controlling for potentially conflating perceptual or decisional processes (Romo and de Lafuente, 2013; Romo et al., 2012). Romo and colleagues were able to show parametric mnemonic representations in secondary somatosensory, medial frontal, and premotor cortices and the PFC (Romo and de Lafuente, 2013; Romo et al., 2012). Likewise, several human oscillatory EEG studies have emphasized the role of the right PFC in maintaining information about the stimulus feature (i.e., vibrotactile frequency; Spitzer and Blankenburg, 2012; Spitzer et al., 2014a). However, it is still unclear whether such codes in humans indeed reflect the WM content.

Noninvasive human brain imaging research has initially been limited to the study of elevated activity during WM delay periods. However, delayed activity cannot dissociate between the representation of WM content and other content-independent functions that assist in retaining and manipulating mental representations (Baddeley, 2012; Riggall and Postle, 2012; Sreenivasan et al., 2014a; D'Esposito and Postle, 2015). Multivoxel pattern analysis (MVPA) for fMRI has recently received substantial attention in WM research because this approach allows patterns of local activity to be related directly to particular WM contents (Norman et al., 2006; Haynes, 2015; Lee and Baker, 2016; Christophel et al., 2017) and therefore to identify brain regions that code specific types of mental content. Controversially, these MVPA studies largely failed to find stimulus-specific activity in the PFC (Riley and Constantinidis, 2016). Instead, they suggest sensory and parietal regions to exhibit stimulus-specific activation (Lee and Baker, 2016), favoring sensory recruitment models of WM (Pasternak and Greenlee, 2005). Therefore, the view of the human PFC's function for WM has been reevaluated. It remains controversial if, or under which conditions, the PFC exhibits stimulus-specific codes (D'Esposito and Postle, 2015; Ester et al., 2015; Postle, 2015; Lee and Baker, 2016; Riley and Constantinidis, 2016).

Meanwhile, evidence has accumulated that different types of information are coded in different brain regions depending on their degree of abstraction (Christophel et al., 2017). Vibratory frequencies are not thought to be mere low-level sensory stimulus features but are considered as “analog properties” together with stimulus features such as intensity or duration. Most recent studies focused on sensory aspects of WM and mainly fostered sensory recruitment models of WM (Sreenivasan et al., 2014). Research on such more abstract (analog) features has been limited.

Based on the monkey work of Romo and colleagues (Romo and de Lafuente, 2013), we sought to investigate whether similar regions in the human brain also exhibit activation patterns to which MVPA is sensitive. To test which brain regions parametrically represent vibratory frequency, we applied a searchlight approach using support vector regression (Kriegeskorte et al., 2006; Kahnt et al., 2011) in a human version of the vibrotactile WM task. Crucially, this approach does not make any a priori assumptions on the localization of such representational codes.

Materials and Methods

Participants

All subjects (N = 24) were neurologically intact and right-handed, as assessed by the Edinburgh Handedness Inventory (EHI) (Oldfield, 1971). A total of N = 22 subjects were included in the analysis (age: 26.46 ± 3.38 years, 16 female, EHI: 0.87 ± 0.15) because two subjects were excluded due to excessive head motion (>9 mm). All participants provided written informed consent for the procedure in accordance with protocols approved by the local ethics committee of Freie Universität Berlin.

Experimental design

We applied a retro-cue WM paradigm in which the presentation of two vibrotactile sample frequencies was followed by a visual cue indicating which of the two frequencies had to be memorized during a subsequent delay period (Fig. 1). Subjects performed a two-alternative forced choice task after a delay of 12 s to indicate which of two test stimuli was identical to the memorized sample. Responses were given via right middle and index finger button presses and left/right response mapping was randomized across subjects. Vibrotactile stimuli were delivered to the left index finger using a 16-dot piezoelectric Braille-like display (4 × 4 matrix with 2.5 mm spacing) controlled by a programmable stimulator (QuaeroSys, St. Johann, Germany).

Figure 1.

Figure 1.

Open in a new tab

WM task for vibrotactile frequency stimuli. A, The stimulus set was composed of four vibratory frequencies of 10, 22, 34, and 46 Hz. Vibrotactile stimuli were presented via a 4 × 4 pin Braille-like display to the left index finger. B, Delayed-match-to-sample paradigm was applied where two vibrotactile sample stimuli and a mask stimulus were presented consecutively to the left index finger. A visual retro-cue indicated which of the sample frequencies had to be memorized during a subsequent 12 s delay period. After this delay, subjects performed a two-alternative forced choice on two consecutively presented target stimuli. One stimulus was identical to the memorized stimulus and the other was a foil (randomized order). Subjects responded with right hand button presses to indicate which of the two target stimuli was identical to their WM content. Beta estimates were derived from a FIR model for each 2 s time bin of the delay phase and entered into independent decoding analyses. Prediction accuracy maps from the eight searchlight SVR analyses and all subjects were entered into a second-level analysis to test for above-chance decoding using a t-contrast across the six WM delay time bins.

The set of vibrotactile sample stimuli contained four frequencies: 10, 22, 34, and 46 Hz (Fig. 1). All 16 pins of the Braille-like display vibrated with equal amplitude for 600 ms with smoothed onsets and offsets. One of the test stimuli matched the sample stimulus' frequency, whereas the foil stimulus was higher or lower in 50% of the trials, respectively. To achieve equal difficulty in each trial, we adjusted the frequency of the foil following the Weber–Fechner law (logarithmic relation between stimulus and perception; Fechner, 1966) anchored at 28 + 9 Hz, resulting in 7.6 and 13.2 Hz for the 10 Hz stimulus, 16.6 and 29.1 Hz for the 22 Hz stimulus, 25.7 and 44.9 Hz for the 34 Hz stimulus, and 34.8 and 60.8 Hz for the 46 Hz stimulus. The mask stimulus was composed of pins vibrating at all four sample frequencies and 180° phase-shifted equivalents and was presented for 500 ms with smoothed onsets and offsets.

Each experimental run consisted of 48 trials with a 12 s WM delay, supplemented by 8 catch trials with delays of 4 or 8 s to ensure sustained WM throughout the retention period. Each of six stimulus pairs was presented equally often in a balanced stimulus order, in which each of the four samples was memorized 12 times. Subjects were familiarized with the task by completing 1 or 2 practice runs outside of the scanner 1–5 d before the fMRI experiment.

fMRI data acquisition and preprocessing

fMRI data were acquired in 4 runs of 20 min on a 3 T scanner (TIM, TRIO, Siemens) at the MRI facility of the Freie Universtät, Berlin. For each run, 600 functional images were acquired (T2*-weighted gradient-echo EPI: 37 slices; interleaved order; 20% gap; whole brain; TR = 2000 ms; TE = 30 ms; 3 × 3 × 3 mm3 voxel; flip angle = 90°; 64 × 64 matrix). In addition, structural MRI data were acquired (MPRAGE, 176 sagittal slices, TR = 1900 ms, TE = 2.52 ms, 1 × 1 × 1 mm3 voxel).

Trial onsets were time locked to the functional images. This approach allows a TR-wise analysis of the 12 s WM phase in six consecutive functional images, as applied previously in the study of visual short-term memory (Christophel et al., 2012).

Statistical analysis

Time-resolved searchlight decoding of parametric multivariate frequency representations.

To preserve the spatiotemporal structure of the fMRI signal, no smoothing, normalization, or slice-time correction was performed before classification and preprocessing was limited to spatial realignment. fMRI data processing was performed using SPM8 (Wellcome Trust Centre for Neuroimaging, Institute for Neurology, University College London). Finite impulse response (FIR) models were used to obtain runwise beta estimates for each 2 s time bin of the WM delay (to display time courses, an additional 2 s time bin before and after the WM delay was modeled). High-pass filtered data (cutoff 192 s) were included in a model with a total of 132 beta estimates (4 stimuli × 8 time bins × 4 runs + 4 constants). All trials of the experiment except the catch trials were modeled to avoid any potential biases from the exclusion of trials. As control analysis, the identical decoding approach was performed with beta estimates from a model containing only correct trials and revealed virtually identical results.

To identify where in the brain information about the memorized vibratory frequency was encoded, we applied a time-resolved multivariate decoding analysis using a support vector regression (SVR) model, which has been described previously (Kahnt et al., 2011). This approach allows testing for local multivariate representations (activation patterns of voxels) that code the vibratory frequency maintained during WM. Compared with classic support vector machine approaches, SVR does not only distinguish between two classes of stimuli. Instead, an SVR model is trained to predict the value of a continuous variable, here vibratory frequency, based on a multivariate data vector. This is similar to a univariate regression approach, in which the prediction of a dependent variable is only based on one independent variable. In SVR, a data vector with multiple independent variables (activation pattern of voxels) is used to predict a single dependent variable (the maintained vibratory frequency).

All decoding analyses were performed using The Decoding Toolbox (TDT) (Hebart et al., 2015), which provides an interface to apply the computational routines of LIBSVM (Chang and Lin, 2011) and LIBLINEAR (Fan et al., 2008) to neuroimaging data. Crucially, we did not limit our analysis by assumptions on where in the brain local activation patterns relate to WM content. Instead, we applied a searchlight analysis (Kriegeskorte et al., 2006) in which a spherical volume of interest is moved step-by step (voxel-by-voxel) through the entire brain. In each searchlight step, a SVR model is trained and tested on data extracted from a local sphere of voxels. Thereby, 3-dimensional prediction accuracy maps for the whole brain are derived for each experimental subject. Consecutively, these maps allow group-level statistical testing to identify those regions that exhibit above-chance predictions across subjects and thus, maintain information on the memorized vibratory frequency.

The subject-level decoding analysis was performed using a leave-one-out cross-validation scheme in combination with an SVR kernel (linear kernel and constant regularization parameter of c = 1). Independent whole-brain searchlight decoding analyses were performed for each time bin. Beta estimates of each of the eight time bins were first z-scaled (normalized) across the samples for each voxel, as implemented in TDT. In each searchlight step, z-scaled beta estimates of voxels in a four-voxel radius sphere were extracted from each of the four maps corresponding to the four vibratory frequencies. Next, z-scaled beta estimates were forwarded to a fourfold cross-validation scheme in which data from three of the four runs are used to “train” the SVR. This SVR is then used to predict values of vibratory frequency based on the independent fourth run's data. To make our data comparable to previous reports, we used as a “test” measure the prediction accuracy, defined as the Fisher's z-transformed correlation coefficient between the predicted value levels and the actual value levels of the test dataset (Kahnt et al., 2011) (in TDT: “zcorr”). The center of the searchlight was moved voxelwise through the brain and prediction accuracy values were saved in the corresponding whole-brain accuracy maps. In this way, we gained a prediction accuracy map per subject reflecting which brain regions code the value of the vibratory frequency in a multivariate way, which is considered the multivariate pendant of a parametric code (Kahnt et al., 2011). This analysis yielded eight accuracy maps per subject, corresponding to the six timebins spanning the WM interval and one volume before and after it (Fig. 1).

Accuracy maps were normalized to MNI space using unified segmentation (as implemented in SPM8) and smoothed with a 3 mm full-width half-maximum kernel. These were entered consecutively into a flexible factorial second-level design (repeated measures) and a t-contrast for the mean of the six WM time bins against zero was computed. Voxels with above-chance prediction accuracies are reported at a threshold of p < 0.05 family-wise error (FWE) corrected for multiple comparisons and displayed with a cluster extent threshold of 35 voxels. All reported coordinates are in MNI space. The SPM anatomy toolbox was used to establish cytoarchitectonical references (Eickhoff et al., 2007).

Control analysis: decoding the nonmemorized stimulus.

As a control analysis, we tested for above-chance prediction accuracy for the nonmemorized stimulus. New FIR models were estimated, in which the four sets of FIR regressors modeled those trials in which a particular stimulus was presented but not memorized. Therefore, each beta image was estimated with equal amounts of data (modeling the same amount of trials) as in the original analysis. Beta images were entered into an identical SVR searchlight and second-level analysis as in the main analysis. No clusters with above-chance prediction accuracies were found.

Control analysis: label permutation tests.

To determine the prediction accuracy of an SVR analysis, the beta estimates within a searchlight volume for all four ordered stimulus conditions are entered into a SVR model. A high prediction accuracy is only expected if the activation patterns in a given region represent this order of the four frequencies. To test for the specificity of our analysis, we performed exhaustive whole-brain SVR searchlight analyses for all possible permutations of frequency labels. To this end, we determined for all possible permutations the distance from the rank order calculated as the sum of absolute difference of adjacent ranks (e.g., the linear order of 10, 22, 34, 46 Hz has a distance of ranks of sum(|1 − 2| + |2 − 3| + |3 − 4|) = 3 and the permutation 22, 10, 34, 46 Hz corresponds to the sum(|2 − 1| + |1 − 3| + |3 − 4|) = 4, resulting in a difference of 1 from the linear order). Because rank order is symmetric, the permutation analyses span 12 instead of 24 possible permutations congregated into four classes of distances from the linear order. We extracted the prediction accuracies from the group peak voxels of the original analysis. Figure 2C presents averaged prediction accuracies for the four groups of distances from the linear order.

Figure 2.

Figure 2.

Open in a new tab

Brain regions exhibiting a WM code for the memorized vibratory frequency. A, Regions with above-chance prediction accuracy across the WM delay period revealed by a t-contrast displayed at p < 0.05, FWE corrected. This network of regions overlaps with areas previously identified in nonhuman primates (see also Table 1) and is constituted of left and right dorsal PMC (dPMC), right IFG (rIFG; 54.1% in right Area 44 according to the anatomy toolbox; Eickhoff et al., 2005, 2007), and a cluster in the SMA/CC. B, Time courses of prediction accuracy through the WM delay phase for the peak voxel of the five clusters depicted in A (mean ± SEM). Time courses of the control analysis decoding the nonmemorized stimulus are presented in gray and do not show above-chance prediction accuracies throughout the delay phase. C, Results of permutation testing in which the same SVR analysis was performed with permuted frequency labels of the data. The divergence of the permutations from the order of 10, 22, 34, and 46 Hz is expressed as difference in rank order (see “Materials and Methods”). As expected if the order of frequencies determines the performance of the SVR then a decrease in the order of frequency labels resulted in decreased prediction accuracy. This analysis thereby demonstrates that it is indeed the linear order of memorized frequencies that constitutes the basis for high prediction accuracies in these regions. Bars with asterisk indicate a significant correlation of order and prediction accuracy for p < 0.05.

For a statistical assessment, we tested for each time bin whether an increase in ordering correlated with an increase in prediction accuracy. We calculated Spearman correlation coefficients for each time bin and report significant effects at p < 0.05 (Figure 2C).

Results

Behavioral performance

Subjects (N = 22) successfully memorized the relevant information in the demanding task, as indicated by the 62 ± 4% (mean ± SD) correct responses over 4 fMRI runs. As expected, performance was better for the second (64.5 ± 5%) than for the first (60 ± 7%) presented stimulus (paired t-test t(21) = −2.65, p = 0.015), which was accounted for by a counterbalanced stimulus presentation.

Multivariate mapping of regions that code the content of WM

To test which brain regions exhibit a WM code for vibratory frequency, we conducted a time-resolved whole-brain SVR analysis. We applied an SVR approach to test in which regions information on the order of increasing frequencies can be found by testing where in the brain decoding performance exceeded chance level. We computed a t-contrast of prediction accuracy versus zero across the WM delay period within a second-level flexible factorial design. After accounting for multiple comparisons with a conservative thresholding of p < 0.05 and FWE correction, we identified five clusters of above chance decoding that match with reports from monkey research: bilateral dorsal premotor cortex (dPMC), a cluster in the supplementary motor area/cingulate cortex (SMA/CC), and a cluster in the right inferior frontal gyrus (IFG), as displayed in Figure 2A and Table 1.

Table 1.

Regions identified to exhibit above-chance prediction accuracy across the WM delay period

Cluster size Anatomical region Peak MNI coordinates
 z-score
X Y Z
961 Right dPMC 48 2 44 Inf
661 CC −2 12 38 7.36
478 Left dPMC −60 −4 42 5.84
67 Right IFG 54 8 4 5.02
65 SMA 10 12 70 4.93
Open in a new tab

Regions were revealed by a t-contrast displayed at p < 0.05, FWE corrected.

As a control analysis, the study design allowed us to perform an identical MVPA for the noncued stimulus because two sample stimuli were presented on each trial and pairs of stimuli were balanced (Christophel et al., 2012). Decoding of the nonmemorized stimulus did not reveal any significant clusters of voxels, demonstrating the specificity of the main analysis.

To test for the specificity of the linear ordering of stimuli with the applied SVR approach, we performed permutation tests. To this end, we relabeled the training and test data with all possible permutations of frequency labels. If the order of frequencies were crucial for the SVR's performance, then one would expect increased prediction accuracies with increasing order. We particularly expected this effect during the WM delay period, but not before or after it. The time courses for all four types of permutations across the delay period are plotted in Figure 2C. Note that completely unordered labeling leads to chance performance of the SVR throughout the entire WM period. With increased order, the prediction accuracy increased for all identified regions across the delay period. As expected from the dynamics of the BOLD response, we found that this effect peaked between 4 and 8 s after the WM onset.

In addition to the multivariate analysis, we performed a classic general linear model analysis and tested for univariate parametric modulations during the WM delay phase, which did not reveal any significant voxels (p < 0.001, uncorrected). This suggests that it is the combination of multiple voxels, and thus, distributed neuronal populations, that code for the vibratory frequency rather than an overall parametrically increased activity throughout voxels in the identified clusters.

Discussion

Studies in nonhuman primates (Romo et al., 2012) have identified a network of brain regions where neurons exhibit firing behaviors that are parametrically modulated by the maintained vibrotactile frequency. In a series of EEG experiments, Spitzer and coworkers extended this work to humans by showing that, consistent with the monkey reports, the retention of vibration frequency induces a content-specific parametric beta-band modulation during delay periods, which was source-localized to the right PFC (Spitzer and Blankenburg, 2012; Spitzer et al., 2014a). However, until now, the mapping of brain regions that code vibrotactile frequency in the form of multivariate patterns during human WM has been lacking. Here, we used a multivariate regression approach to test which brain regions exhibit specific patterns of activation for maintained vibratory frequencies. The applied SVR approach can be considered as the multivariate equivalent of univariate tests for parametric modulations; that is, a classifier is trained to predict a value (here: the frequency) instead of a class label (Fan et al., 2008; Kahnt et al., 2011; Hebart et al., 2015). Therefore, the order of frequencies needs to be represented in a region to yield above-chance decoding results. With a whole-brain searchlight approach, testing within the entire brain for such local content-specific codes, we found that the bilateral premotor cortices, medial frontal regions, and right IFG code frequency-specific information. Our control analyses proved the specificity of these results because it was not possible to detect information on the nonmemorized stimuli and because permutation tests showed that it was indeed the ordering of the frequencies that enabled to decode information from these regions.

Interestingly, the identified network overlaps with brain regions reported in Romo and colleagues' monkey research (Romo and de Lafuente, 2013; Romo et al., 2012), as well as with those in the human EEG reports (Spitzer and Blankenburg, 2012; Spitzer et al., 2014a). Here, we extend these findings by showing for the first time that the right PFC also codes multivariate content-specific stimulus information.

In parallel to the monkey data, WM content was not limited to a single brain region; rather, a network of regions was identified. It is reasonable to assume that the brain represents mental content depending on task demands in distributed cortical networks, which can also exhibit different types of codes (Schlegel et al., 2013; Larocque et al., 2014; Lewis-Peacock et al., 2015; Postle, 2015; Lee and Baker, 2016; Christophel et al., 2017). In addition to the PFC, our analysis revealed that the bilateral dorsal premotor cortices and supplementary motor regions (Romo and Salinas, 2003; Hernández et al., 2010; Romo and de Lafuente, 2013; Romo et al., 2012) also exhibit these codes. Dorsal premotor cortices have been indicated to contribute to rehearsal processes (Fegen et al., 2015), whereas details on the mechanisms underlying rehearsal or refreshment of sensory contents (Chun and Johnson, 2011) are still relatively unclear (Baddeley, 2012). Our study shows that these regions exhibit a parametric representation of the WM content. Even though this is consistent with the monkey reports, a direct mapping between monkey and human anatomy remains speculative until future studies provide more direct evidence for these regions' functional roles. Although our study provides a first link for the apparent gap between unit-level recordings and fMRI-based classification approaches, MVPA studies are limited in their ability to identify a specific coding schema. They also allow us to test which regions code WM information but cannot resolve the nature of the code as such.

Our findings are in contrast to several recent WM MVPA studies. Most of these studies have argued for sensory recruitment models because they found content-specific codes predominantly in sensory and posterior regions (Pasternak and Greenlee, 2005; Sreenivasan et al., 2014; Riley and Constantinidis, 2016). Similarly, early monkey research on tactile WM supported the notion that sensory regions are predominantly recruited to represent WM content (Zhou and Fuster, 1996, 2000). This view aligns with the famous WM model by Alan Baddeley (Baddeley, 2012), which is based on the idea of a visuospatial sketchpad, which is now often equated with sensory regions that are supposed to act as a kind of whiteboard on which mental content is drawn (Pasternak and Greenlee, 2005). In addition to WM studies, research on mental imagery also builds heavily on the idea that activity in sensory regions represents the content of mental images (Kosslyn, 2005; Tong, 2013). Indeed, it has been demonstrated that primary somatosensory cortex is recruited for tactile imagery (Schmidt et al., 2014).

On the contrary, frontal and prefrontal regions are well documented to activate in WM studies across different modalities, including tactile WM (Preuschhof et al., 2006). However, these regions are most often considered to fulfill supporting, top-down functions rather than to represent WM content (D'Esposito and Postle, 2015; Myers et al., 2017). With the exception of studies on vibrotactile frequency representations, only a few fMRI studies have revealed prefrontal WM codes that reflect aspects of the WM content (Lee et al., 2013; Ester et al., 2015).

One interpretation of these conflicting findings might be that different tasks require various types of codes. Visual WM studies mostly necessitate mentally representing very precise visuospatial stimulus features such as orientation (Albers et al., 2013), position (Jerde et al., 2012), motion direction (Emrich et al., 2013; Christophel and Haynes, 2014), or the layout of a stimulus (Christophel et al., 2012; Lee et al., 2013). These studies convincingly demonstrate that such low-level or sensory-like features are maintained in brain regions which also deal with such type of information during perception. Therefore, Christophel and coworkers recently suggested that it might be the level of “abstraction” of the WM content that determines the network of regions coding it (Christophel et al., 2017). Although primary sensory cortices deal with low-level sensory-like material, regions higher up in the cortical hierarchy, such as the PFC, might process more abstract stimulus features. This view resonates well with evidence that the right PFC processes nonverbal quantitative types of information (Nieder and Dehaene, 2009) and the left lateral PFC, particularly BA44, is well known for its pivotal role in language processing. In monkeys, the right PFC has also been associated with codes for size/length/quantity or numerosity (Nieder and Dehaene, 2009; Nieder, 2013). These magnitude-related stimulus properties are considered to be represented independently of sensory aspects (Walsh, 2003) and human fMRI and EEG studies also associate them with the PFC (Pinel et al., 2004; Nieder and Dehaene, 2009; Spitzer et al., 2014b, 2014c). Consistent with this reasoning, the maintenance of vibrotactile frequency might be considered a representation of an abstract, magnitude-like representation despite a low-level stimulus feature, a type of code for hierarchically higher brain regions such as the PFC. Even though this idea is appealing and in accordance with our data, future research has to provide evidence for it by dissecting different types of information and codes, as well as their functional contribution to goal-directed purposeful behavior and decision making (Romo et al., 2012), including other modalities.

Conclusion

Our study found the right PFC, SMA/CC, and bilateral dPMC to contain multivariate codes of retained vibratory frequencies during human WM. These codes represented the parametric order of the memorized frequencies and the identified network overlapped with previous monkey studies. Future studies will elicit what types of WM contents are represented differentially across species and which cellular mechanisms can be translated more directly between them. For this endeavor, it is crucial to establish links between different observational levels in neuroscience. The bridges between monkey and human data in the study of WM are rare. Our results allow initial new perspectives to unite the different lines of literature using distinct methodologies.

Footnotes

Author contributions: T.T.S. and F.B. designed research; T.T.S. and Y.W. performed research; T.T.S., Y.W., and F.B. analyzed data; T.T.S., Y.W., and F.B. wrote the paper.

T.T.S. was supported by the Research Training Group (Grant GRK 1589/2, Sensory Computation in Neural Systems, by the German Research Foundation, DFG).

The authors declare no competing financial interests.

References

  1. Albers AM, Kok P, Toni I, Dijkerman HC, de Lange FP (2013) Shared representations for working memory and mental imagery in early visual cortex. Curr Biol 23:1427–1431. 10.1016/j.cub.2013.05.065 [DOI] [PubMed] [Google Scholar]
  2. Baddeley A. (2012) Working memory: theories, models, and controversies. Annu Rev Psychol 63:1–29. 10.1146/annurev-psych-120710-100422 [DOI] [PubMed] [Google Scholar]
  3. Chang C, Lin C (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:1–39. [Google Scholar]
  4. Christophel TB, Haynes JD (2014) Decoding complex flow-field patterns in visual working memory. Neuroimage 91:43–51. 10.1016/j.neuroimage.2014.01.025 [DOI] [PubMed] [Google Scholar]
  5. Christophel TB, Hebart MN, Haynes JD (2012) Decoding the contents of visual short-term memory from human visual and parietal cortex. J Neurosci 32:12983–12989. 10.1523/JNEUROSCI.0184-12.2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Christophel TB, Klink PC, Spitzer B, Roelfsema PR, Haynes JD (2017) The distributed nature of working memory. Trends Cogn Sci 21:111–124. 10.1016/j.tics.2016.12.007 [DOI] [PubMed] [Google Scholar]
  7. Chun MM, Johnson MK (2011) Memory: enduring traces of perceptual and reflective attention. Neuron 72:520–535. 10.1016/j.neuron.2011.10.026 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. D'Esposito M, Postle BR (2015) The cognitive neuroscience of working memory. Annu Rev Psychol 66:115–142. 10.1146/annurev-psych-010814-015031 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Eickhoff SB, Stephan KE, Mohlberg H, Grefkes C, Fink GR, Amunts K, Zilles K (2005) A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage 25:1325–1335. 10.1016/j.neuroimage.2004.12.034 [DOI] [PubMed] [Google Scholar]
  10. Eickhoff SB, Paus T, Caspers S, Grosbras MH, Evans AC, Zilles K, Amunts K (2007) Assignment of functional activations to probabilistic cytoarchitectonic areas revisited. Neuroimage 36:511–521. 10.1016/j.neuroimage.2007.03.060 [DOI] [PubMed] [Google Scholar]
  11. Emrich SM, Riggall AC, Larocque JJ, Postle BR (2013) Distributed patterns of activity in sensory cortex reflect the precision of multiple items maintained in visual short-term memory. J Neurosci 33:6516–6523. 10.1523/JNEUROSCI.5732-12.2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Ester EF, Sprague TC, Serences JT (2015) Parietal and frontal cortex encode stimulus-specific mnemonic representations during visual working memory parietal and frontal cortex encode stimulus-specific mnemonic representations during visual working memory. Neuron 87:893–905. 10.1016/j.neuron.2015.07.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Fan R-E, Chang K-W, Hsieh C-J, Wang X-R, Lin C-J (2008) LIBLINEAR: a library for large linear classification. J Mach Learn Res 9:1871–1874. [Google Scholar]
  14. Fechner G. (1966) Elements of psychophysics. Vol. I. Elem Psychophys Vol I. [Google Scholar]
  15. Fegen D, Buchsbaum BR, D'Esposito M (2015) The effect of rehearsal rate and memory load on verbal working memory. Neuroimage 105:120–131. 10.1016/j.neuroimage.2014.10.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Goldman-Rakic PS. (1995) Cellular basis of working memory review. Neuron 14:477–485. 10.1016/0896-6273(95)90304-6 [DOI] [PubMed] [Google Scholar]
  17. Haynes JD. (2015) A primer on pattern-based approaches to fMRI: principles, pitfalls, and perspectives. Neuron 87:257–270. 10.1016/j.neuron.2015.05.025 [DOI] [PubMed] [Google Scholar]
  18. Hebart MN, Görgen K, Haynes JD (2015) The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data. Front Neuroinform 8:88. 10.3389/fninf.2014.00088 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hernández A, Zainos A, Romo R (2000) Neuronal correlates of sensory discrimination in the somatosensory cortex. Proc Natl Acad Sci U S A 97:6191–6196. 10.1073/pnas.120018597 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hernández A, Nácher V, Luna R, Zainos A, Lemus L, Alvarez M, Vázquez Y, Camarillo L, Romo R (2010) Decoding a perceptual decision process across cortex. Neuron 66:300–314. 10.1016/j.neuron.2010.03.031 [DOI] [PubMed] [Google Scholar]
  21. Jerde TA, Merriam EP, Riggall AC, Hedges JH, Curtis CE (2012) Prioritized maps of space in human frontoparietal cortex. J Neurosci 32:17382–17390. 10.1523/JNEUROSCI.3810-12.2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kahnt T, Heinzle J, Park SQ, Haynes JD (2011) Decoding different roles for vmPFC and dlPFC in multi-attribute decision making. Neuroimage 56:709–715. 10.1016/j.neuroimage.2010.05.058 [DOI] [PubMed] [Google Scholar]
  23. Kosslyn SM. (2005) Mental images and the brain. Cogn Neuropsychol 22:333–347. 10.1080/02643290442000130 [DOI] [PubMed] [Google Scholar]
  24. Kriegeskorte N, Goebel R, Bandettini P (2006) Information-based functional brain mapping. Proc Natl Acad Sci U S A 103:3863–3868. 10.1073/pnas.0600244103 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Larocque JJ, Lewis-Peacock JA, Postle BR (2014) Multiple neural states of representation in short-term memory? It's a matter of attention. Front Hum Neurosci 8:5. 10.3389/fnhum.2014.00005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lee SH, Baker CI (2016) Multi-voxel decoding and the topography of maintained information during visual working memory. Front Syst Neurosci 10:2. 10.3389/fnsys.2016.00002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Lee SH, Kravitz DJ, Baker CI (2013) Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nat Neurosci 16:997–999. 10.1038/nn.3452 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lewis-Peacock JA, Drysdale AT, Postle BR (2015) Neural evidence for the flexible control of mental representations. Cereb Cortex 25:3303–3313. 10.1093/cercor/bhu130 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Myers NE, Stokes MG, Nobre AC (2017) Prioritizing information during working memory: beyond sustained internal attention. Trends Cogn Sci 21:449–461. 10.1016/j.tics.2017.03.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Nieder A. (2013) Coding of abstract quantity by “number neurons” of the primate brain. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 199:1–16. 10.1007/s00359-012-0763-9 [DOI] [PubMed] [Google Scholar]
  31. Nieder A, Dehaene S (2009) Representation of number in the brain. Annu Rev Neurosci 32:185–208. 10.1146/annurev.neuro.051508.135550 [DOI] [PubMed] [Google Scholar]
  32. Norman KA, Polyn SM, Detre GJ, Haxby JV (2006) Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn Sci 10:424–430. 10.1016/j.tics.2006.07.005 [DOI] [PubMed] [Google Scholar]
  33. Oldfield RC. (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9:97–113. 10.1016/0028-3932(71)90067-4 [DOI] [PubMed] [Google Scholar]
  34. Pasternak T, Greenlee MW (2005) Working memory in primate sensory systems. Nat Rev Neurosci 6:97–107. 10.1038/nrn1603 [DOI] [PubMed] [Google Scholar]
  35. Pinel P, Piazza M, Le Bihan D, Dehaene S (2004) Distributed and overlapping cerebral representations of number, size, and luminance during comparative judgments. Neuron 41:983–993. 10.1016/S0896-6273(04)00107-2 [DOI] [PubMed] [Google Scholar]
  36. Postle BR. (2015) The cognitive neuroscience of visual short-term memory. Curr Opin Behav Sci 1:40–46. 10.1016/j.cobeha.2014.08.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Preuschhof C, Heekeren HR, Taskin B, Schubert T, Villringer A (2006) Neural correlates of vibrotactile working memory in the human brain. J Neurosci 26:13231–13239. 10.1523/JNEUROSCI.2767-06.2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Riggall AC, Postle BR (2012) The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging. J Neurosci 32:12990–12998. 10.1523/JNEUROSCI.1892-12.2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Riley MR, Constantinidis C (2016) Role of prefrontal persistent activity in working memory. Front Syst Neurosci 9:181. 10.3389/fnsys.2015.00181 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Romo R, de Lafuente V (2013) Conversion of sensory signals into perceptual decisions. Prog Neurobiol 103:41–75. 10.1016/j.pneurobio.2012.03.007 [DOI] [PubMed] [Google Scholar]
  41. Romo R, Salinas E (2001) Touch and go: decision-making mechanisms in somatosensation. Annu Rev Neurosci 24:107–137. 10.1146/annurev.neuro.24.1.107 [DOI] [PubMed] [Google Scholar]
  42. Romo R, Salinas E (2003) Flutter discrimination: neural codes, perception, memory and decision making. Nat Rev Neurosci 4:203–218. 10.1038/nrn1058 [DOI] [PubMed] [Google Scholar]
  43. Romo R, Brody CD, Hernández A, Lemus L (1999) Neuronal correlates of parametric working memory in the prefrontal cortex. Nature 399:470–473. 10.1038/20939 [DOI] [PubMed] [Google Scholar]
  44. Romo R, Hernández A, Zainos A, Lemus L, Brody CD (2002) Neuronal correlates of decision-making in secondary somatosensory cortex. Nat Neurosci 5:1217–1225. 10.1038/nn950 [DOI] [PubMed] [Google Scholar]
  45. Romo R, Hernández A, Zainos A (2004) Neuronal correlates of a perceptual decision in ventral premotor cortex. Neuron 41:165–173. 10.1016/S0896-6273(03)00817-1 [DOI] [PubMed] [Google Scholar]
  46. Romo R, Lemus L, de Lafuente V (2012) Sense, memory, and decision-making in the somatosensory cortical network. Curr Opin Neurobiol 22:914–919. 10.1016/j.conb.2012.08.002 [DOI] [PubMed] [Google Scholar]
  47. Schlegel A, Kohler PJ, Fogelson SV, Alexander P, Konuthula D, Tse PU (2013) Network structure and dynamics of the mental workspace. Proc Natl Acad Sci U S A 110:16277–16282. 10.1073/pnas.1311149110 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Schmidt TT, Ostwald D, Blankenburg F (2014) Imaging tactile imagery: Changes in brain connectivity support perceptual grounding of mental images in primary sensory cortices. Neuroimage 98:216–224. 10.1016/j.neuroimage.2014.05.014 [DOI] [PubMed] [Google Scholar]
  49. Spitzer B, Blankenburg F (2012) Supramodal parametric working memory processing in humans. J Neurosci 32:3287–3295. 10.1523/JNEUROSCI.5280-11.2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Spitzer B, Gloel M, Schmidt TT, Blankenburg F (2014a) Working memory coding of analog stimulus properties in the human prefrontal cortex. Cereb Cortex 24:2229–2236. 10.1093/cercor/bht084 [DOI] [PubMed] [Google Scholar]
  51. Spitzer B, Fleck S, Blankenburg F (2014b) Parametric alpha- and beta-band signatures of supramodal numerosity information in human working memory. J Neurosci 34:4293–4302. 10.1523/JNEUROSCI.4580-13.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Spitzer B, Goltz D, Wacker E, Auksztulewicz R, Blankenburg F (2014b) Maintenance and manipulation of somatosensory information in ventrolateral prefrontal cortex. Hum Brain Mapp 35:2412–2423. 10.1002/hbm.22337 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Sreenivasan KK, Curtis CE, D'Esposito M (2014) Revisiting the role of persistent neural activity during working memory. Trends Cogn Sci 18:82–89. 10.1016/j.tics.2013.12.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Tong F. (2013) Imagery and visual working memory: one and the same? Trends Cogn Sci 17:489–490. 10.1016/j.tics.2013.08.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Walsh V. (2003) A theory of magnitude: common cortical metrics of time, space and quantity. Trends Cogn Sci 7:483–488. 10.1016/j.tics.2003.09.002 [DOI] [PubMed] [Google Scholar]
  56. Zhou YD, Fuster JM (1996) Mnemonic neuronal activity in somatosensory cortex. Proc Natl Acad Sci U S A 93:10533–10537. 10.1073/pnas.93.19.10533 [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Zhou YD, Fuster JM (2000) Visuo-tactile cross-modal associations in cortical somatosensory cells. Proc Natl Acad Sci U S A 97:9777–9782. 10.1073/pnas.97.17.9777 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES