Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Oct 1.
Published in final edited form as: Neuroimage. 2008 Jun 10;42(4):1678–1685. doi: 10.1016/j.neuroimage.2008.05.057

Neurodevelopmental Changes in Verbal Working Memory Load-Dependency: An fMRI Investigation

Elizabeth D O’Hare 1,2, Lisa H Lu 1,4, Suzanne M Houston 1, Susan Y Bookheimer 2,3, Elizabeth R Sowell 1,2
PMCID: PMC2570587  NIHMSID: NIHMS70468  PMID: 18586110

Abstract

Development of working memory (WM) aptitude parallels structural changes in the frontal-parietal association cortices important for performance within this cognitive domain. The cerebellum has been proposed to function in support of the postulated phonological loop component of verbal WM, and along with frontal and parietal cortices, has been shown to exhibit linear WM load-dependent activation in adults. It is not known if these kinds of WM load-dependent relationships exist for cerebro-cerebellar networks in developmental populations, and whether there are age-related changes in the nature of load-dependency between childhood, adolescence, and adulthood. The present study used fMRI and a verbal Sternberg WM task with three load levels to investigate developmental changes in WM load-dependent cerebro-cerebellar activation in a sample of 30 children, adolescents, and young adults between the ages of 7 and 28. The neural substrates of linear load-dependency were found to change with age. Among adolescents and adults, frontal, parietal and cerebellar regions showed linear load-dependency, or increasing activation under conditions of increasing WM load. In contrast, children recruited only left ventral prefrontal cortex in response to increasing WM load. These results demonstrate that, while children, adolescents, and young adults activate similar cerebro-cerebellar verbal working memory networks, the extent to which they rely on parietal and cerebellar regions in response to increasing task difficulty changes significantly between childhood and adolescence.

Introduction

Working memory (WM) is a psychological construct thought to represent the transient storage and manipulation of information that is required for concurrent cognitive computation and behavioral guidance (Baddeley, 1986, Baddeley, 1992, Smith and Jonides, 1998, Smith et al., 1998, Baddeley, 2003). Behavioral studies have shown that the development of working memory follows a protracted course of maturation reaching adult-like levels around middle adolescence (Luciana and Nelson, 1998, Gathercole, 1999, Luna et al., 2004). Development of WM ability parallels the lengthy course of structural changes in grey matter (Giedd et al., 1999, Sowell et al., 1999, Sowell et al., 2003, Sowell et al., 2004) and white matter (Klingberg, 2002, Nagy, 2004) in the frontal-parietal association cortices important for the performance of WM.

Studies of verbal WM in adults have typically used variations of the classic Sternberg delayed response task (Sternberg, 1966) to define the functional specificity of prefrontal, parietal, and cerebellar regions for WM operations, because these tasks are especially effective at dissociating the encoding, maintenance, manipulation, and retrieval components of WM (Rypma, 2006), and have been used within parametric designs to characterize the load-dependency of the WM network. The observation of linear increases in activation with increasing WM memory load in frontal and parietal regions suggests that these regions track task difficulty, and imply that increases in WM demands result on frontal-parietal networks “working harder” to perform a more difficult task (Gould et al., 2003). However, nonlinear relationships between activation and task difficulty could reflect a lack of distinction between some but not other load levels, or could also indicate that different brain regions are differentially recruited in response to increasing task difficulty (Gould et al., 2003). Systematic analyses of the effects of WM load on frontal, parietal (Callicott et al., 1999) and cerebellar (Kirschen et al., 2005) activity in adults have demonstrated that these regions show linear responses to increasing WM load, however, no studies have investigated potential developmental changes in the dynamics of load-dependency.

Several fMRI studies have examined developmental changes in visual-spatial WM (VSWM). These studies have demonstrated that children recruit superior frontal and parietal cortical regions during task performance (Thomas et al., 1999, Nelson et al., 2000, Klingberg, 2002, Kwon et al., 2002, Olesen, 2003), a pattern of activation similar to that found in adults, where numerous investigations have demonstrated activation in dorsal lateral prefrontal cortex (DLPFC), ventral lateral prefrontal cortex (VLPFC), premotor cortex, and posterior parietal cortex (Courtney et al., 1998, Postle et al., 2000, Rowe et al., 2000, Pessoa et al., 2002, Curtis et al., 2004). Studies directly comparing children, adolescents, and adults have demonstrated that increasing age correlates with increasing activation in this VSWM network (Klingberg, 2002, Kwon et al., 2002). Recent reports have emphasized developmental shifts in the networks of brain regions that children, adolescents, and adults rely upon during performance of VSWM (Scherf et al., 2006) and categorical object-based WM (Ciesielski et al., 2006). Specifically, children and adolescents have been shown to recruit regions such as the striatum and cerebellum during the performance of VSWM (Scherf et al., 2006) and complex object WM (Ciesielski et al., 2006). These regions have been conceptualized to function in support of WM processes (Scherf et al., 2006). In contrast, adults appear to recruit prefrontal and parietal regions for performance of the same tasks, suggesting reliance on brain regions that have been conceptualized as highly critical for WM performance (Scherf et al., 2006). These developmental changes in brain network recruitment were associated with only modest behavioral improvements, possibly suggesting a distinction between brain regions that are sufficient for task performance (striatal and cerebellar regions) and those that are performance enhancing (association corticies).

In contrast to the number of studies that have investigated developmental changes in VSWM and object WM, to our knowledge, only two fMRI studies (Casey et al., 1995; Crone et al., 2006) have examined verbal WM in children, and neither of these examined potential shifts in cognitive networks as increasing WM load placed higher demands on the system. Casey and colleagues reported activation of inferior and middle frontal gyri during the performance of a verbal WM task among children aged 9–11 years (Casey et al., 1995). These results mirrored those observed in adults performing an identical version of the task (Cohen et al., 1994), suggesting, that like VSWM, both children and adults recruit similar brain regions in response to verbal WM challenge. More recently, Crone and colleagues observed that while children and adults had similar activation profiles within the VLPFC during the maintenance phase of a object WM task (objects were chosen to be highly verbalizable, thus tapping verbal maintenance processes), children failed to recruit right DLPFC and bilateral superior parietal cortex during a delay period containing an additional manipulation demand (Crone et al., 2006). This suggests that frontal-parietal WM networks are less functionally developed in children and raises the possibility that differences in activation between children and adults may not be evident except under conditions of increasing WM demand.

Given findings demonstrating a developmental shift from reliance on brain regions thought to support task performance to recruitment of highly task-specific networks for visual-spatial and object WM (Scherf et al., 2006; Ciesielski et al., 2006), there may be age-related changes in the brain regions recruited in response to increasing verbal WM load. To date, however, no studies have conducted systematic analyses of developmental changes in neocortical and cerebellar verbal WM load-dependency. Thus, the purpose of the present study was to investigate the contribution of cerebro-cerebellar networks to verbal working memory in children and adolescents, and to characterize developmental changes in the WM load-dependency of this network by examining linear and quadratic components of WM load-dependent brain activation across childhood, adolescence, and adulthood. Based on the demonstration of linear load-dependency in cerebro-cerebellar networks in adults on verbal WM tasks (Braver et al., 1997; Kirschen et al., 2005), and recent findings suggesting less developed posterior cortical representations of verbal WM among children and adolescents (Crone et al., 2006), we predicted that linear load-dependency in older participants would be characterized by increased recruitment of posterior cortical regions important for verbal WM maintenance.

Materials and Methods

Participants

Thirty-two typically developing individuals participated in this study. Participants included 14 children (aged 7–10, mean=9.0, SD=1.35), 10 adolescents (aged 11–15, mean=13.6, SD=3.29), and 8 young adults (aged 20–28, mean=24.0, SD=2.93) (see Table 1). Because there is no definitive age range that defines adolescence (reviewed in Spear, 2000), we used age groupings roughly similar to those used in previous reports (Sowell et al., 2003) in an effort to equate the numbers of subjects in each group and to allow for comparability across studies. One child subject was excluded due to excessive head motion. A second child subject failed to make behavioral responses during scanning and was also excluded. All subjects were screened for neurological impairments, psychiatric illness, history of learning disability, and developmental delay. All participants (and their parents if the participant was under age 18) gave their informed assent/consent to participate in the study, which was approved by the Institutional Review Board of the University of California, Los Angeles.

Table 1.

Demographics and Behavioral Performance on the Verbal Sternberg Working Memory Task

Children (n=12) Adolescents (n=10) Young Adults (n=8)
Age 9.0 (1.4) 13.6 (1.5) 23.8 (3.0)
Female/Male 6/6 6/4 4/4
FSIQ 116 (18) 106 (15) 113 (17)
Low Accuracy 96 (7) 95 (6) 99 (2)
Medium Accuracy 93 (16) 92 (14) 98 (3)
High Accuracy 74 (16) 77 (18) 82 (13)
Low RT 0.86 (0.18) 0.71 (0.18) 0.66 (0.10)
Medium RT 0.96 (0.17) 0.84 (0.13) 0.77 (0.10)
High RT 0.91 (0.17) 0.91 (0.16) 0.85 (0.11)

Values presented as mean (SD). Accuracy given as percentage correct. Reaction Time (RT) in seconds.

Stimuli and Task Parameters

Subjects performed a parametric verbal Sternberg working memory task, similar to that used by Desmond and colleagues to examine cerebro-cerebellar networks in adults (Desmond et al., 2003, Chen and Desmond, 2005). Stimuli consisted of a horizontal array of uppercase consonant letters and pound (#) symbols. There were three types of arrays: high WM load (6 letters), medium WM load (3 letters and three “#” symbols), and low WM load (one letter and five “#” symbols). For the medium and low load stimuli, letter array position was counterbalanced across all possible array positions. Within the task, the overall appearance of a given letter was counterbalanced, as was the order of presentation of load conditions. Stimuli were generated using MacStim 3.0 psychological experimentation software (CogState, West Melbourne, Victoria, Australia) and were visually presented to subjects in the scanner using magnet-compatible 3-D goggles (Resonance Technology, Northridge, CA).

Subjects were instructed to remember the letters in each array. Letter arrays were presented for 1.5 seconds, followed by a three second delay (see Figure 1). Following this delay, a lowercase letter probe stimulus was presented for 1.5s. For all load conditions, the probe stimulus matched the previously presented letter(s) on half of the trials. Position of the probe stimulus within the array was counterbalanced. Subjects were instructed to respond by pushing a button with their index finger if the probe stimulus matched an array letter and to push a button with their middle finger when the probe stimulus did not match an array letter. The task consisted of four blocks of each load condition (12 WM blocks total), with four trials in each 24-second block. Thirteen 12-second rest blocks, in which subjects fixated on a cross hair, alternated with the WM blocks, for a total task time of 7 minutes, 24 seconds.

Figure 1.

Figure 1

Verbal Sternberg Working Memory Task. The correct responses for the three items presented in the figure are “yes” for the high load and low load examples, and “no” for the medium load example.

Subjects were trained on the task prior to the start of the scanning session. Training included a verbal description of the task and then two practice runs, each containing 5 trials. All subjects were able to perform the task prior to the start of the scanning session. Once the subject was in the scanner, task instructions were reviewed again before the start of the task.

Data acquisition

Functional imaging data were collected on the UCLA Division of Brain Mapping’s research-dedicated 3 Tesla Siemens Allegra head-only magnet. Multi-slice echo-planar imaging (EPI) was used with a gradient echo EPI sequence. We used a TR of 3 sec, with a TE of 25. Slice thickness was 3 mm with a 1 mm skip, 36 total slices, with 64x64 pixels yielding a resolution in-plane of 3mm with whole-brain acquisition. A high-resolution T2-weighted EPI volume was collected in the anterior commissure-posterior commissure plane (slice thickness=3mm, 36 total axial slices covering the entire brain with 1mm gaps between slices, TR=5000ms, TE=33ms, flip angle=90 degrees, matrix size 128 x 128 with 1.5 x 1.5 mm in-plane voxel dimensions), coplanar with the functional scan, to allow for spatial registration of each subject’s data into a standard coordinate space.

Image and Statistical Analysis

Functional imaging data were analyzed using FSL (FMRIB’s Software Library, http://www.fmrib.ox.ac.uk/fsl/index.html, (Smith et al., 2004). Data were corrected for possible motion artifacts by co-registering each BOLD image in the time series to the middle volume in the series using a 6-parameter rigid-body transformation. Volumes that had more than 2mm of head motion were excluded and the remaining time series data were analyzed. For a given subject, if excessive head motion occurred on more than 17 volumes within each task condition, that subject was excluded. Slice timing correction was applied to correct each voxel’s time series given that slices were acquired in an interleaved fashion. Data were spatially smoothed using a 6 mm (FWHM) Gaussian kernel and a high pass filter cutoff period of 50 seconds was imposed. Finally, all data were registered in a two-step process. First, each subject’s EPI data were registered to their own T2-weighted structural image with a 6-parameter transformation for within subject analyses, and then to the MNI-152 standard space template with a 12-parameter transformation for group averaging (Jenkinson and Smith, 2001, Jenkinson et al., 2002).

Single subject analyses were carried out using FMRIB’s fMRI Expert Analysis Tool (FEAT, version 5.63). In these analyses, low, medium, and high WM loads were modeled separately. The hemodynamic response function (HRF) was specified as a gamma variate function (mean lag of 6 seconds) and convolved with the modeled components. Statistical analysis of the time series data was carried out using FMRIB’s Improved Linear Model (FILM). A voxel-wise general linear model was applied so that each voxel’s time course was individually fit to the model with local autocorrelation correction (Woolrich et al., 2001). Overall WM responses were determined by comparing WM activity (collapsed across loads) with rest. Orthogonal contrasts were then used to compute linear and nonlinear response functions across load conditions. Linear functions were modeled as [−1 0 1] and identified voxels that showed increasing activity with increasing WM load. Nonlinear functions were modeled as [−1 2 −1] to investigate the extent to which the change between easy and medium loads differed from the change between medium and hard loads. Z statistic images were thresholded using clusters determined by Z > 1.7 and a (corrected) cluster significance threshold of p = 0.05.

Higher-level group analysis was carried out using FLAME (FMRIB’s Local Analysis of Mixed Effects). FLAME takes each subject’s time series data and models all subjects in each group (children, adolescents, young adults). Thus, group average responses for the linear and nonlinear contrasts were derived. Thus, a one-way analysis of variance (ANOVA) with the factor of age group was used to examine group differences in activation for these contrasts.

Results

Demographic and Behavioral data

Demographic, accuracy (percent correct) and response time (RT) data for all subjects are presented in Table 1. Age groups did not differ significantly on IQ [F(29)=1.124, NS], and a chi-square analysis confirmed that gender distributions were equal between groups [χ2 =0.268, NS]. A repeated measures ANOVA examining the effects of WM load level on accuracy found no interaction of age and load [F(4,29)=0.163, p=0.957], and no main effect of age [F(2,29)=1.549, p=0.219]. There was a main effect of load [F(2,29)=20.878, p<0.001]. Post hoc tests revealed that accuracy for high trials was significantly less than for either medium (p<0.001) or low load trials (p<0.001). There was no significant difference for accuracy between low and medium trials. A repeated measures ANOVA examining the effects of WM load on RT found no interaction of RT and load [F(4,29)=0.951, p=0.439]. There was a main effect of age [F(2,29)=7.260, p=0.001]. Post hoc tests revealed that adults had significantly faster RTs than children (p=0.002), but not adolescents (p=0.092). There was also a main effect of load [F(2,29)=7.126, p=0.001]. Post hoc tests revealed that RT for low trials was significantly faster than for either medium (p=0.029) or high trials (p=0.006).

Functional Imaging Data

Talairach coordinates, local Z statistic values, and Brodmann areas for the maximum Z value within each cluster are presented for all analyses in Tables 2 through 5. Anatomical locations of neocortical activations were determined using Talairach and Tournoux’s stereotaxic atlas (Talairach, 1988). Cerebellar anatomical locations were determined using Schmahmann’s atlas of the human cerebellum (Schmahmann et al., 2000). Note that space limitations led to rendering of only the most relevant slice views for all analyses described below, and some regions listed in the tables of cluster maximums may not be shown in the figures.

Table 2.

CHILDREN: Cluster level analyses (P<0.05) for group average activation maps. Talairach coordinates for the maximum Z-value within each cluster are presented.

Linear Load-Dependent Activation
Structure Hem. BA x y z Local Max Z statistic
Inferior Frontal Gyrus L 47 −30 30 4 3.42
Inferior Frontal Gyrus L 9 −34 8 22 3.4
Inferior Frontal Gyrus L 9 −46 4 22 3.13
Middle Frontal Gyrus L 9 −34 12 28 3.1
Superior Frontal Gyrus L 10 −30 50 22 3.13
Precentral Gyrus L 6 −46 0 28 3.11

Table 5.

Cluster level analyses (P<0.05) for the group ANOVA. Talairach coordinates for the maximum Z-value within each cluster are presented.

Linear Activation Adolescents > Children
Structure Hem. BA x y z Local Max Z statistic
Posterior Cerebellum, IX L N/A −20 −54 −40 3.56
Posterior Cerebellum, IX L N/A −26 −52 −42 3.44
Posterior Cerebellum, IX R N/A 28 −32 −38 3.46
Parahippocampal Gyrus R 19 30 −46 0 3.50
Thalamus L N/A −14 −18 6 3.71
Linear Activation Young Adults >Children
Structure Hem. BA x y z Local Max Z statistic
Inferior Parietal Lobule L 40 −52 −38 44 3.48
Inferior Parietal Lobule L 40 −36 −44 36 3.05
Superior Parietal Lobule L 7 −26 −66 56 4.3
Precuneus L 7 −18 −74 52 3.42
Precuneus L 7 −8 −82 50 3.02
Precuneus L 7 −22 −70 36 2.93
Posterior Cerebellum, IX R N/A 6 −80 −36 3.67
Posterior Cerebellum, IX R N/A 30 −62 −32 3.59
Posterior Cerebellum, IX R N/A 36 −54 −36 3.4
Posterior Cerebellum, IX L N/A −10 −68 −34 3.62
Posterior Cerebellum, VIIIA/VIIIB L N/A −26 −64 −28 3.33
Posterior Cerebellum, IV/V L N/A −22 −48 −22 3.18
Linear Activation Adolescents > Young Adults
Structure Hem. BA x y z Local Max Z statistic
Superior Frontal Gyrus R 6 6 −2 66 3.56
Middle Frontal Gyrus R 6 4 −6 58 3.57
Middle Frontal Gyrus L 6 0 −10 54 3.77
Middle Frontal Gyrus L 6 −10 −8 52 3.50
Postcentral Gyrus L 1 −56 −18 50 4.47
Postcentral Gyrus L 3 −52 −14 48 4.03
Nonlinear Activation Young Adults > Children
Insula R 13 30 −16 22 2.85
Postcentral Gyrus R 3 24 −28 58 2.91
Globus Pallidus R N/A 16 −6 −4 2.85
Globus Pallidus R N/A 22 −14 4 2.82
Claustrum R N/A 26 −10 20 3.11

Linear and Nonlinear Verbal WM Load Effects: Children

Among children, voxels exhibiting linear load-dependency, or increasing activation under conditions of increasing WM load were limited to left ventral prefrontal regions (Table 2 and Figure 2A). No regions reached significance for nonlinear responses to increasing WM load in children.

Figure 2.

Figure 2

Surface renderings and coronal sections displaying linear and nonlinear load-dependent cerebro-cerebellar activation during verbal WM for children (A), adolescents (B), and young adults (C). The left-hand panel depicts regions where significant (Z > 1.7 and a corrected cluster significance threshold of p = 0.05) linear increases in activation were observed with increasing WM load for the three age groups. Regions in red correspond to those voxels that displayed a linear response to increasing WM load. The right-hand panel depicts regions where significant (Z > 1.7 and a corrected cluster significance threshold of p = 0.05) nonlinear responses were observed for the three age groups. Regions in blue correspond to those voxels that displayed an inverse U-shaped quadratic response to increasing WM load. Voxels are displayed on the brain surface only if they intersect with the surface. Axial and coronal slices show frontal, cerebellar, parietal, and precuneus activation. Y values correspond to Talairach standard space coordinates.

Linear and Nonlinear Verbal WM Load Effects: Adolescents

As predicted, among adolescents, linear load-dependency was observed bilaterally in ventral lateral prefrontal cortex, left middle and superior frontal gyri, and left precentral gyri (Table 3 and Figure 2B). Additional regions showing linear load-dependency were observed in visual cortices, including left middle and inferior occipital gyri, and the fusiform gyrus. Linear load-dependency was also observed bilaterally in the superior cerebellar hemispheres (lobule VI/Crus I), in the right inferior cerebellum (VIIA/VIIB), and in inferior lobule IX.

Table 3.

ADOLESCENTS: Cluster level analyses (P<0.05) for group average activation. Talairach coordinates for the maximum Z-value within each cluster are presented.

Linear Load-Dependent Activation
Structure Hem. BA x y z Local Max Z statistic
Superior Frontal Gyrus L 6 −4 14 50 4.78
Middle Frontal Gyrus L 32 0 10 44 4.72
Middle Frontal Gyrus R 10 40 36 24 4.27
Middle Frontal Gyrus R 9 30 44 34 3.83
Inferior Frontal Gyrus R 47 38 18 −4 3.78
Inferior Frontal Gyrus R 47 42 16 −4 3.78
Precentral Gyrus L 6 −44 0 32 5.02
Precentral Gyrus L 4 −48 −8 48 4.66
Precentral Gyrus L 4 −50 −6 52 4.64
Cingulate Gyrus L 32 −8 22 32 4.75
Insula R 13 34 22 10 4.15
Insula R 47 32 24 2 3.98
Nonlinear Load-Dependent Activation
Precuneus R 7 2 −60 32 3.65
Precuneus L 31 0 −70 26 3.4
Precuneus L 31 0 −74 28 3.29
Precuneus L 31 −8 −66 28 3.29
Cuneus L 19 −6 −80 30 3.35
Superior Occipital Gyrus L 19 −36 −78 32 3.21
Anterior Cingulate R 32 4 34 −6 3.76
Medial Frontal Gyrus L 10 −2 62 20 3.33
Medial Frontal Gyrus L 10 −4 64 16 3.29
Medial Frontal Gyrus L 11 −2 62 −12 3.19
Superior Frontal Gyrus R 10 6 62 −10 3.22
Superior Frontal Gyrus R 10 4 60 −2 3.12

Figure 2B (right-side) shows statistical maps of regions displaying significant inverted U-shaped quadratic effects (i.e., the response to the medium load is greater than that to low and/or high loads) for adolescents. Regions displaying this pattern were found in medial prefrontal and parietal areas, and the left angular gyrus.

Linear and Nonlinear Verbal WM Load Effects: Young Adults

Among adults, voxels exhibiting linear load-dependency, or increasing activation under conditions of increasing WM load, were found bilaterally in middle frontal and inferior frontal gyri and the left superior parietal lobule (Table 4 and Figure 2C). Linear load-dependency was also observed in the cerebellum. The bilateral superior cerebellar hemispheres (lobule VI/Crus I) and right inferior cerebellum (VIIA/VIIB) demonstrated increasing activation with increasing WM load. Adults did not display any regions showing nonlinear responses to increasing WM load (right side of Figure 2C).

Table 4.

YOUNG ADULTS: Cluster level analyses (P<0.05) for group average activation. Talairach coordinates for the maximum Z-value within each cluster are presented.

Linear Load-Dependent Activation
Structure Hem. BA x y z Local Max Z statistic
Superior Frontal Gyrus R 9 36 36 26 4.55
Middle Frontal Gyrus R 9 34 40 26 4.4
Middle Frontal Gyrus R 10 32 40 22 4.37
Inferior Frontal Gyrus L 9 −54 6 28 4.37
Medial Frontal Gyrus L 8 −2 20 46 4.42
Precentral Gyrus L 6 −44 0 32 4.51
Superior Parietal Lobule L 7 −28 −62 48 4.92
Superior Parietal Lobule L 7 −26 −66 56 4.43
Superior Parietal Lobule L 7 −26 −68 42 3.47
Inferior Parietal Lobule L 40 −52 −38 44 4.04
Inferior Parietal Lobule L 40 −36 −44 36 3.43
Precuneus L 7 −22 −70 36 3.46
Posterior Cerebellum, IX R N/A 30 −62 −32 5.3
Posterior Cerebellum, IX R N/A 6 −80 −38 4.52
Posterior Cerebellum, IX R N/A 20 −76 −24 4.08
Posterior Cerebellum, IX R N/A 42 −60 −32 4.07

Age Group Differences in WM Load-Dependency

Group differences were observed for linear load-dependent relationships. As predicted, linear load-dependent functional activity was significantly increased among adolescents and young adults, relative to children, in the left superior parietal lobule (BA7), the right superior cerebellum (VI/Crus I), and the right inferior cerebellum (lobules VIIA/VIIB) (Table 5 and Figure 3). Relative to young adults, adolescents showed increased activation, in premotor and supplementary motor areas, under conditions of increasing WM load. Nonlinear load-dependent functional activity (the response to medium load trials was greater than to either low or high load trials) was significantly increased among young adults, relative to children in right ventral prefrontal cortex (Table 5 and Figure 3).

Figure 3.

Figure 3

Surface renderings and coronal sections displaying age group differences for linear (panels A, B, and C) and nonlinear (panel D) load-dependent cerebro-cerebellar activation. Shown here are regions of significant (Z > 1.7 and a corrected cluster significance threshold of p = 0.05) activation for linear and nonlinear load-dependency. Panel A depicts regions where adolescent subjects display greater linear load-dependent activation relative to children. Panel B depicts region where young adults display greater linear load-dependent activation relative to children. Panel C depicts regions where adolescents display greater linear load-dependent activation, relative to young adults. Panel D depicts regions where young adults display greater nonlinear load-dependent activation, relative to children. Voxels are displayed on the brain surface only if they intersect with the surface. Y values correspond to Talairach standard space coordinates.

Discussion

The results of the present study demonstrate that children, adolescents, and adults engage cerebro-cerebellar networks in response to verbal WM challenge. Qualitatively, differences among age groups in WM load-dependency suggest that with increasing age the number of brain regions showing linear load-dependency expands. Specifically, while children display increasing activation in response to increasing WM load in left ventral prefrontal cortex only, adolescents and young adults show this pattern in right prefrontal, left parietal, superior cerebellum bilaterally (VI/Crus I), and right inferior cerebellum (VIIA/VIIB) regions as well. Age group effects for linear load-dependency were found to be statistically significant. Specifically, adolescents and young adults showed increased linear load-dependent activation in the left superior parietal lobule and right superior cerebellum when compared to the children. These data extend findings from adult studies by showing that the neural substrates of linear load-dependency change from childhood to adolescence, and implicate increasing reliance on posterior parietal and cerebellar regions as crucial for this process.

The observation of load-dependent increases in bilateral middle and inferior frontal gyri, the left superior parietal lobule, right inferior cerebellar regions (VIIA/VIIB), and bilateral superior cerebellar hemispheres (VI/Crus I) among the adults studied here replicates previous findings within the prefrontal cortex (Braver et al., 1997, Rypma et al., 1999) and cerebellum (Desmond et al., 1997, Kirschen et al., 2005). We extend these observations by demonstrating that linear load effects in frontal, parietal, and cerebellar regions, while present on a limited basis among children, are fully present in adolescence. Consistent with Desmond’s model of cerebellar function in verbal WM, we postulate that load-dependent increases in activation in bilateral superior cerebellar hemispheres (VI/Crus I) reflects increased input from ventral frontal regions involved in the articulatory control system of the phonological loop. Further, we speculate that load-dependent increases in right inferior cerebellar regions (VIIA/VIIB) reflect increased input from parietal regions important for phonological storage (Desmond et al., 1997; Kirschen et al., 2005; Chen and Desmond, 2005a). This cortical-cerebellar network relies on anatomical connectivity between these regions (Schmahmann, 1991, Schmahmann, 1996, Schmahmann and Pandya, 1997). Thus in our study, parametric increases in activation in cerebellar regions are interpreted to reflect increased WM processing demands.

The increased cerebellar activation among adolescents and adults relative to children during verbal WM reported here is inconsistent with previous studies that have reported decreasing cerebellar activation with age during the performance of object WM (Ciesielski et al., 2006) and VSWM (Scherf et al., 2006). It is important to note however, that the cerebellum is thought to play a crucial role in covert speech and phonological storage (Desmond & Fiez, 1998; Chen & Desmond, 2005a; Chen and Desmond, 2005b), processes that are presumably more involved in the maintenance of verbal information than that of the visual information presented in other studies. Thus, our findings provide evidence of developmental changes in cerebellar involvement in verbal WM that are consistent with known developmental improvements in behavioral measures of phonological storage and covert speech (Gathercole, 1999). These changes may be functionally distinct from developmental changes in cerebellar involvement in other types of WM and suggest a conceptualization of the role of the cerebellum in verbal WM development. It is also important to note that the previous studies reporting decreasing cerebellar activation with increasing age were examining the main effects of cerebellar activation for object (Ciesielski et al., 2006) and VSWM (Scherf et al., 2006). In contrast, the present study examined linear and nonlinear changes in brain activation under conditions of increasing WM load. It is also possible that the differences between the results of the present study and those of previous investigations are related to the use of differences contrasts. Future studies comparing developmental changes in cerebellar involvement in different types of WM within the same subjects will be needed to test these hypotheses.

Consistent with the role of the ventral prefrontal cortex in WM maintenance processes (D'Esposito et al., 1999, D'Esposito et al., 2000, Veltman et al., 2003), we observed increasing activation with increasing WM load in ventral prefrontal cortex within all age groups studied here, suggesting that WM maintenance systems are mature by age 7. In contrast, parietal and cerebellar involvement in linear load-dependency was observed only in the adolescent and young adult age groups, suggesting continued development of those WM processes supported by parietal and cerebellar regions between childhood and adolescence. Although there is some disagreement about its precise location (Becker et al., 1999), recent neuropsychological (for a review see (Muller and Knight, 2006) and neuroimaging (Ravizza et al., 2004) work has implicated the parietal cortex as the site of the phonological storage component of verbal WM and suggests that the dorsal parietal cortex is particularly sensitive to manipulations of verbal WM load. As noted above, several studies have described a role for the cerebellum in the phonological loop component of verbal WM (Desmond et al., 1997; Chen et al., 2005; Kirschen et al., 2005). Thus, our findings suggest that developmental differences in verbal WM may relate specifically to changes in the phonological storage component of verbal WM maintenance. Additional investigations using event related designs will be needed to isolate the maintenance stage of verbal WM within a given trial to further address this hypothesis. Future studies will also need to directly manipulate phonological storage processes by varying both the length of the maintenance period or the quality of encoding.

Nonlinear responses to increasing WM load were observed in adolescents, but not children or young adults. Among adolescents, inverted U-shaped responses were observed in medial parietal and prefrontal regions, consistent with previous observations in adults (Rypma et al., 1999; Kirschen et al., 2005; Caseras et al., 2006). Nonlinear load-dependent activation was significantly increased in right ventral prefrontal cortex among young adults, relative to children, although this region was not observed in the young adult mean activation map, perhaps to due to differences in the sample sizes between age groups. Nonetheless, young adult subjects displayed statistically significant increases in nonlinear load-dependent activation, relative to children, in right ventral prefrontal cortex. Nonlinear load-dependency may indicate recruitment of different working memory mechanisms at different WM loads (Braver et al., 1997, Gould et al., 2003). Regions showing nonlinear responses to increasing WM load showed increased activation to medium load trials and similar levels of activation to low and high load trials. This pattern is consistent with the notion of a brain region being recruited as task demands increase (from low to medium WM loads), but then “dropping out” as task demands become overly difficult (from medium to high WM loads).

Dissociating developmental changes in brain activation from changes that may be due to performance differences among different age groups is an important and complex issue (Durston and Casey, 2006). In the present study, adolescents and adults recruited several brain regions not recruited by children, with no apparent benefit to accuracy, given that accuracy did not vary with age. Response time differences may contribute to observed differences in brain activation between young adults and children, but it is important to note that although adults were significantly faster than children there were no accuracy differences between these groups. Furthermore, there were no significant differences in RT between adolescents and children, and adolescents displayed increased linear-load dependent activation relative to children in the same location as young adults subjects. Perhaps group differences in RT stem from adults performing low and medium trials at ceiling, and the more diffuse pattern of activation among older participants would facilitate performance only if the task were more demanding. Three WM load levels may not be sufficient to characterize the full spectrum of parametric changes in both behavior and BOLD signal that occur in response to WM challenge.

The present study extends knowledge of WM development in several important ways. First, although linear load effects have been shown in adults, the present findings are the first, to our knowledge, to investigate load effects in verbal WM across childhood, adolescence, and adulthood. Developmental changes in the network of regions recruited in response to increasing WM load suggest that changes in phonological processing may be most prominent, as evidenced by increasing activation with age in parietal and cerebellar regions crucial for this cognitive process. Second, cerebellar support of verbal WM develops between childhood and adolescence, and our results suggest that the role of the cerebellum in the cognitive development of verbal WM may differ from its role in visual-spatial WM. These results demonstrate that, while children, adolescents, and young adults activate similar cerebro-cerebellar verbal working memory networks, the extent to which they reply on parietal and cerebellar regions in response to increasing task difficulty changes significantly between childhood and adolescence.

Acknowledgments

Support was provided by the National Institutes of Health NIDA R21 DA15878, RO1 DA017831 (awarded to ERS); NIAAA NRSA F31AA16039 (awarded to EDO’H); other support was provided by NCRR P41 RR13642 and U54 RR021813.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Baddeley A. Working memory. Oxford University Press; New York: 1986. [Google Scholar]
  2. Baddeley A. Working memory. Science. 1992;255:556–559. doi: 10.1126/science.1736359. [DOI] [PubMed] [Google Scholar]
  3. Baddeley A. Working memory: looking back and looking forward. Nat Rev Neurosci. 2003;4:829–839. doi: 10.1038/nrn1201. [DOI] [PubMed] [Google Scholar]
  4. Braver TS, Cohen JD, Nystrom LE, Jonides J, Smith EE, Noll DC. A parametric study of prefrontal cortex involvement in human working memory. Neuroimage. 1997;5:49–62. doi: 10.1006/nimg.1996.0247. [DOI] [PubMed] [Google Scholar]
  5. Callicott JH, Mattay VS, Bertolino A, Finn K, Coppola R, Frank JA, Goldberg TE, Weinberger DR. Physiological characteristics of capacity constraints in working memory as revealed by functional MRI. Cereb Cortex. 1999;9:20–26. doi: 10.1093/cercor/9.1.20. [DOI] [PubMed] [Google Scholar]
  6. Casey BJ, Cohen JD, Jezzard P, Turner R, Noll DC, Trainor RJ, Giedd J, Kaysen D, Hertz-Pannier L, Rapoport JL. Activation of prefrontal cortex in children during a nonspatial working memory task with functional MRI. Neuroimage. 1995;2:221–229. doi: 10.1006/nimg.1995.1029. [DOI] [PubMed] [Google Scholar]
  7. Chen SH, Desmond JE. Temporal dynamics of cerebro-cerebellar network recruitment during a cognitive task. Neuropsychologia. 2005;43:1227–1237. doi: 10.1016/j.neuropsychologia.2004.12.015. [DOI] [PubMed] [Google Scholar]
  8. Ciesielski KT, Lesnik PG, Savoy RL, Grant EP, Ahlfors SP. Developmental neural networks in children performing a Categorical N-Back Task. Neuroimage. 2006;33:980–990. doi: 10.1016/j.neuroimage.2006.07.028. [DOI] [PubMed] [Google Scholar]
  9. Cohen JD, Forman SD, Braver TS, Casey BJ, Servan-Schreiber D, Noll DC. Activation of the prefrontal cortex in a nonspatial working memory task with functional MRI. Human Brain Mapping. 1994;1:293–304. doi: 10.1002/hbm.460010407. [DOI] [PubMed] [Google Scholar]
  10. Courtney SM, Petit L, Maisog JM, Ungerleider LG, Haxby JV. An area specialized for spatial working memory in human frontal cortex. Science. 1998;279:1347–1351. doi: 10.1126/science.279.5355.1347. [DOI] [PubMed] [Google Scholar]
  11. Crone EA, Wendelken C, Donohue S, van Leijenhorst L, Bunge SA. Neurocognitive development of the ability to manipulate information in working memory. Proc Natl Acad Sci U S A. 2006;103:9315–9320. doi: 10.1073/pnas.0510088103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Curtis CE, Rao VY, D'Esposito M. Maintenance of spatial and motor codes during oculomotor delayed response tasks. J Neurosci. 2004;24:3944–3952. doi: 10.1523/JNEUROSCI.5640-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. D'Esposito M, Postle BR, Ballard D, Lease J. Maintenance versus manipulation of information held in working memory: an event-related fMRI study. Brain Cogn. 1999;41:66–86. doi: 10.1006/brcg.1999.1096. [DOI] [PubMed] [Google Scholar]
  14. D'Esposito M, Postle BR, Rypma B. Prefrontal cortical contributions to working memory: evidence from event-related fMRI studies. Exp Brain Res. 2000;133:3–11. doi: 10.1007/s002210000395. [DOI] [PubMed] [Google Scholar]
  15. Desmond JE, Chen SH, DeRosa E, Pryor MR, Pfefferbaum A, Sullivan EV. Increased frontocerebellar activation in alcoholics during verbal working memory: an fMRI study. Neuroimage. 2003;19:1510–1520. doi: 10.1016/s1053-8119(03)00102-2. [DOI] [PubMed] [Google Scholar]
  16. Desmond JE, Gabrieli JD, Wagner AD, Ginier BL, Glover GH. Lobular patterns of cerebellar activation in verbal working-memory and finger-tapping tasks as revealed by functional MRI. J Neurosci. 1997;17:9675–9685. doi: 10.1523/JNEUROSCI.17-24-09675.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Durston S, Casey BJ. What have we learned about cognitive development from neuroimaging? Neuropsychologia. 2006;44:2149–2157. doi: 10.1016/j.neuropsychologia.2005.10.010. [DOI] [PubMed] [Google Scholar]
  18. Gathercole SE. Cognitive approaches to the development of short-term memory. Trends Cogn Sci. 1999;3:410–419. doi: 10.1016/s1364-6613(99)01388-1. [DOI] [PubMed] [Google Scholar]
  19. Giedd JN, Blumenthal J, Jeffries NO, Castellanos FX, Liu H, Zijdenbos A, Paus T, Evans AC, Rapoport JL. Brain development during childhood and adolescence: a longitudinal MRI study. Nat Neurosci. 1999;2:861–863. doi: 10.1038/13158. [DOI] [PubMed] [Google Scholar]
  20. Gould RL, Brown RG, Owen AM, ffytche DH, Howard RJ. fMRI BOLD response to increasing task difficulty during successful paired associates learning. Neuroimage. 2003;20:1006–1019. doi: 10.1016/S1053-8119(03)00365-3. [DOI] [PubMed] [Google Scholar]
  21. Jenkinson M, Bannister P, Brady M, Smith S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage. 2002;17:825–841. doi: 10.1016/s1053-8119(02)91132-8. [DOI] [PubMed] [Google Scholar]
  22. Jenkinson M, Smith S. A global optimisation method for robust affine registration of brain images. Med Image Anal. 2001;5:143–156. doi: 10.1016/s1361-8415(01)00036-6. [DOI] [PubMed] [Google Scholar]
  23. Kirschen MP, Chen SH, Schraedley-Desmond P, Desmond JE. Load- and practice-dependent increases in cerebro-cerebellar activation in verbal working memory: an fMRI study. Neuroimage. 2005;24:462–472. doi: 10.1016/j.neuroimage.2004.08.036. [DOI] [PubMed] [Google Scholar]
  24. Klingberg T, Forssberg H, Westerberg H. IncreasedBrain Activity in Frontal and Parietal Cortex Underlies the Development of Visuospatial Working Memory Capacity during Childhood. Journal of Cognitive Neuroscience. 2002;14:1–10. doi: 10.1162/089892902317205276. [DOI] [PubMed] [Google Scholar]
  25. Kwon H, Reiss AL, Menon V. Neural basis of protracted developmental changes in visuo-spatial working memory. Proc Natl Acad Sci U S A. 2002;99:13336–13341. doi: 10.1073/pnas.162486399. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Luciana M, Nelson CA. The functional emergence of prefrontally-guided working memory systems in four- to eight-year-old children. Neuropsychologia. 1998;36:273–293. doi: 10.1016/s0028-3932(97)00109-7. [DOI] [PubMed] [Google Scholar]
  27. Luna B, Garver KE, Urban TA, Lazar NA, Sweeney JA. Maturation of cognitive processes from late childhood to adulthood. Child Dev. 2004;75:1357–1372. doi: 10.1111/j.1467-8624.2004.00745.x. [DOI] [PubMed] [Google Scholar]
  28. Muller NG, Knight RT. The functional neuroanatomy of working memory: contributions of human brain lesion studies. Neuroscience. 2006;139:51–58. doi: 10.1016/j.neuroscience.2005.09.018. [DOI] [PubMed] [Google Scholar]
  29. Nagy Z, Westerberg H, Klingberg T. Maturation of White Matter is Asociated with the Development of Cognitive Functions during Childhood. Journal of Cognitive Neuroscience. 2004;16:1227–1233. doi: 10.1162/0898929041920441. [DOI] [PubMed] [Google Scholar]
  30. Nelson CA, Monk CS, Lin J, Carver LJ, Thomas KM, Truwit CL. Functional neuroanatomy of spatial working memory in children. Dev Psychol. 2000;36:109–116. doi: 10.1037//0012-1649.36.1.109. [DOI] [PubMed] [Google Scholar]
  31. Olesen P, Nagy Z, Westerberg H, Kingberg T. Combined analysis of DTI and fMRI data reveals a joint maturation of white and grey matter in a fronto-parietal network. Cognitive Brain Research. 2003;18:48–57. doi: 10.1016/j.cogbrainres.2003.09.003. [DOI] [PubMed] [Google Scholar]
  32. Pessoa L, Gutierrez E, Bandettini P, Ungerleider L. Neural correlates of visual working memory: fMRI amplitude predicts task performance. Neuron. 2002;35:975–987. doi: 10.1016/s0896-6273(02)00817-6. [DOI] [PubMed] [Google Scholar]
  33. Postle BR, Berger JS, Taich AM, D'Esposito M. Activity in human frontal cortex associated with spatial working memory and saccadic behavior. J Cogn Neurosci. 2000;12(Suppl 2):2–14. doi: 10.1162/089892900564028. [DOI] [PubMed] [Google Scholar]
  34. Rowe JB, Toni I, Josephs O, Frackowiak RS, Passingham RE. The prefrontal cortex: response selection or maintenance within working memory? Science. 2000;288:1656–1660. doi: 10.1126/science.288.5471.1656. [DOI] [PubMed] [Google Scholar]
  35. Rypma B. Factors controlling neural activity during delayed-response task performance: testing a memory organization hypothesis of prefrontal function. Neuroscience. 2006;139:223–235. doi: 10.1016/j.neuroscience.2005.07.062. [DOI] [PubMed] [Google Scholar]
  36. Rypma B, Prabhakaran V, Desmond JE, Glover GH, Gabrieli JD. Load-dependent roles of frontal brain regions in the maintenance of working memory. Neuroimage. 1999;9:216–226. doi: 10.1006/nimg.1998.0404. [DOI] [PubMed] [Google Scholar]
  37. Scherf KS, Sweeney JA, Luna B. Brain basis of developmental change in visuospatial working memory. J Cogn Neurosci. 2006;18:1045–1058. doi: 10.1162/jocn.2006.18.7.1045. [DOI] [PubMed] [Google Scholar]
  38. Schlaggar BL, Brown TT, Lugar HM, Visscher KM, Miezin FM, Petersen SE. Functional neuroanatomical differences between adults and school-age children in the processing of single words. Science. 2002;296:1476–1479. doi: 10.1126/science.1069464. [DOI] [PubMed] [Google Scholar]
  39. Schmahmann JD. An emerging concept. The cerebellar contribution to higher function. Arch Neurol. 1991;48:1178–1187. doi: 10.1001/archneur.1991.00530230086029. [DOI] [PubMed] [Google Scholar]
  40. Schmahmann JD. From Movement to THought: Anatomic Substrates of the Cerebellar Contribution to Cognitive Processing. Human Brain Mapping. 1996;4:174–198. doi: 10.1002/(SICI)1097-0193(1996)4:3<174::AID-HBM3>3.0.CO;2-0. [DOI] [PubMed] [Google Scholar]
  41. Schmahmann JD, Doyon J, Toga AW, Petrides M, Evans AC. MRI Atlas of the Human Cerebellum. Academic Press; San Diego, CA: 2000. [DOI] [PubMed] [Google Scholar]
  42. Schmahmann JD, Pandya DN. The cerebrocerebellar system. Int Rev Neurobiol. 1997;41:31–60. doi: 10.1016/s0074-7742(08)60346-3. [DOI] [PubMed] [Google Scholar]
  43. Smith EE, Jonides J. Neuroimaging analyses of human working memory. Proc Natl Acad Sci U S A. 1998;95:12061–12068. doi: 10.1073/pnas.95.20.12061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Smith EE, Jonides J, Marshuetz C, Koeppe RA. Components of verbal working memory: evidence from neuroimaging. Proc Natl Acad Sci U S A. 1998;95:876–882. doi: 10.1073/pnas.95.3.876. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TE, Johansen-Berg H, Bannister PR, De Luca M, Drobnjak I, Flitney DE, Niazy RK, Saunders J, Vickers J, Zhang Y, De Stefano N, Brady JM, Matthews PM. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage. 2004;23(Suppl 1):S208–219. doi: 10.1016/j.neuroimage.2004.07.051. [DOI] [PubMed] [Google Scholar]
  46. Sowell ER, Peterson BS, Thompson PM, Welcome SE, Henkenius AL, Toga AW. Mapping Cortical Change Across the Human Life Span. Nature Neuroscience. 2003;6:309–315. doi: 10.1038/nn1008. [DOI] [PubMed] [Google Scholar]
  47. Sowell ER, Thompson PM, Holmes CJ, Jernigan TL, Toga AW. In vivo evidence for post-adolescent brain maturation in frontal and striatal regions. Nat Neurosci. 1999;2:859–861. doi: 10.1038/13154. [DOI] [PubMed] [Google Scholar]
  48. Sowell ER, Thompson PM, Toga AW. Mapping changes in the human cortex throughout the span of life. Neuroscientist. 2004;10:372–392. doi: 10.1177/1073858404263960. [DOI] [PubMed] [Google Scholar]
  49. Spear LP. The adolescent brain and age-related behavioral manifestations. Neurosci Biobehav Rev. 2000;24:417–463. doi: 10.1016/s0149-7634(00)00014-2. [DOI] [PubMed] [Google Scholar]
  50. Sternberg S. High-speed scanning in human memory. Science. 1966;153:652–654. doi: 10.1126/science.153.3736.652. [DOI] [PubMed] [Google Scholar]
  51. Talairach JaTP. Co-planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System-an Approach to Cerebral Imaging. Thieme Medical Publishers; New York: 1988. [Google Scholar]
  52. Thomas KM, King SW, Franzen PL, Welsh TF, Berkowitz AL, Noll DC, Birmaher V, Casey BJ. A developmental functional MRI study of spatial working memory. Neuroimage. 1999;10:327–338. doi: 10.1006/nimg.1999.0466. [DOI] [PubMed] [Google Scholar]
  53. Veltman DJ, Rombouts SA, Dolan RJ. Maintenance versus manipulation in verbal working memory revisited: an fMRI study. Neuroimage. 2003;18:247–256. doi: 10.1016/s1053-8119(02)00049-6. [DOI] [PubMed] [Google Scholar]
  54. Wood AG, Harvey AS, Wellard RM, Abbott DF, Anderson V, Kean M, Saling MM, Jackson GD. Language cortex activation in normal children. Neurology. 2004;63:1035–1044. doi: 10.1212/01.wnl.0000140707.61952.ca. [DOI] [PubMed] [Google Scholar]
  55. Woolrich MW, Ripley BD, Brady M, Smith SM. Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage. 2001;14:1370–1386. doi: 10.1006/nimg.2001.0931. [DOI] [PubMed] [Google Scholar]

RESOURCES