Abstract
Sleep timing shifts later during adolescence, thus conflicting with early school start times. This can lead to irregular weekday-weekend schedules and circadian misalignment, which have been linked to depression and substance abuse, consistent with disruptions in the processing of rewards. We tested associations between weekend-weekday shifts in sleep timing and the neural response to monetary reward in healthy adolescents, using actigraphy and a functional magnetic resonance imaging paradigm. Region-of-interest analyses focused on the medial prefrontal cortex (mPFC) and striatum, both of which are implicated in reward function. Analyses adjusted for pubertal stage, sex, and total sleep time. Greater weekend-weekday advances in midsleep were associated with decreased mPFC and striatal reactivity to reward, which could reflect reduced regulatory response and reward sensitivity. We speculate that circadian misalignment associated with weekend shifts in sleep timing may contribute to reward-related problems such as depression and substance abuse.
Keywords: Sleep timing, adolescence, reward, medial prefrontal cortex, striatum, fMRI
INTRODUCTION
Adolescence is a tumultuous time of development, with remarkable changes across psychological, behavioral, and physiological domains. Even when developmentally normative, these changes can interact with psychosocial and environmental influences in ways that threaten adolescents’ physical and psychological functioning. A striking example of this involves sleep; preferred sleep timing shifts later during adolescence, driven in part by biological factors, including a delay in circadian phase that occurs in conjunction with pubertal development (Carskadon, Acebo, & Jenni, 2004). This shift towards later sleep timing is mismatched with a simultaneous shift towards early secondary school start times, leading to marked and recurring swings between earlier weekday sleep schedules and later weekend sleep schedules (Crowley, Acebo, & Carskadon, 2007; National Sleep Foundation, 2006; Wolfson & Carskadon, 2003). This vicious cycle of shifts in sleep timing is accompanied by both sleep loss and recurring circadian misalignment, which together may impair affective and motivation regulation. Reward function—that is, the processing of stimuli that inspire behavior and reinforce it after it occurs—may be particularly sensitive to these developmental changes in sleep and circadian function, given the striking alterations in reward-seeking behaviors and reward-related brain function that occur during adolescence in general (Ernst & Fudge, 2009; Galvan, 2010; Spear, 2000) and during puberty in particular (Forbes et al., 2010). Altered reward function, including disrupted neural response to rewarding stimuli, may put adolescents at risk for mood disorders, as well as substance abuse and other risk-taking behaviors that threaten adolescent health.
Cross-sectional studies link weekday-weekend shifts in sleep timing to reward-related problems, including depression and risk-taking behaviors. Evidence from two studies of high-school students suggests these effects are not totally due to sleep loss. O'Brien and Mindell (2005) reported that weekend delays in bedtime—but not total sleep time— were related to increased self-reported risk-taking, including greater drug and alcohol use, unsafe behaviors (riding in a car without a seatbelt), and sexual behaviors. Neither weeknight total sleep time, nor weekday-weekend changes in total sleep time, were related to risk-taking, although weeknight total sleep was associated with higher rates of depression. Similarly, Pasch and colleagues (2010) reported that larger weekday-weekend shifts in bedtime and wake-up time were associated with greater self-reported substance use (cigarette smoking, alcohol, and marijuana), alcohol intoxication, and skipping school, all after controlling for potential confounding factors such as gender, grade, SES, BMI, pubertal development, parent's education, and both weekday and weekend sleep duration. These weekday-weekend shifts may be a proxy for circadian misalignment, analogous to jet lag, in which the sleep-wake schedule is at odds with internal circadian timing. Later weekend sleep timing results in later exposure to the light/dark cycle, cueing adolescents’ circadian clocks to delay even further during the weekend. As demonstrated in controlled studies (Crowley & Carskadon, 2010; Taylor, Wright, & Lack, 2008), these delays are particularly problematic, as they make it even more difficult for adolescents’ circadian clocks to advance sufficiently to adjust to the earlier timing of the school week. This results in difficulty in initiating sleep at the desired time, increased daytime sleepiness as the school week begins, and consequences for daytime functioning, including academic performance (Wolfson & Carskadon, 2003). Notably, the findings from O'Brien and Mindell, along with those from Pasch and colleagues, suggest that sleep loss per se may not be the only important factor, but that circadian misalignment may independently be associated with affective and behavioral dysregulation.
A role for circadian misalignment in mood disturbance and problem behaviors would be consistent with other evidence that circadian misalignment is associated with altered reward function. Recent studies have reported that the degree of circadian misalignment correlates with the severity of depression in adults (Emens, Lewy, Kinzie, Arntz, & Rough, 2009; Hasler, Buysse, Kupfer, & Germain, 2010) and the extent of substance abuse in adolescents (Hasler, Bootzin, Cousins, Fridel, & Wenk, 2008). Furthermore, a substantial literature consistently reports that evening chronotypes exhibit greater depression, substance use, and risk-taking behavior, as well as more variable weekday-weekend sleep timing (Chelminski, Ferraro, Petros, & Plaud 1999; Drennan, Klauber, Kripke, & Goyette, 1991; Gau et al., 2007; Giannotti, Cortesi, Sebastiani, & Ottaviano, 2002; Hasler, Allen, Sbarra, Bootzin, & Bernert, 2010; Negriff, Dorn, Pabst, & Susman, 2011; Pieters, Van Der Vorst, Burk, Wiers, & Engels, 2010; Randler, 2008), presumably because of the “social jet lag” evening-types suffer when trying to accommodate work and school schedules better geared towards morning-types (Wittmann, Dinich, Merrow, & Roenneberg, 2006). However, despite the converging evidence, the previous data relating weekday-weekend shifts in sleep timing to altered reward function are limited by the exclusive use of retrospective self-report measures of sleep, mood, and behavior. Given the errors in bias and recall inherent to retrospective self-report (Shiffman, Stone, & Hufford, 2008), new studies using observational and objective measures to investigate the association between variability in weekday-weekend sleep timing and reward function would complement the extant literature.
Thus, the aim of the current analyses was to examine whether weekend-weekday shifts in sleep timing are associated with alterations in objective measures of reward function in a sample of healthy adolescents. We employed a functional magnetic resonance imaging (fMRI) guessing task with monetary reward as our objective measure of reward function, and we collected four days of wrist actigraphy as our objective measure of sleep-wake behavior. Our design remains cross-sectional, precluding determination of whether shifts in sleep timing cause changes in reward function, or vice versa. That said, we conceptualize weekend-weekday shifts in sleep timing as a reflection of the relative match between preferred sleep timing (based on chronotype) and actual sleep timing (imposed by academics or other constraints). Given the trait-like nature of chronotype (Roenneberg, Wirz-Justice, & Merrow, 2003), these shifts in sleep timing may reflect a relatively stable individual difference putatively associated with chronic alterations in reward function. Although our cross-sectional analyses must be interpreted with caution, positive findings using these objective measures would lend tentative credence to a pathway in which shifts in sleep timing impact reward function and risk-taking rather than later sleep schedules being an epiphenomenon of risk-taking behavior occurring on weekend nights.
Given prior evidence that sleep loss impacts reward-related brain function (Gujar, Yoo, Hu, & Walker, 2011; Venkatraman, Chuah, Huettel, & Chee, 2007; Venkatraman, Huettel, Chuah, Payne, & Chee, 2011), we also took care to account for total sleep time in our analyses in an attempt to isolate the specific association with putative circadian misalignment. Furthermore, given that our sample was recruited to range in development from pre-puberty to late puberty, we accounted for pubertal development in the primary analyses. Although the cross-sectional design precludes definitive analyses of developmental contributions, this range also allowed us to explore developmental differences in any observed effects.
Previous data from our group focusing on weekend sleep alone suggest that variation in weekend sleep parameters are associated with reward-related activation in various subregions of the striatum (Holm et al., 2009). Generally, that analysis found that increased sleep duration and sleep quality were associated with increases in caudate reactivity to reward, while measures of sleep timing (sleep onset and offset) were associated with both increases and decreases. In the current analyses, we investigated reward-related activation in both the striatum and the medial prefrontal cortex (mPFC). These two regions are targets of midbrain dopamine neurons (Haber & Knutson, 2010) and figure prominently in current theories of adolescent reward function, which propose that such changes during adolescence are partly due to a cortical-subcortical imbalance in which a relatively immature mPFC is less effective in modulating a sensitive and more rapidly developing ventral striatum (VS) (Ernst & Fudge, 2009; Forbes, et al., 2010; Galvan, 2010; Spear, 2000).
Based on both these theories and the extant literature on sleep and reward function, we predicted in the current analyses that larger weekday-weekend differences in sleep timing would be associated with reduced mPFC reactivity during the anticipation and receipt of rewards, consonant with less regulatory control. Given the more ambiguous findings from our previous analyses of sleep and reward function, however, we did not have a strong a priori directional hypotheses for how dynamics in sleep timing might relate to the extent of subcortical (VS) reactivity to reward.
METHODS
Participants
The final sample included 56 adolescents for the reported analyses (Table 1). A total of 128 adolescents enrolled in a larger study of normal pubertal development, but participants were excluded for excessive head movement during the scan (n = 22), claustrophobia (n = 3), and missing actigraphy data (n = 47). The final sample did not differ from the excluded participants on any measures. Adolescents were recruited from the community through advertisements, flyers, and demographically-targeted phone lists. Adolescents were recruited to be in a relatively narrow age range (11-13 years) but vary in pubertal development. Because girls in the United States achieve mid-puberty at a somewhat earlier age than boys (Herman-Giddens et al., 1997; Karpati, Rubin, Kieszak, Marcus, & Troiano, 2002; Wu, Mendola, & Buck, 2002), girls were recruited at an earlier age-range (11-12 years old versus 12-13 years old for boys). See Table 1 for final mean ages. Adolescents were free of current and lifetime psychiatric disorders, history of head injury, serious medical illness, psychotropic medications, alcohol use, or illicit drug use, and did not have braces. No participants were current smokers, and only one participant reported having previously tried cigarettes. All participants’ parents gave informed consent, and all participants gave verbal assent, prior to participation. The study was approved by the University of Pittsburgh Institutional Review Board. Participants were studied throughout the year, with 18 participants studied during summer or holiday break.
Table 1.
Age | 12.34 ± 0.88 |
Sex (% female) | 55.4 |
Pubertal status (%) | |
Pre/early | 33.9 |
Mid/late | 66.1 |
Race (%) | |
Caucasian | 71.4 |
African-American | 23.2 |
Latino | 0 |
Asian | 1.8 |
Native-American | 3.6 |
Mood | |
Depressive symptoms (MFQ) | 6.73 ± 7.27 |
Positive affect (PANAS) | 3.24 ± 0.75 |
Note: Values are mean ± SD unless otherwise stated
Pubertal development
A trained research nurse performed a physical exam to determine the Tanner stage of development. Adolescents were classified as pre/early pubertal if they met criteria for Tanner stage one or two, and mid/late pubertal if they met criteria for Tanner stage three, four, or five. In total, 19 participants (34%) were classified as pre/early pubertal adolescents.
Actigraphy-measured sleep variables
We used wrist actigraphy (Octagonal Basic Motionlogger Actigraph, Ambulatory Monitoring, Ardsley, NY) to assess sleep-wake behavior. Participants wore actigraphs on their non-dominant wrists. For all participants, actigraphy collection began on Friday afternoon and concluded on Tuesday morning, thus collecting two weekend (Friday and Saturday) and two weekday (Sunday and Monday) nights in the participant's home environments. Actigraphy data were preprocessed and scored in 60-s epochs using the Cole-Kripke option within Action 2.5. A number of measures of sleep timing, sleep continuity, and sleep duration were calculated using this procedure. Participants pressed a button to indicate when they were trying to go to sleep and after their final awakening, which inserted markers into the actigraph record that served as outside boundaries for subsequent calculation of sleep onset and offset. When this was not done, the estimated lights-off and lights-on times given by the participant in their daily sleep diaries (data not reported here) were used. Sleep onset and sleep offset times were determined by the automated algorithm, which calculates a moving average to determine whether the current 1-minute epoch should be scored as sleep or wake. Midsleep time was the midpoint between sleep onset and sleep offset. Sleep latency was the number of minutes between when the participant retired and sleep onset occurred. Wake after sleep onset was the number of minutes scored as wake in between sleep onset and sleep offset. Total sleep time (TST) was the interval between sleep onset and sleep offset, minus wake after sleep onset. Finally, sleep efficiency was the % of epochs in between sleep onset and sleep offset scored as sleep.
We focused on two actigraphy-derived variables for our primary analyses. We selected midsleep as the most parsimonious measure of sleep timing, as well as one less confounded by sleep duration than sleep onset or offset. We selected TST to capture the extent of sleep restriction (i.e., chronic sleep loss), which was included as a covariate in all the neuroimaging analyses.
Self-reported affective functioning
Self-reported positive affect was assessed using the PANAS-C (Laurent et al., 1999), a mood questionnaire with good psychometric properties, which was administered in the participants’ natural environments using a cell-phone ecological momentary assessment protocol (see (Forbes et al., 2009) for details) . Mood is described by adjectives (e.g., happy) using a 5-point response scale (1=very slightly or not at all, 5=extremely). All 20 items were administered once per day, and a subset of eight items (four reflecting positive affect: happy, joyful, energetic, excited) was administered at all other calls. Because mean positive affect score within the day was not excessively variable (SD=0.85–1.11), mean positive affect across the weekend was computed for analyses.
Symptoms of depression were assessed using the Moods and Feelings Questionnaire (MFQ) (Angold, Erkanli, Silberg, Eaves, & Costello, 2002), a widely-used self-report measure with strong psychometric properties (Katon, Russo, Richardson, McCauley, & Lozano, 2008; Wood, Kroll, Moore, & Harrington, 1995). The MFQ is comprised of a series of descriptive phrases regarding how the subject has been feeling or acting over the past two weeks. The MFQ was completed on the day of the fMRI scan.
Reward processing task
The fMRI paradigm was a slow event-related card-guessing game (Forbes, et al., 2009) that allowed the probing of neural response to the anticipation and receipt of monetary reward feedback. Participants received win, loss, or no-change feedback for each trial. Participants were told that their performance would determine a monetary reward after the scan, with $1 for each win and 50 cents deducted for each loss. Trials were presented in pseudorandom order with predetermined outcomes. Earnings totaled $6. Trials were presented in four runs, with 12 trials per run and a balanced number of trial types within runs (i.e., six possible-win and six possible-loss trials in each). During each trial, participants guessed via button press whether the value of a visually presented card was high or low (3s), learned the trial type (possible-win or possible-loss) and anticipated feedback (12s), and received outcome feedback (1s plus 11s intertrial interval). Participants were unaware of fixed outcome probabilities, and their engagement and motivation were maintained by verbal encouragement between runs. Because participants practiced the task before the scan and did not exhibit a change in reaction time across task runs, it is likely that they understood the anticipation period. During debriefing, all participants stated that they understood the task, thought that outcomes were due to chance, and found the task engaging.
fMRI procedures
Scans occurred midweek, and prior to the weekend of actigraphy assessment for most participants (48/56, 85.7%), on average 2.33 ± 4.07 days before the first day (Friday) of actigraphy assessment. Each participant was scanned using a Siemens 3T Allegra scanner. BOLD functional images were acquired with a gradient echo planar imaging (EPI) sequence and covered 34 axial slices (3 mm thick) beginning at the cerebral vertex and encompassing the entire cerebrum and the majority of the cerebellum (TR/TE=2,000/ 25 ms, FOV =20 cm, matrix = 64 × 64). All scanning parameters were selected to optimize the quality of the BOLD signal while maintaining a sufficient number of slices to acquire whole-brain data. Before the collection of fMRI data for each participant, we acquired a reference EPI scan that we visually inspected for artifacts (e.g., ghosting) and for good signal across the entire volume of acquisition. The fMRI data from all included participants were cleared of such problems. Whole-brain image analysis was completed using SPM2 (http://www.fil.ion.ucl.ac.uk/spm). For each scan, images for each participant were realigned to correct for head motion. Data sets were then selected for quality based on our standard small-motion correction for adolescents (<4 mm) (Forbes, et al., 2009). Realigned images were spatially normalized into standard stereotactic space (Montreal Neurological Institute template) using a 12-parameter affine model. Normalized images were smoothed with a 6-mm full width at half-maximum Gaussian filter. Voxelwise signal intensities were ratio normalized to the whole brain global mean. Preprocessed data sets were then analyzed in SPM8 using random effects models that account for both scan-to scan and participant-to-participant variability to determine task-specific regional responses. For each participant and scan, predetermined condition effects (i.e., main effects of task) at each voxel were calculated using a t statistic, producing a statistical image for the two contrasts of interest: 1) reward anticipation >baseline and 2) reward outcome (i.e., win) > baseline. Baseline was defined as the last 3s of each intertrial interval. Because the study focused on reward-related brain function, analyses included trials involving reward anticipation and reward outcome. Based on our hypotheses and previous strategy with this task (Forbes, et al., 2009), analyses focused on the first run, as these data are less likely to reflect fatigue, boredom, frustration with task length, and habituation. Habituation is a particular concern because striatal response tends to diminish with repeated experience of a reward (Koob & Le Moal, 2008). Individual contrast images were then included in second-level region of interest (ROI) analyses to determine group-level main effects of task within striatal and mPFC ROIs using within-group t-tests, thresholded at a voxel level of p < .05 and a minimum extent of 10 contiguous voxels. The striatal region of interest (ROI) was anatomically-defined using the nucleus accumbens, caudate head, and putamen regions from WFU PickAtlas (v3.0.3). The mPFC ROI was constructed using the PickAtlas and defined as a 5,393-voxel sphere including medial Brodmann Area (BA) 10 and BA32, which are part of the anterior rostral mPFC region implicated in social cognition and self-processing (Amodio & Frith, 2006).
Data analysis
Analyses of actigraphy variables were completed in SPSS (v 18.0.2). Response to reward was examined in SPM8 using second-level random effects regressions, with the shift in midsleep from Saturday night to Sunday night as the independent variable, and sex, development group, and total sleep time as covariates. Additional exploratory regression analyses included a developmental group X midsleep shift interaction term. Total sleep time was included to adjust for the possible confound of sleep restriction. Because of multicollinearity between age and puberty, age was not included in the final model. The results of the SPM regressions were exported to SPSS for linear regression analyses yielding R2.
RESULTS
Participant characteristics
Participant characteristics are reported in Table 1. Approximately 2/3 of the sample was in the mid-to-late puberty stage, and participants were not depressed.
Sleep continuity, duration, and timing
On average, the sample did not show clinically significant levels of insomnia, with mean SL, WASO, and efficiency all in the normal range (Table 2). Mean TST was typical for this age range (8.1-8.4 hours, (National Sleep Foundation, 2006)) but less than previous estimates of sleep need (also 9+ hours, (Jenni & Carskadon, 2007)), indicating that these participants may be relatively sleep-restricted (Table 2). Sleep timing was later on the weekends, and sleep efficiency was higher, relative to weeknights, according to two-tailed t-tests. Based on a oneway ANOVA and post-hoc Scheffe comparisons, TST was relatively stable across Friday-Sunday nights (8.45, 8.47, and 8.08 h, respectively), but was significantly shorter on Monday night (7.74 h) compared to Saturday night only (overall ANOVA: F = 3.68, p = 0.01; Scheffe post-hoc Saturday-Monday comparison: Mean difference = 0.64, SE = 0.20, p = 0.02).
Table 2.
Overall | Weekend (Friday/Saturday) | Weekday (Sunday/Monday) | Paired t-test | ||||||
---|---|---|---|---|---|---|---|---|---|
Sleep parameter | Mean | SD | Range | Mean | SD | Mean | SD | t | p |
Sleep onset | 23:33 | 1:00 | 21:35 – 2:01 | 23:52 | 1:24 | 23:13 | 1:06 | 3.96 | <.001 |
Midsleep | 3:49 | 0:54 | 2:06 – 6:06 | 4:16 | 1:08 | 3:22 | 0:58 | 6.47 | <.001 |
Sleep offset | 8:07 | 0:52 | 6:35-10:11 | 8:41 | 1:02 | 7:32 | 1:05 | 7.19 | <.001 |
Total sleep time (h) | 8.20 | 0.59 | 5.8 – 9.7 | 8.46 | 0.95 | 7.92 | 1.04 | -0.81 | ns |
Sleep latency (min) | 20.44 | 10.13 | 6-72 | 20.28 | 17.58 | 22.52 | 14.26 | -1.02 | ns |
Wake after sleep onset (min) | 24.99 | 24.42 | 4-98 | 21.07 | 20.69 | 23.91 | 24.12 | 1.49 | ns |
Sleep efficiency (%) | 95.18 | 4.5 | 80-99 | 96.02 | 3.66 | 95.19 | 4.86 | 3.34 | 0.002 |
Saturday-Sunday midsleep shift (h) | 0.82 | 1.15 | -2.00 – 3.02 |
NOTE: There were no significant sex-related differences on any sleep parameters, and only sleep onset significantly differed between pre/early and mid/late pubertal groups (t(54)=-2.09, p=0.04)
The largest and only statistically significant one-day shift in midsleep occurred between Saturday and Sunday night (Figure 1a); on average, the sample's midsleep advanced (shifted earlier) by 0.82 hours (t(55) = 5.28, p < 0.001). There was no difference in midsleep shift based on gender or developmental group. Overall, the sample's Saturday night-Sunday night shift in midsleep ranged from a 3.02 h advance to a 2.00 h delay, with 76.8% of the sample (43 participants) showing advances (Figure 1b). Based on these data, we selected the shift in midsleep from Saturday night to Sunday night as our primary sleep variable.
Weekend-weekday advances and reward-related brain function
In the reward anticipation phase, participants with larger midsleep advances from Saturday night to Sunday night exhibited less activation in portions of the mPFC (Table 3) and in the caudate (Table 3), all after statistically controlling for sex, developmental group, and mean TST. A parallel effect was observed in the reward outcome phase; participants with larger midsleep advances from Saturday night to Sunday night exhibited less activation across the mPFC (Table 3, Figure 2A) and in the ventral striatum (Table 3, Figure 2B), all after statistically controlling for sex, pubertal stage, and mean TST. No clusters exhibited greater activation in association with larger midsleep advances from Saturday to Sunday night during either the anticipation or outcome phases.
Table 3.
Talairach coordinates of maximum voxel in cluster |
|||||||
---|---|---|---|---|---|---|---|
Region | Hemisphere | x | y | z | Cluster Size | t | p |
Reward anticipation | |||||||
Anterior cingulate, BA 24 | Right | 3 | 28 | 23 | 1831 | 3.47 | 0.001a |
Putamen | Left | -16 | 6 | 1 | 106 | 2.75 | 0.004 |
Caudate body | Left | -14 | 8 | 17 | 18 | 2.47 | 0.008 |
Reward outcome | |||||||
Superior frontal gyrus, BA9 | Left | -7 | 55 | 20 | 3094 | 4.75 | 0.000a |
Ventral striatum | Right | 8 | 8 | 5 | 70 | 2.78 | 0.004 |
Putamen | Left | -21 | -4 | 4 | 62 | 2.36 | 0.011 |
Putamen | Right | 27 | -9 | -3 | 24 | 2.42 | 0.010 |
These clusters survive FWE correction
Because the sleep schedules of the 18 participants on summer or holiday break were ostensibly under fewer environmental constraints, which might lead to a differential relationship to reward function, we re-ran the above analyses with only the 38 participants assessed during the academic year. Findings based on these analyses closely paralleled those using the full sample, albeit resulting in somewhat smaller cluster sizes for the mPFC region of interest (data available upon request), suggesting there was not a qualitative difference in the association between weekend shifts in midsleep and reward-related brain function among the participants on break.
Developmental group as moderator
We explored the effect of pubertal development on these relationships by including a developmental group X midsleep shift interaction term in the regression analyses, retaining developmental group and midsleep shift as main effects, and again controlling for sex and mean TST. Results were consistent with the mid/late group driving the mPFC effects observed in the primary analyses, and were mostly consistent with the pre/early group driving the striatal effects in the primary analyses.
Larger midsleep advances were associated with relatively greater decrements in mPFC reactivity in the mid/late pubertal group during both anticipation (185 voxel cluster, peak [ -7 33 31], t = 2.59, p = .006; 32 voxel cluster, peak [12 32 26], t = 2.51, p = 0.008; 21 voxel cluster, peak [-1 35 13], t = 2.48, p = .008) and outcome (394 voxel cluster,peak [6 46 14], t= 3.45, p =.001, cluster-level uncorrected p = .07; 94 voxel cluster, peak [6 27 36], t = 2.45, p = 0.009). Peak voxels were in the bilateral anterior cingulate (BA 32 and 24) during anticipation and the right medial frontal gyrus (BA 9 and 6) during outcome. No mPFC clusters were associated with relatively greater decrements in reactivity among the pre/early group.
In contrast, larger midsleep advances were associated with relatively greater decrements in striatal reactivity in the mid/late group during anticipation only, and this was restricted to a single 55 voxel cluster in the right putamen (peak [21 15 1], t = 2.50, p = 0.008). The pre/early group showed relatively more diminished striatal reactivity in three bilateral caudate clusters during anticipation (67 voxel cluster, peak [-8 16 -4], t = 2.93, p = .003; 66 voxel cluster, peak [-7 1 13], t = 2.49, p = 0.008, and 24 voxel cluster, peak [14 5 14], t = 2.12, p = 0.019) and one 23 voxel cluster in the left caudate during outcome (peak [-16 12 16], t = 2.48, p =.008).
Associations with affective function
No significant associations were observed between the Saturday-Sunday shift in midsleep and measures of positive affect (PANAS-C; r = -.17, p = ns) or depression (MFQ; r = 0.07, p = ns). Likewise, none of reward-related brain responses—data extracted from the SPM analyses—correlated with positive affect (r < .20 for all analyses).
DISCUSSION
These data provide preliminary evidence that marked shifts between objectively-measured weekend and weekday sleep timing are associated with alterations in the neural correlates of reward function in healthy adolescents. Specifically, larger advances in the timing of midsleep from Saturday to Sunday night—which reflect later natural sleep phase—were associated with diminished activation during the anticipation and receipt of reward in regions within the mPFC and ventrial striatum. Notably, these associations held after controlling for sex and pubertal stage, as well as mean sleep total sleep time. This suggests that sleep loss is not the only critical factor leading to affective and behavioral dysregulation in adolescents confronted with early school start times. That is, despite the overwhelming focus in the lay and scientific literatures on sleep loss outcomes, circadian misalignment also appears to be related to reward function and has the potential to play an important role in the increase in reward-related behavior during adolescence. Next, our exploratory analyses suggest that circadian misalignment may be differentially related to reward function based on development. Importantly, these novel findings are based on objective data, yet are consistent with cross-sectional studies reporting links between shifts in weekday-weekend sleep timing and self-reported measures of altered reward function, including depression, substance use, and other risk-taking behaviors.
Our findings suggest that the link between weekend sleep shifts and increased risk-taking behaviors, including substance use, may not be purely due to substance use or peer influence driving later bedtimes and waketimes, later bedtimes allowing opportunity for greater substance use, or reduced parental control. Instead, weekend sleep shifts are associated with neural response to reward, a key mechanism thought to underlie reward-related behavior. Nonetheless, the cross-sectional design precludes determining any direction of causation. The weekend shifts could be leading to circadian dysregulation and subsequent impaired reward function, or the impaired reward function could drive adolescents to larger swings between weekday and weekend sleep timing, perhaps due to greater socializing and substance use on weekend nights. For example, elevated sensation seeking and reduced regulatory control could stimulate greater substance use, and/or could encourage adolescents to prioritize socializing over sleep, in both cases leading to later sleep times on weekend nights and altered function in neural reward circuits. These pathways are not mutually-exclusive, of course, as impaired reward function may leave adolescents more susceptible to peer pressure to use substances, stay out later, and generally engage in more risk-taking behaviors, all at the expense of maintaining a consistent sleep schedule. In fact, the combination of social context changes, increased access to rewarding (and risky activities), and maturational changes in circadian function and reward function might create spirals of mutual influence to exacerbate sleep shifts and risky behavior during adolescence. More fine-grained, longitudinal studies of adolescent development—including those that follow youth through late adolescence and focus on at-risk populations such as youth with substance-abusing parents—will provide invaluable information on the unfolding of these processes over time.
Although our study did not include any explicit measures of risk-taking, and the age and health of the sample precluded the presence of sufficient variability in substance use to use it as an outcome measure, the neuroimaging findings are somewhat consistent with other data linking adolescent brain function to risk-taking behavior. For example, a recent study reported that adolescents exhibiting more high-risk behavior in the Wheel of Fortune task had lower activation of the ACC, mPFC, and orbitofrontal cortex (OFC) (Shad et al., 2011). This link between reduced ACC and mPFC activation and high-risk behavior supports our presumption that the similarly diminished mPFC (and ACC) reactivity that we observed in association with larger midsleep advances plausibly reflects the increased levels of self-reported risk-taking behavior reported in other studies examining weekday-weekend shifts in sleep timing (O'Brien & Mindell, 2005; Pasch, et al., 2010).
In contrast, the reduced reactivity within the caudate is inconsistent with the hypothesis of increased ventral striatal activation associated with elevated risk-taking (Forbes, et al., 2010; Galvan, 2010). The greater reward-seeking behavior of adolescence has been postulated to reflect more striatal sensitivity to reward, which would suggest greater response to monetary reward. One possible interpretation of this nonintuitive finding, albeit one not testable within the current design, is that the reduced caudate reactivity represents less reactivity to low-intensity rewards. Such a pattern might lead to compensatory seeking of high-intensity rewards, which when paired with the reduced regulatory control implied by the diminished mPFC reactivity, might lead in turn to elevated risk-taking behavior.
Our finding of reduced striatal reactivity in association with larger weekend-weekday advances also differs from adult studies of the effect of sleep deprivation on reward processing. In those adult studies, increased striatal reactivity while winning money or observing pleasant images has been reported after sleep deprivation (Gujar, et al., 2011; Venkatraman, et al., 2007; Venkatraman, et al., 2011). This difference between midsleep shift and sleep deprivation in the direction of correlation with neural response to reward buttresses our contention that there is an important distinction between circadian misalignment and sleep loss in adolescents. Alternatively, the differences between our findings and these previous studies could reflect developmental differences in reward processing between adolescents and adults.
We did not find any relationships between weekend shift-associated reward-related brain activation and our measures of affective functioning (the PANAS-C and MFQ). Nor were the weekend shifts in sleep timing related to positive affect or depression severity. These null findings may stem from the use of a healthy adolescent sample with a restricted range of affective function. An alternative and non-mutually exclusive interpretation is that variable weekday-weekend sleep timing is more linked to the regulation of risk-taking than to affective function. This latter explanation is consistent with prior studies that also reported that weekend shifts in sleep timing were related to substance use and other risk-taking behavior, but were not related to depression severity (O'Brien & Mindell, 2005; Pasch, et al., 2010). Likewise, Hasler and colleagues (2008) reported that circadian misalignment was associated with substance abuse, but not general mental health symptoms. Notably, sleep loss may be more closely linked to affective function; weeknight sleep duration has been more consistently linked to depression severity than to risk-taking behaviors (O'Brien & Mindell, 2005; Pasch, et al., 2010). While preliminary, taken together, these data suggest a dissociation in the relationship between different aspects of sleep/circadian function and reward-related function in which circadian misalignment is linked to risk-taking but sleep loss is linked to affective dysregulation.
Our exploratory analyses of the effect of pubertal stage on the association between weekend shifts and reward-related brain function produced somewhat puzzling findings. The data suggest that large weekend shifts in sleep timing are more strongly linked to impaired mPFC response to reward in the more mature adolescents. Meanwhile, while there was less disparity between the two groups when focusing on the striatum, if anything, it was the less developed adolescents showing a stronger association between weekend shifts and reduced striatal response to reward. One possible interpretation is that weekend shifts are relatively more linked to reward regulation in the more mature group (based on less prefrontal control of reward responding), and relatively more linked to basic reward reactivity in the less developed group (based on less basic striatal response to reward). Perhaps maturing self-regulatory regions of the PFC are particularly vulnerable to disrupted function during mid-to-late puberty, which could set the stage for difficulty regulating responses to reward and initiating risk-taking behaviors. Because reward-seeking behaviors tend to peak in mid-adolescence, after puberty is completed, the development of effective engagement in mPFC regions relevant to regulating reward reactivity could be sensitive during mid/late puberty. For adolescents with a more pronounced shift in sleep, puberty could influence the disinhibition of regulatory regions that influence reward-related behavior. This could indicate that variable sleep timing poses a greater risk to the more mature adolescents, as it might diminish their ability to self-regulate. However, the opposite pathway is also plausible; the weakened self-regulation in the mid/late pubertal group, which could be a correlate of links between puberty and brain development, could lead to more varying schedules. Finally, an alternative explanation of pubertal status differences is that the two developmental groups, who broadly differed in age, perceived the financial reward in the task differently.
While novel and focused on objective measures of sleep and reward function, our data are similarly limited by the same cross-sectional design as previous studies of weekday-weekend sleep timing shifts in adolescents. In order to ascertain the direction of effect—does variable sleep timing lead to alterations in reward-related brain function, or do these alterations lead to dysregulated adolescents selecting more variable sleep schedules—prospective, experimental designs will be necessary. In addition, although some extent of circadian misalignment can be presumed based on these sleep shifts, precise quantification of circadian misalignment will require physiological assessments of circadian phase (e.g., dim light melatonin onset). Similarly, our study would have benefitted from a different timing of the fMRI scan relative to actigraphy assessment. Our scans occurred midweek, and mostly prior to the actigraphy assessment, suggesting that we may be tapping more of a trait-level of reward-related brain function or previous weekend sleep shifts rather than any proximal effect of the assessed weekend shift in sleep timing. Future studies should include scans both immediately before and immediately after the weekend to address this, as well as collecting actigraphy throughout the week to allow comparisons between weekend and midweek sleep timing. Longer spans of assessment that include multiple weeks would improve the reliability of the weekend-weekday shift measure. Furthermore, our study included assessments during summer and holiday breaks, during which weekend-weekday shifts in sleep timing tended to be of a smaller magnitude. Although this serves to increase the heterogeneity in weekend-weekday sleep shifts—thereby improving the chances of detecting a statistical effect—future studies should be designed for more systematic consideration of differences in sleep and reward functioning during, and outside, the academic year. Next, our sample of young, healthy adolescents with low, subclinical levels of depression and next-to-no substance use reduced our statistical power to examine associations with affective function and risk-taking behavior. A more representative sample of adolescents, spanning the range of healthy to disordered or older adolescents who are more likely to have initiated substance use and other risky behaviors, would address this limitation. Future studies in this area should include both self-report and behavioral measures of sensation-seeking or other reward-driven behaviors. Finally, to test the direction of effects rather than simply correlations among these constructs, it will be essential to conduct experimental laboratory studies in which sleep timing is manipulated and its effects on neural and behavioral responses to reward can be measured.
In conclusion, these preliminary data suggest that adolescents who pursue a later sleep schedule on the weekend, only to have return to a much earlier sleep schedule during the school week, show an altered pattern of brain reactivity to the anticipation and receipt of rewards. Notably, the results suggest that larger disparities between weekday and weekend sleep timing, as a proxy for circadian misalignment, are associated with reduced activation in both cortical and subcortical reward-related brain regions. These effects could reflect reduced regulatory response, and reduced reward sensitivity, which could result in compensatory reward-seeking behavior. Circadian misalignment associated with weekend shifts in sleep timing may contribute to reward-related problems such as depression, substance abuse and other risk-taking behavior. Furthermore, we have recently reported that neural response to reward in this study was associated with a polymorphism in the circadian-relevant PER2 gene that has also been linked to psychosis (Forbes et al., 2012). This suggests that the combination of genetic and behavioral circadian variability could have relevance to forms of psychopathology associated with disrupted reward function. Adolescents with a greater tendency toward eveningness and/or more delayed circadian phase are likely to be vulnerable to these effects.
HIGHLIGHTS.
We examined irregular sleep timing and reward-related brain function in adolescents.
We used actigraphy to assess weekend-weekday shifts in the timing of midsleep.
We used an fMRI monetary reward paradigm to probe reward reactivity.
Larger weekend shifts in midsleep correlated with diminished reward reactivity.
ACKNOWLEDGEMENTS
The authors would like to thank the adolescents and their parents who participated in this study. The authors would also like to thank Ashleigh Dominick, Thomas Olino, PhD, and the staff of the University of Pittsburgh Magnetic Resonance Research Center for their invaluable technical assistance. This work was supported by grants from the National Institutes of Health, including DA018910 (Dahl), DA026222 (Forbes, Shaw), MH074769 (Forbes), and T32HL082610 (Buysse).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
REFERENCES CITED
- Amodio DM, Frith CD. Meeting of minds: the medial frontal cortex and social cognition. Nature reviews. Neuroscience. 2006;7(4):268–277. doi: 10.1038/nrn1884. [DOI] [PubMed] [Google Scholar]
- Angold A, Erkanli A, Silberg J, Eaves L, Costello EJ. Depression scale scores in 8-17-year-olds: effects of age and gender. Journal of child psychology and psychiatry, and allied disciplines. 2002;43(8):1052–1063. doi: 10.1111/1469-7610.00232. [DOI] [PubMed] [Google Scholar]
- Carskadon MA, Acebo C, Jenni OG. Regulation of Adolescent Sleep: Implications for Behavior. Annals of the New York Academy of Sciences. 2004;1021(1):276–291. doi: 10.1196/annals.1308.032. [DOI] [PubMed] [Google Scholar]
- Chelminski I, Ferraro F, Petros T, Plaud J. An analysis of the “eveningnessmorningness” dimension in “depressive” college students. Journal of Affective Disorders. 1999;52(1-3):19–29. doi: 10.1016/s0165-0327(98)00051-2. [DOI] [PubMed] [Google Scholar]
- Crowley SJ, Acebo C, Carskadon MA. Sleep, circadian rhythms, and delayed phase in adolescence. Sleep Medicine. 2007;8(6):602–612. doi: 10.1016/j.sleep.2006.12.002. [DOI] [PubMed] [Google Scholar]
- Crowley SJ, Carskadon MA. Modifications to weekend recovery sleep delay circadian phase in older adolescents. Chronobiology international. 2010;27(7):1469–1492. doi: 10.3109/07420528.2010.503293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Drennan M, Klauber M, Kripke D, Goyette L. The effects of depression and age on the Horne-Ostberg morningness-eveningness score. Journal of Affective Disorders. 1991;23(2):93–98. doi: 10.1016/0165-0327(91)90096-b. [DOI] [PubMed] [Google Scholar]
- Emens J, Lewy A, Kinzie JM, Arntz D, Rough J. Circadian misalignment in major depressive disorder. Psychiatry Research. 2009;168(3):259–261. doi: 10.1016/j.psychres.2009.04.009. [DOI] [PubMed] [Google Scholar]
- Ernst M, Fudge JL. A developmental neurobiological model of motivated behavior: anatomy, connectivity and ontogeny of the triadic nodes. Neuroscience & Biobehavioral Reviews. 2009;33(3):367–382. doi: 10.1016/j.neubiorev.2008.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Forbes EE, Dahl RE, Almeida JR, Ferrell RE, Nimgaonkar VL, Mansour H, Phillips ML. PER2 rs2304672 polymorphism moderates circadian-relevant reward circuitry activity in adolescents. Biological psychiatry. 2012;71(5):451–457. doi: 10.1016/j.biopsych.2011.10.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Forbes EE, Hariri AR, Martin SL, Silk JS, Moyles DL, Fisher PM, Dahl RE. Altered striatal activation predicting real-world positive affect in adolescent major depressive disorder. American Journal of Psychiatry. 2009;166(1):64–73. doi: 10.1176/appi.ajp.2008.07081336. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Forbes EE, Ryan ND, Phillips ML, Manuck SB, Worthman CM, Moyles DL, Dahl RE. Healthy adolescents’ neural response to reward: associations with puberty, positive affect, and depressive symptoms. Journal of the American Academy of Child and Adolescent Psychiatry. 2010;49(2):162–172. e161–165. doi: 10.1097/00004583-201002000-00010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Sleep Foundation [July 18, 2011];2006 Sleep in America poll: Summary findings. 2006 http://www.sleepfoundation.org/_content/hottopics/2006_summary_of_findings.pdf.
- Galvan A. Adolescent development of the reward system. Frontiers in Human Neuroscience. 2010;4:6. doi: 10.3389/neuro.09.006.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gau SS, Shang CY, Merikangas KR, Chiu YN, Soong WT, Cheng AT. Association between morningness-eveningness and behavioral/emotional problems among adolescents. Journal of Biological Rhythms. 2007;22(3):268–274. doi: 10.1177/0748730406298447. [DOI] [PubMed] [Google Scholar]
- Giannotti F, Cortesi F, Sebastiani T, Ottaviano S. Circadian preference, sleep and daytime behaviour in adolescence. Journal of Sleep Research. 2002;11(3):191–199. doi: 10.1046/j.1365-2869.2002.00302.x. [DOI] [PubMed] [Google Scholar]
- Gujar N, Yoo SS, Hu P, Walker MP. Sleep deprivation amplifies reactivity of brain reward networks, biasing the appraisal of positive emotional experiences. Journal of Neuroscience. 2011;31(12):4466–4474. doi: 10.1523/JNEUROSCI.3220-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haber SN, Knutson B. The reward circuit: linking primate anatomy and human imaging. [Review]. Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology. 2010;35(1):4–26. doi: 10.1038/npp.2009.129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hasler BP, Allen JJB, Sbarra DA, Bootzin RR, Bernert RA. Morningness-eveningness and depression: Preliminary evidence for the role of BAS and positive affect. Psychiatry Research. 2010;176(2-3):166–173. doi: 10.1016/j.psychres.2009.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hasler BP, Bootzin RR, Cousins JC, Fridel K, Wenk GL. Circadian phase in sleep-disturbed adolescents with a history of substance abuse: a pilot study. Behavioral Sleep Medicine. 2008;6(1):55–73. doi: 10.1080/15402000701796049. [DOI] [PubMed] [Google Scholar]
- Hasler BP, Buysse DJ, Kupfer DJ, Germain A. Phase relationships between core body temperature, melatonin, and sleep are associated with depression severity: Further evidence for circadian misalignment in non-seasonal depression. Psychiatry Research. 2010;178(1):205–207. doi: 10.1016/j.psychres.2010.04.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Herman-Giddens ME, Slora EJ, Wasserman RC, Bourdony CJ, Bhapkar MV, Koch GG, Hasemeier CM. Secondary sexual characteristics and menses in young girls seen in office practice: a study from the Pediatric Research in Office Settings network. Pediatrics. 1997;99(4):505–512. doi: 10.1542/peds.99.4.505. [DOI] [PubMed] [Google Scholar]
- Holm SM, Forbes EE, Ryan ND, Phillips ML, Tarr JA, Dahl RE. Reward-related brain function and sleep in pre/early pubertal and mid/late pubertal adolescents. Journal of Adolescent Health. 2009;45(4):326–334. doi: 10.1016/j.jadohealth.2009.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jenni OG, Carskadon MA. Sleep Behavior and Sleep Regulation from Infancy through Adolescence: Normative Aspects. Sleep Medicine Clinics. 2007;2(3):321–329. [Google Scholar]
- Karpati AM, Rubin CH, Kieszak SM, Marcus M, Troiano RP. Stature and pubertal stage assessment in American boys: the 1988-1994 Third National Health and Nutrition Examination Survey. The Journal of adolescent health : official publication of the Society for Adolescent Medicine. 2002;30(3):205–212. doi: 10.1016/s1054-139x(01)00320-2. [DOI] [PubMed] [Google Scholar]
- Katon W, Russo J, Richardson L, McCauley E, Lozano P. Anxiety and depression screening for youth in a primary care population. Ambulatory pediatrics : the official journal of the Ambulatory Pediatric Association. 2008;8(3):182–188. doi: 10.1016/j.ambp.2008.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koob GF, Le Moal M. Neurobiological mechanisms for opponent motivational processes in addiction. Philosophical transactions of the Royal Society of London. Series B, Biological sciences. 2008;363(1507):3113–3123. doi: 10.1098/rstb.2008.0094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Laurent J, Catanzaro SJ, Joiner TE, Rudolph KD, Potter KI, Lambert S, Gathright T. A measure of positive and negative affect for children: Scale development and preliminary validation. Psychological assessment. 1999;11(3):326–338. [Google Scholar]
- Negriff S, Dorn LD, Pabst SR, Susman EJ. Morningness/Eveningness, pubertal timing, and substance use in adolescent girls. Psychiatry Research. 2011;185(3):408–413. doi: 10.1016/j.psychres.2010.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O'Brien EM, Mindell JA. Sleep and risk-taking behavior in adolescents. Behavioral Sleep Medicine. 2005;3(3):113–133. doi: 10.1207/s15402010bsm0303_1. [DOI] [PubMed] [Google Scholar]
- Pasch KE, Laska MN, Lytle LA, Moe SG. Adolescent sleep, risk behaviors, and depressive symptoms: are they linked? American Journal of Health Behavior. 2010;34(2):237–248. doi: 10.5993/ajhb.34.2.11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pieters S, Van Der Vorst H, Burk WJ, Wiers RW, Engels RC. Puberty-dependent sleep regulation and alcohol use in early adolescents. Alcoholism: Clinical and Experimental Research. 2010;34(9):1512–1518. doi: 10.1111/j.1530-0277.2010.01235.x. [DOI] [PubMed] [Google Scholar]
- Randler C. Differences between smokers and nonsmokers in morningness-eveningness. Social Behavior and Personality. 2008;36(5):673–680. [Google Scholar]
- Roenneberg T, Wirz-Justice A, Merrow M. Life between clocks: Daily temporal patterns of human chronotypes. Journal of Biological Rhythms. 2003;18(1):80–90. doi: 10.1177/0748730402239679. [DOI] [PubMed] [Google Scholar]
- Shad MU, Bidesi AS, Chen LA, Thomas BP, Ernst M, Rao U. Neurobiology of decision-making in adolescents. Behavioural brain research. 2011;217(1):67–76. doi: 10.1016/j.bbr.2010.09.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annual review of clinical psychology. 2008;4:1–32. doi: 10.1146/annurev.clinpsy.3.022806.091415. [DOI] [PubMed] [Google Scholar]
- Spear LP. The adolescent brain and age-related behavioral manifestations. Neuroscience & Biobehavioral Reviews. 2000;24(4):417–463. doi: 10.1016/s0149-7634(00)00014-2. [DOI] [PubMed] [Google Scholar]
- Taylor A, Wright HR, Lack LC. Sleeping-in on the weekend delays circadian phase and increases sleepiness the following week. Sleep and Biological Rhythms. 2008;6(3):172–179. [Google Scholar]
- Venkatraman V, Chuah YM, Huettel SA, Chee MW. Sleep deprivation elevates expectation of gains and attenuates response to losses following risky decisions. Sleep. 2007;30(5):603–609. doi: 10.1093/sleep/30.5.603. [DOI] [PubMed] [Google Scholar]
- Venkatraman V, Huettel SA, Chuah LY, Payne JW, Chee MW. Sleep deprivation biases the neural mechanisms underlying economic preferences. Journal of Neuroscience. 2011;31(10):3712–3718. doi: 10.1523/JNEUROSCI.4407-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wittmann M, Dinich J, Merrow M, Roenneberg T. Social jetlag: Misalignment of biological and social time. Chronobiology International. 2006;23(1&2):497–509. doi: 10.1080/07420520500545979. [DOI] [PubMed] [Google Scholar]
- Wolfson AR, Carskadon MA. Understanding adolescents’ sleep patterns and school performance: a critical appraisal. Sleep Medicine Reviews. 2003;7(6):491–506. doi: 10.1016/s1087-0792(03)90003-7. [DOI] [PubMed] [Google Scholar]
- Wood A, Kroll L, Moore A, Harrington R. Properties of the mood and feelings questionnaire in adolescent psychiatric outpatients: a research note. Journal of child psychology and psychiatry, and allied disciplines. 1995;36(2):327–334. doi: 10.1111/j.1469-7610.1995.tb01828.x. [DOI] [PubMed] [Google Scholar]
- Wu T, Mendola P, Buck GM. Ethnic differences in the presence of secondary sex characteristics and menarche among US girls: the Third National Health and Nutrition Examination Survey, 1988-1994. [Comparative Study]. Pediatrics. 2002;110(4):752–757. doi: 10.1542/peds.110.4.752. [DOI] [PubMed] [Google Scholar]