Abstract
Objective
Given the growing concern about research reproducibility, we conceptually replicated a previous analysis of the relationships between adolescent sleep and mental well-being, using a new dataset.
Methods
We conceptually reproduced an earlier analysis (Sleep Health, June 2017) using baseline data from the START Study. START is a longitudinal research study designed to evaluate a natural experiment in delaying high school start times, examining the impact of sleep duration on weight change in adolescents. In both START and the previous study, school day bedtime, wake up time, and answers to a six-item depression subscale were self-reported using a survey administered during the school day. Logistic regression models were used to compute the association and 95% confidence intervals (CI) between the sleep variables (sleep duration, wake-up time and bed time) and a range of outcomes.
Results
In both analyses, greater sleep duration was associated with lower odds (P < 0.0001) of all six indicators of depressive mood. Five of the six sleep duration point estimates from the START Study, and four of the six wake-up time point estimates, fell within the 95% confidence intervals from the previous analysis. However, the associations between wake-up time and outcomes differed between the two studies’ analyses.
Conclusion
Our findings add strength to the evidence supporting an association between short sleep duration and depression. This issue deserves attention from school districts, given the current epidemic of short sleep duration among youth and the potential impact school scheduling can have on teen sleep.
Keywords: Sleep, high school start time, replication, adolescents, epidemiology, mental health
Introduction
Mental health concerns are prevalent among youth with an estimate of up to 20% of US children experiencing a mental health disorder in the past year.1 Adolescents who do not get adequate sleep suffer diminished mental well-being. Among teens, shorter sleep duration has been correlated with higher rates of mood disorders2–7 and more thoughts of suicide.5,7–9 Further, sleep debt appears to relate to lower self-esteem10, a more negative outlook in life,11 and increased struggles with emotional regulation.12 Our recently published report13 of a school-based study revealed that shorter sleep duration was associated with decreased mental well-being. Currently, there is an epidemic of short sleep duration among adolescents with less than 25% of US teens reporting they attain at least eight hours of sleep per night;13 the National Sleep Foundation considers eight to 10 hours to be the optimal amount of sleep for this age group.14
Mechanistically, insufficient sleep in teens may contribute generally to a reduction in mental well-being by diminishing their executive cognitive functioning and emotional regulation.15 Getting inadequate sleep throughout the week causes many adolescents to live in ongoing “circadian misalignment” due to the discrepancy between the school-imposed schedule and their own internal clocks. Further, due to being exhausted from a week of inadequate sleep, adolescents will sleep dramatically more hours on weekends to make up for school day sleep loss. Differences in weekend and weekday sleep hours have been associated with impaired regulation of reward processing,16,17 which can manifest as increased sensation-seeking and diminished regulatory control.15,18 Finally, getting insufficient sleep might also influence teens’ mental health and outlook by altering their overall daily structure and how they choose to use their time. For instance, having more waking hours, especially in the evening could hypothetically lead to youth engaging in more risk behaviors, such as substance use. Alternatively, longer after-school times could promote potentially protective activities such as participating in an extracurricular activity, which in turn can influence mental well-being.
In the earlier Sleep Health (June 2017) article, we conducted a “conceptual” replication of a previously published analysis of observational data looking at the relationship between sleep and mental well-being among high school students,19 using a dataset obtained from ninth grade students at five high schools. This prior analysis had revealed that greater sleep duration was associated with greater reported mental well-being and that later wake-up times were also associated with several (though not all) indicators of improved mental well-being.
In recent years, multiple authors20,21 have written about a “replication crisis” in science, where many health and social sciences findings have not been reproduced or where even an attempt at replication has not been possible. Scientific replication is often thought of as an exact duplication of the treatment, outcome measurement, procedure, and population of an original study.22 However, given the realities of research with free-living human populations, numerous social scientists have argued for broader conceptions of replication, which they have termed partial or conceptual replications, allowing both the re-test of hypotheses and/or examining results of earlier studies with extension to new populations or methods.22–25 With the growing concern about the reproducibility of observational biomedical research,20,22 we sought to present a conceptual replication of the prior study.
Participants and Methods
Objectives
Our June 2017 article19 in Sleep Health (hereafter referred to as Study 1) described the relationships between sleep duration and adolescent behaviors in a multi-site study. We aimed to replicate that analysis using data from the baseline wave of data collection (Spring 2016) of the START study, a longitudinal cohort study designed to identify the effects of a natural experimental change in high school start times on weight and weight-related behavior.
Study 1
Study 1’s methodology was reported in depth in its original publication.19 Study 1 was conducted from 2010–2013, and involved 9,089 students in grades 9–12 attending eight high schools in five US school districts. Data was collected in Minnesota (Spring 2011), Colorado (Spring 2012), and Wyoming (Spring 2013). In one participating high school in Jackson Hole, Wyoming, students were surveyed at baseline in 2012 which was the year prior to their high school’s start time delay, and again after the delay was in effect in 2013. The other seven participating schools were surveyed just once, after a later start time had been implemented. The survey focused on sleep and selected health, academic, and behavioral issues. The self-administered paper and pencil surveys were sent to all schools and completed by students during the school day. The response proportions for this study varied by school, from 65–85%.19
Study 2, START
The START study was designed to leverage a natural experiment in high school start time modification to test how sleep duration influences weight change in adolescents. At baseline, in 2016, all ninth graders at five participating Minnesota high schools, with start times at 7:30am, were invited to enroll in the study. The cohort of students who enrolled in the study as ninth-graders are being followed through their 11th grade year. The current replication analysis uses data from START’s baseline wave when 90% of eligible adolescents participated in the survey portion of START (n = 2,133; final analytic sample n = 2,004), which includes items related to mental health.
Measures
In both studies, school-day bedtime and wake up time were self-reported and used to calculate sleep duration. START participants were asked to select one time from a list for each of, “About what time do you usually go to bed ON SCHOOL DAYS?” and “About what time do you usually wake up ON SCHOOL DAYS?” Depressive mood was assessed with six items derived from the Kandel-Davies depressed mood scale (see Table 1).26 Study 1 and START had slightly different response options for this measure. In START, we dichotomized participant responses into “not at all” or “somewhat” vs. “very much” bothered/troubled during the past two weeks for each item. For Study 1, we dichotomized and compared the response options indicating several times or more vs. twice or less during the past two weeks for each item. In both studies the correlations between the individual items within the scale ranged from r = 0.27- r = 0.61.
Table 1.
Measures of depressive mood in relation to sleep duration, wake-up time and bedtime on school days in Study 1 (2010–2013: previously published19) [n = 8261] and Study 2 (2016: the START study) [n = 2004].
| During the last two weeks, how often have you been bothered or troubled by... | Bedtime | Wake-up time | Sleep duration | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| aORa | 95% CI | P | aORa | 95% CI | P | aORa | 95% CI | P | |||||
| Feeling too tired to do things | Study 1b | 1.48 | 1.41 | 1.55 | <.0001 | 0.94 | 0.83 | 1.06 | 0.308 | 0.68 | 0.67 | 0.70 | <.0001 |
| Study 2 (START)c | 1.12e | 0.96 | 1.31 | 0.136 | 1.12d | 1.02 | 1.23 | 0.015 | 0.70 | 0.65 | 0.76 | <.0001 | |
| Having trouble going to sleep or staying asleep | Study 1b | 1.31 | 1.23 | 1.41 | <.0001 | 0.97 | 0.92 | 1.02 | 0.214 | 0.77 | 0.71 | 0.84 | <.0001 |
| Study 2 (START)c | 1.32 | 0.77 | 2.26 | 0.309 | 0.93 | 0.53 | 1.62 | 0.787 | 0.59d | 0.49 | 0.73 | <.0001 | |
| Feeling unhappy, sad, or depressed | Study 1b | 1.37 | 1.30 | 1.44 | <.0001 | 0.87 | 0.78 | 0.96 | 0.006 | 0.72 | 0.68 | 0.75 | <.0001 |
| Study 2 (START)c | 1.15e | 0.86 | 1.53 | 0.335 | 0.92 | 0.75 | 1.13 | 0.446 | 0.68 | 0.61 | 0.77 | <.0001 | |
| Feeling hopeless about the future | Study 1b | 1.32 | 1.26 | 1.39 | <.0001 | 0.85 | 0.78 | 0.92 | <.0001 | 0.72 | 0.69 | 0.76 | <.0001 |
| Study 2 (START)c | 1.07e | 0.75 | 1.52 | 0.724 | 0.90 | 0.63 | 1.28 | 0.560 | 0.71 | 0.61 | 0.84 | <.0001 | |
| Feeling nervous or tense | Study 1b | 1.28 | 1.22 | 1.35 | <.0001 | 0.79 | 0.68 | 0.93 | 0.006 | 0.74 | 0.70 | 0.78 | <.0001 |
| Study 2 (START)c | 0.75e | 0.56 | 1.01 | 0.057 | 0.75 | 0.58 | 0.96 | 0.022 | 0.75 | 0.71 | 0.80 | <.0001 | |
| Worrying too much about things | Study 1b | 1.30 | 1.25 | 1.36 | <.0001 | 0.78 | 0.70 | 0.88 | <.0001 | 0.72 | 0.69 | 0.76 | <.0001 |
| Study 2 (START)c | 0.78e | 0.69 | 0.88 | <.0001 | 0.68d | 0.57 | 0.82 | <.0001 | 0.76 | 0.71 | 0.80 | <.0001 | |
aOR, adjusted odds ratio; CI, confidence interval
Odds ratios for difference in risk of depressive mood measure for each additional hour of delayed wake-up or sleep duration time, adjusted in previously published form for grade level, sex, race, medication for learning disability, usual grades in school, and school-level free and reduced lunch eligibility proportion. START Study ORs adjusted for sex, race, and school-level free and reduced lunch eligibility proportion.
Teen Sleep Habits Survey: “Several times or more” vs. “Twice or less” during the last 2 weeks.
START Study: “Very much” versus “Somewhat” or “Not at all” during the last 2 weeks.
START Study point estimate falling outside the 95% CI of our published estimate.
START Study point estimate falling outside the 95% CI of previously unpublished association with Study 1 data.
Analysis
We took the following similar analytic approach to the START (Study 2) data as we had done in our previously published report, which used data from Study 1. Logistic regression models were used to compute the association and 95% confidence intervals (CI) between the sleep variables (sleep duration, wake-up time and bed time) and the mental well-being outcomes. As a supplemental analysis, we also examined associations between bedtime and mental well-being in START and also ran these analyses on the Study 1 dataset. To account for variance inflation that results from the clustering of students within school and school district, we used generalized estimating equations. Consistent with the previous analysis, all START models were adjusted for gender, racial/ethnic background and school-level eligibility for free and reduced lunch.
Results
Table 1 shows the associations between aspects of sleep timing (bedtime, wake up time, and sleep duration) and each of the six indicators of depressive mood (feeling too tired to do things; having trouble going to sleep or staying asleep; feeling unhappy, sad or depressed; feeling hopeless about the future; feeling nervous or tense; and worrying too much about things). Greater sleep duration was associated with lower odds (P < 0.0001) of all six indicators of depressive mood in the START Study, as it was also found in Study 1. We did not find any evidence of effect modification by either race/ethnicity or sex.
Five of the six sleep duration point estimates from the START Study, and four of the six wake-up time point estimates, fell within the 95% confidence intervals published in the June 2017 Sleep Health article that had used data from Study 1. However, we did not find significant associations in START data between wake-up time and two of the items for which we reported a significant association in the Study 1 analysis, specifically “feeling unhappy, sad, or depressed,” and “feeling hopeless about the future.” An additional divergence from the findings of the earlier study was a significant positive association between wake-up time and “feeling too tired to do things” (aOR 1.12, 95% CI 1.02 – 1.23) found in the START study.
Later bedtime was significantly associated with all six outcomes in Study 1, with a fairly narrow range for the association (one-hour delay in bedtime, aOR between 1.28 and 1.48, P < .0001 for all outcomes). Among START Study participants there were no significant association between bedtime and feeling too tired to do things, having trouble going to sleep or staying asleep, feeling unhappy, sad or depressed, feeling hopeless about the future, or feeling nervous or tense. Those with later bedtimes in START were less likely to report “worrying too much about things” (aOR for a one-hour delay in bed time 0.78, P < .0001). Each hour of later start time in Study 1 associated with a 14-minute later bedtime and a 42 minute later wake-up time.
Discussion
The overall concordance of the START study (Study 2) findings with the previously published Study 1 results provides additional support for both greater sleep duration and later wake-up time being associated with fewer mental health-related issues, although later wake-up time was positively associated with one symptom of depression (feeling too tired to do things) in the START study. The association of later bedtime with poorer mental well-being observed in Study 1 was not seen in START. Two outcomes that were positively associated with later bedtime in Study 1 (“worrying too much about things” and feeling nervous ) were negatively associated with later bedtime in START, with “worrying too much about things” having a statistically significant association with bedtime.
An important difference between the START Study and Study 1 is that more of the variation in sleep timing in START (where all schools started at 7:30am) may be due to differences among the students themselves and their sleep timing rather than their schools, because in Study 1, participants’ school start times varied considerably. For example, the differing estimates of the associations between bedtime or wake-up time and mental well-being in Study 1 vs. START (Study 2) may be due to the limited range of potential sleep timing in the baseline wave of START data. The observed associations between either bedtimes or wakeup times and mental well-being is likely due to a mixture of the causal effect of the timing of sleep on depression and anxiety, the fact that particular bedtimes and wakeup times define individuals’ total sleep duration, and the unobserved confounding variables, such as family relationships, that affect both sleep timing and mental health. Within START, timing might be less influential since it would be inherently more constrained within this study’s sample, since all START schools started at 7:30am.
A better understanding of sleep’s influence on mood is a consequential area of inquiry, since adolescent sleep duration could potentially be improved population-wide with a policy action to delay high school start time. Although there is a multitude of factors that influence the number of hours that adolescents sleep, research has shown that the time high schools begin is one of the most salient forces determining the amount of sleep teens get.27 At puberty, there are neurobiological changes to children’s circadian clocks and sleep-wake homeostasis that postpone adolescents’ sleep-wake cycles relative to those of younger children.28 Even in highly sleep-conducive settings, due to teens’ circadian biology, many adolescents will struggle to fall asleep earlier than 11pm or wake before 8am. High schools that start before 8:30am inevitably truncate students’ necessary sleep. Fortunately, when high schools shift their start times to 8:30am or later,29 adolescents are enabled to get more sleep.30,31 There is a suggestion in the literature that later high school start times are associated with better mental well-being by delaying students’ required wake-up time and enhancing sleep duration. However, studies on this topic are few19,32,33 and often have methodological limitations.34 If increasing adolescent sleep duration by delaying school start time is to have an effect on their risk of depression, there would need to be a causal relationship sleep duration and depressive mood.
We feel this work is important, not just because it adds to the previous literature on the relationship between adolescent sleep and mental well-being which has school policy implications, but also because previous studies are difficult to compare directly due to differing methodologies and measures.35 “Conceptual” or “partial” replications, like the one presented here, have the same information focus as previous research (e.g. measuring the association between sleep and depressive mood), while duplicating some procedural variables or even using radically different methods.22 A broader conception of replication allows not only for control of sampling error and/or fraud, but also allows one to generalize from results and to verify the underlying hypotheses of previous research.25Although START (Study 2) and Study 1 differ in the selection of participating schools, the students recruited, the sample sizes, the temporal setting, and the precise content of their respective sleep and behavior questionnaires, those differences do not diminish the value of re-testing the core hypothesis of Study 1. In fact, we argue that these differences strengthen the value of the replication as our results suggest the findings have greater external validity and transportability. The findings from START (Study 2), where we used nearly identical measures and analytical methods as those used in Study 1, add weight to what was previously reported in Study 1 which took place in different populations of adolescents roughly five years prior. Had START failed to reproduce the findings from Study 1, we may have been left wondering if the differing context or sampling strategy caused the failure, or if the association never existed in the first place. By reproducing the earlier findings, however, we are able to increase our confidence that this association exists in various settings and with differing populations, and may be modifiable.
A topic such as population-level sleep change and its impact on various outcomes over longer periods of time is an important direction for future research, but unfortunately is not amenable to exploration through a randomized, controlled experiments (RCT), which leaves us observational studies for discovery in this area. Despite this dilemma, inference on causal questions is not an intractable problem. In the case of adolescent sleep, at any given time, there are schools across the US revising their start times in response to a variety of issues including transportation, population changes within a district, and concern for how school schedules impact student well-being. Given that it is established within a large body of longitudinal and cross-sectional research that school start times determine students’ sleep duration,34,36 there are opportunities to study outcomes of these natural experiments and to potentially conduct quasi-replication studies. Developments in methodologies for evaluating natural experiments have allowed for strengthened causal inference in these types of exposure-outcome questions where an RCT is not possible.37 Reassuringly, within-study comparisons have shown that natural experiment evaluation methodologies can be reasonably comparable to results from RCTs.38
Conclusions
Although we know that adolescents require 8–10 hours of sleep for optimal functioning,14 very few teens are able to get this much sleep on school nights.13 Improving sleep duration could potentially enhance mental well-being. Given that short sleep duration among adolescents is a population-wide issue, a public health approach is needed. Although there is a multitude of factors that influence the number of hours that adolescents sleep, research has shown that the time high schools begin is one of the most salient forces determining the amount of sleep teens get.27 Because adolescent sleep/wake times are determined, in a large part, by biological factors, a policy that allows for later wake-up times can facilitate a more optimal dose of sleep. This issue deserves attention from school districts, given adolescents’ widespread exposure to early start times in the US, the current epidemic of short sleep duration among youth, and the growing evidence that short sleep duration can seriously and negatively impact health youth development.13,39
Acknowledgements
This work was supported by the National Institutes of Health Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01 HD088176), the Centers for Disease Control and Prevention, (CDC 5U48DP001939 -SIP10-035), and the Teton County School District. The study sponsors had no role in the study design, data collection, analysis, interpretation of the data, writing, or decision to submit for publication.
Bibliography
- 1.Perou R, Bitsko RH, Blumberg SJ, Pastor P, Ghandour RM, Gfroerer JC, Hedden SL, Crosby AE, Visser SN, Schieve LA, Parks SE, Hall JE, Brody D, Simile CM, Thompson WW, Baio J, Avenevoli S, Kogan MD, Huang LN, Centers for Disease Control and Prevention (CDC). Mental health surveillance among children--United States, 2005–2011. MMWR Suppl. 2013. May 17;62(2):1–35. [PubMed] [Google Scholar]
- 2.Harvey AG, Alfano CA, Clarke G. Mood disorders In: Wolfson AR, Montgomery-Downs HE, editors. Oxf Handb Infant Child Adolesc Sleep Behav. Ontario, Canada: Oxford University Press; In Press. p. 515–531. [Google Scholar]
- 3.Beebe DW. Cognitive, behavioral, and functional consequences of inadequate sleep in children and adolescents. Pediatr Clin North Am. 2011. June;58(3):649–665. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Shochat T, Cohen-Zion M, Tzischinsky O. Functional consequences of inadequate sleep in adolescents: A systematic review. Sleep Med Rev. 2014. February;18(1):75–87. [DOI] [PubMed] [Google Scholar]
- 5.Zhang J, Paksarian D, Lamers F, Hickie IB, He J, Merikangas KR. Sleep Patterns and Mental Health Correlates in US Adolescents. J Pediatr. 2017. March;182:137–143. [DOI] [PubMed] [Google Scholar]
- 6.Pasch KE, Laska MN, Lytle LA, Moe SG. Adolescent sleep, risk behaviors, and depressive symptoms: are they linked? Am J Health Behav. 2010. April;34(2):237–248. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Sarchiapone M, Mandelli L, Carli V, Iosue M, Wasserman C, Hadlaczky G, Hoven CW, Apter A, Balazs J, Bobes J, Brunner R, Corcoran P, Cosman D, Haring C, Kaess M, Keeley H, Keresztény A, Kahn J-P, Postuvan V, Mars U, Saiz PA, Varnik P, Sisask M, Wasserman D. Hours of sleep in adolescents and its association with anxiety, emotional concerns, and suicidal ideation. Sleep Med. 2014. February;15(2):248–254. [DOI] [PubMed] [Google Scholar]
- 8.Fitzgerald CT, Messias E, Buysse DJ. Teen sleep and suicidality: results from the youth risk behavior surveys of 2007 and 2009. J Clin Sleep Med JCSM Off Publ Am Acad Sleep Med. 2011. August 15;7(4):351–356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lee YJ, Cho S-J, Cho IH, Kim SJ. Insufficient sleep and suicidality in adolescents. Sleep. 2012;35(4):455–460. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Fredriksen K, Rhodes J, Reddy R, Way N. Sleepless in Chicago: tracking the effects of adolescent sleep loss during the middle school years. Child Dev. 2004. February;75(1):84–95. [DOI] [PubMed] [Google Scholar]
- 11.Perkinson-Gloor N, Lemola S, Grob A. Sleep duration, positive attitude toward life, and academic achievement: the role of daytime tiredness, behavioral persistence, and school start times. J Adolesc. 2013. April;36(2):311–318. [DOI] [PubMed] [Google Scholar]
- 12.Dahl RE, Lewin DS. Pathways to adolescent health sleep regulation and behavior. J Adolesc Health Off Publ Soc Adolesc Med. 2002. December;31(6 Suppl):175–184. [DOI] [PubMed] [Google Scholar]
- 13.Eaton DK, McKnight-Eily LR, Lowry R, Perry GS, Presley-Cantrell L, Croft JB. Prevalence of insufficient, borderline, and optimal hours of sleep among high school students - United States, 2007. J Adolesc Health Off Publ Soc Adolesc Med. 2010. April;46(4):399–401. [DOI] [PubMed] [Google Scholar]
- 14.Hirshkowitz M, Whiton K, Albert SM, Alessi C, Bruni O, DonCarlos L, Hazen N, Herman J, Katz ES, Kheirandish-Gozal L, Neubauer DN, O’Donnell AE, Ohayon M, Peever J, Rawding R, Sachdeva RC, Setters B, Vitiello MV, Ware JC, Hillard PJA. National Sleep Foundation’s sleep time duration recommendations: methodology and results summary. Sleep Health J Natl Sleep Found. 2015. March 1;1(1):40–43. [DOI] [PubMed] [Google Scholar]
- 15.Edwards S, Reeves GM, Fishbein D. Integrative Model of the Relationship Between Sleep Problems and Risk for Youth Substance Use. Curr Addict Rep. 2015. April 3;2(2):130–140. [Google Scholar]
- 16.Hasler BP, Smith LJ, Cousins JC, Bootzin RR. Circadian rhythms, sleep, and substance abuse. Sleep Med Rev. 2012. February;16(1):67–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hasler BP, Clark DB. Circadian Misalignment, Reward-Related Brain Function, and Adolescent Alcohol Involvement. Alcohol Clin Exp Res. 2013. April 1;37(4):558–565. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hasler BP, Soehner AM, Clark DB. Sleep and circadian contributions to adolescent alcohol use disorder. Alcohol. 2015. June;49(4):377–387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Wahlstrom KL, Berger AT, Widome R. Relationships between school start time, sleep duration, and adolescent behaviors. Sleep Health. 2017. June;3(3):216–221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Collins FS, Tabak LA. Policy: NIH plans to enhance reproducibility. Nature. 2014. January 30;505(7485):612–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Collaboration OS. Estimating the reproducibility of psychological science. Science. 2015. August 28;349(6251):aac4716. [DOI] [PubMed] [Google Scholar]
- 22.Hendrick C Replications, strict replications, and conceptual replications: Are they important? J Soc Behav Personal. 1990;5(4):41–49. [Google Scholar]
- 23.Hunter JE. The desperate need for replications. J Consum Res. 2001;28(1):149–158. [Google Scholar]
- 24.Adams AT, Ajrouch KJ, Henderson H, Heard I. Service-Learning Outcomes Research: The Role and Scarcity of Replication Studies Service-Learning Outcomes Research: The Role and Scarcity of Replication Studies. J Appl Sociol. 2005. September 1;os-22(2):55–71. [Google Scholar]
- 25.Schmidt S Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Rev Gen Psychol. 2009;13(2):90–100. [Google Scholar]
- 26.Kandel DB, Davies M. Epidemiology of depressive mood in adolescents: an empirical study. Arch Gen Psychiatry. 1982. October;39(10):1205–1212. [DOI] [PubMed] [Google Scholar]
- 27.Owens J, Au R, Carskadon M, Millman R, Wolfson A, Braverman PK, Adelman WP, Breuner CC, Levine DA, Marcell AV, Murray PJ, O’Brien RF. Insufficient Sleep in Adolescents and Young Adults: An Update on Causes and Consequences. Pediatrics. 2014. September 1;134(3):e921–e932. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Frey S, Balu S, Greusing S, Rothen N, Cajochen C. Consequences of the Timing of Menarche on Female Adolescent Sleep Phase Preference. PLoS ONE. 2009. April 22;4(4):e5217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Wahlstrom K School start time and sleepy teens. Arch Pediatr Adolesc Med. 2010. July;164(7):676–677. [DOI] [PubMed] [Google Scholar]
- 30.Wahlstrom KL. Accommodating the Sleep Patterns of Adolescents within Current Educational Structures: An Uncharted Path. Adolesc Sleep Patterns [Internet]. Cambridge University Press; 2002. Available from: 10.1017/CBO9780511499999.014 [DOI] [Google Scholar]
- 31.Wahistrom K Changing Times: Findings From the First Longitudinal Study of Later High School Start Times. NASSP Bull. 2002. December 1;86(633):3–21. [Google Scholar]
- 32.Wahlstrom K, Dretzke B, Gordon M, Peterson K, Edwards K, Gdula J. Examining the Impact of Later High School Start Times on the Health and Academic Performance of High School Students: A Multi-Site Study [Internet]. 2014. February Available from: http://conservancy.umn.edu/handle/11299/162769
- 33.Boergers J, Gable CJ, Owens JA. Later school start time is associated with improved sleep and daytime functioning in adolescents. J Dev Behav Pediatr JDBP. 2014. January;35(1):11–17. [DOI] [PubMed] [Google Scholar]
- 34.Marx R, Tanner-Smith EE, Davison CM, Ufholz L-A, Freeman J, Shankar R, Newton L, Brown RS, Parpia AS, Cozma I, Hendrikx S. Later school start times for supporting the education, health, and well-being of high school students. Cochrane Database Syst Rev. 2017. July 3;7:CD009467. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Berger AT, Widome R, Troxel WM. School Start Time and Psychological Health in Adolescents. Curr Sleep Med Rep. 2018. June 1;4(2):110–117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Wheaton AG, Chapman DP, Croft JB. School Start Times, Sleep, Behavioral, Health, and Academic Outcomes: A Review of the Literature. J Sch Health. 86(5):363–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Craig P, Katikireddi SV, Leyland A, Popham F. Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research. Annu Rev Public Health. 2017. March 20;38:39–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Cook TD, Shadish WR, Wong VC. Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. J Policy Anal Manage. 2008;27(4):724–750. [Google Scholar]
- 39.Keyes KM, Maslowsky J, Hamilton A, Schulenberg J. The Great Sleep Recession: Changes in Sleep Duration Among US Adolescents, 1991–2012. Pediatrics. 2015. March;135(3):460–468. [DOI] [PMC free article] [PubMed] [Google Scholar]
