Abstract
Introduction:
Sleep is often quantified using self-report or actigraphy. Self-report is practical and less technically challenging, but prone to bias. We sought to determine whether these methods have comparable sensitivity to measure longitudinal changes in adolescent bedtimes.
Methods:
We measured one week of free-living sleep with wrist actigraphy and usual bedtime on school nights and non-school nights with self-report questionnaire in 144 students at 15 y and 17 y.
Results:
Self-reported and actigraphy-measured bedtimes were correlated with one another at 15 y and 17 y (p < .001), but reported bedtime was consistently earlier (>30 minutes, p < .001) and with wide inter-method confidence intervals (> ±106 minutes). Mean inter-method discrepancy did not differ on school nights at 15 y and 17 y but was greater at 17 y on non-school nights (p = .002). Inter-method discrepancy at 15 y was not correlated to that at 17 y. Mean change in self-reported school night bedtime from 15 y to 17 y did not differ from that by actigraphy, but self-reported bedtime changed less on non-school nights (p = .002). Two-year changes in self-reported bedtime did not correlate with changes measured by actigraphy.
Conclusions:
Although methods were correlated, consistently earlier self-reported bedtime suggests report-bias. More varied non-school night bedtimes challenge the accuracy of self-report and actigraphy, reducing sensitivity to change. On school nights, the methods did not differ in group-level sensitivity to changes in bedtime. However, lack of correlation between bedtime changes by each method suggests sensitivity to individual-level change was different. Methodological differences in sensitivity to individual- and group-level change should be considered in longitudinal studies of adolescent sleep patterns.
Keywords: accelerometer, questionnaire, sleep, sleep onset, teenagers
Adolescent sleep patterns are often measured with self-report (Lewandowski, Toliver-Sokol, & Palermo, 2011) or actigraphy (Galland et al., 2018). Although self-report is easier to administer, lower in cost, and requires less technical expertise than actigraphy, it is often affected by social expectations and recall bias (Wolfson et al., 2003). Self-report can be measured using a survey question about usual sleep habits or with nightly sleep logs (Knutson & Lauderdale, 2007). While sleep logs are shown to be more reliable than survey questions (Arora, Broglia, Pushpakumar, Lodhi, & Taheri, 2013; Werner, Molinari, Guyer, & Jenni, 2008), they are also more burdensome for participants and have lower compliance (Knutson & Lauderdale, 2007). Thus, survey questions can be more easily employed for larger, nationally representative studies. Several studies have assessed agreement between self-report by questionnaire and actigraphy in adolescents (Arora et al., 2013; Biddle, Robillard, Hermens, Hickie, & Glozier, 2015; Guedes et al., 2016; Wolfson et al., 2003), but it is unclear whether the two methods have comparable sensitivity to measure longitudinal change in sleep patterns. We measured bedtime by self-report and actigraphy at age 15 and 17 to determine if discrepancy between the methods is independent of age and if age-related changes in bedtime by questionnaire-based self-report correspond to those by actigraphy.
Methods
Participants
We invited 411 15-year-old students from six schools in Reykjavík, Iceland to participate; 315 agreed, and 270 had complete data. Two years later, 168 of the 270 agreed to repeat the measurements. Data are presented from 144 participants (89 girls) that had complete data at both time points.
The National Bioethics Committee and the Icelandic Data Protection Authority approved the study (Study number: VSNb2015020013/13.07). Written informed consent was attained from participants and guardians of underage participants. Full confidentiality was ensured, and the study was conducted in agreement with the Declaration of Helsinki.
Bedtime Measurements by Actigraphy and Self-report
Participants were instructed to wear an ActiGraph GT3X+ (Actigraph Inc., Pensacola, Florida, USA) on their non-dominant wrist for seven consecutive days during measurements at 15 y and 17 y. Data was processed with Actilife (v6.13.0, Actigraph, Pensacola, FL, USA) and Matlab (vR2016B, Mathworks, Natick, MA, USA), as detailed previously (Rognvaldsdottir et al., 2017). Sleep periods were detected with the Sadeh algorithm validated for adolescents (Sadeh, Sharkey, & Carskadon, 1994) and adjusted by two expert scorers based on visual inspection and daily participant sleep logs completed as part of the actigraphy study. Non-wear was identified as described previously (Rognvaldsdottir et al., 2017). Valid data consisted of ≥14 hours of wear on ≥3 school nights (Sunday through Thursday nights) and ≥1 non-school night (Friday and Saturday night and nights prior to holidays) (Rognvaldsdottir et al., 2017). Average actigraphy-measured bedtimes for school nights and non-school nights at each age were included in all primary analyses. The earliest actigraphy-measured bedtime for each student on school nights and non-school nights at each age were also included for comparison.
Separate from the daily sleep logs, the self-reported measure consisted of participant responses to the following survey questions administered using a tablet-based questionnaire at school prior to the actigraphy study: “What is your usual bedtime on school nights?” and “What is your usual bedtime on the weekend?” Potential answers were in half hour increments from 20:00 to 04:00.
Statistical Analysis
Associations between self-reported and actigraphy-measured bedtimes were evaluated with Pearson correlation. Bland-Altman plots were used to assess inter-method agreement (Bland & Altman, 1986). All other comparisons were made using analysis of variance with Tukey’s post-hoc comparisons with Bonferroni adjustment. Using unpaired analyses, we found few sex differences in self-reported or actigraphy-measured bedtimes at either age (data not shown) and no differences in inter-method discrepancy, contrary to prior work (Guedes et al., 2016). Therefore, data from both sexes were combined and paired analyses were used for all other assessments. The significance threshold was p = .05. Analyses were conducted using RStudio (v1.0.153, Boston, MA, USA) with R (v3.4.2, https://www.r-project.org/) and GraphPad Prism (v7. La Jolla, CA).
Results and Discussion
Comparing Bedtimes Measured by Actigraphy and Self-Report at Age 15
Participant characteristics, average bedtimes by actigraphy and self-report, and discrepancies between methods are presented in Table 1. Self-reported and average actigraphy-measured bedtimes were correlated on school nights (r = 0.47, p < .001) and nonschool nights (r = 0.51, p < .001). Bedtimes on non-school nights were later than those on school nights according to both actigraphy (as previously reported [Rognvaldsdottir et al., 2017]) and self-report (both p < .001; Table 1). Taken together, these results confirm previous observations that self-report and actigraphy measure the same underlying parameter in adolescents (Wolfson et al., 2003). However, self-reported bedtimes were >30 minutes earlier than average actigraphy-measured bedtimes (p < .001), with greater discrepancy on school nights (Table 1). The trend of reporting earlier bedtimes than those measured by actigraphy supports previous findings that self-report can be biased by social expectations and recall limitations (Wolfson et al., 2003), but it is worth noting that mean difference between self-report and actigraphy at 15 y in this study was 3.7 times greater than previous adolescent measurements on school nights (Wolfson et al., 2003). Self-reported bedtimes compare more favorably to the earliest actigraphy-measured bedtime for each participant, as we found no differences between these two measures on school nights and slightly later self-reports on non-school nights at age 15 (Table 1). These results suggest students tend to report aspirational or target bedtimes, perhaps due to expectations of achieving popular sleep guidelines, as previously suggested (Wolfson et al., 2003). Night-to-night bedtime variability, i.e. the standard deviation of actigraphy-measured bedtimes over all nights, was closer to that measured on vacation days (72.9 ± 40.0 min) than school days (43.7 ± 25.8 min) in a group of similarly aged Australian students (Bei et al., 2014) and more than double that measured in older Japanese women (32.1 ± 18.2 min) (Kim, Sasai, Kojima, & Kim, 2015), indicating high variability amongst our participants.
Table 1.
15 y | 17 y | Change (17 y–15 y) | p-value (15 y vs. 17 y) | |
---|---|---|---|---|
Characteristics | ||||
Age (y) | 15.9 ± 0.3 | 17.7 ± 0.3 | 1.8 ± 0.1 | <.001 |
Height (cm) | 171.7 ± 8.2 | 173.4 ± 9.0 | 1.7 ± 2.4 | .09 |
Weight (kg) | 64.4 ± 11.3 | 68.5 ± 12.8 | 4.0 ± 5.2 | .005 |
Body mass index (kg/m2) | 21.8 ± 3.3 | 22.7 ± 3.8 | 0.9 ± 1.6 | .03 |
School Night Bedtimes | ||||
Self-report (clock time ± min) | 23:31 ± 57.1 | 23:57 ± 59.0 | 26.2 ± 60.7 | <.001 |
Average by Actigraphy (clock time ± min) | 00:19 ± 46.1** | 00:55 ± 61.0** | 35.2 ± 59.2 | <.001 |
Earliest by Actigraphy (clock time ± min) | 23:27 ± 63.3 | 23:56 ± 62.3 | 28.8 ± 81.1 | <.001 |
Self-report – Average by Actigraphy (min) | −48.6 ± 54.1 | −57.5 ± 67.7 | −8.9 ± 80.6 | .2 |
Self-report – Earliest by Actigraphy (min) | 4.0 ± 76.7 | 1.5 ± 72.6 | −2.5 ± 106.0 | .8 |
Non-school Night Bedtimes | ||||
Self-report (clock time ± min) | 01:04 ± 68.2# | 01:33 ± 75.5# | 28.1 ± 79.9 | .001 |
Average by Actigraphy (clock time ± min) | 01:36 ± 69.9**,# | 02:26 ± 85.5**,# | 50.4 ± 92.9* | <.001 |
Earliest by Actigraphy (clock time ± min) | 00:51 ± 64.0*,# | 01:42 ± 89.8# | 51.3 ± 94.4* | <.001 |
Self-report – Average by Actigraphy (min) | −31.4 ± 68.1# | −53.7 ± 94.4 | −22.2 ± 113.2 | .02 |
Self-report – Earliest by Actigraphy (min) | 13.3 ± 69.1 | −9.9 ± 100.4 | −23.2 ± 114.0 | .02 |
Night-to-night Bedtime Variability | ||||
Actigraphy (min) | 65.8 ± 36.2 | 73.0 ± 30.9 | 7.2 ± 43.2 | .048 |
p < .05
p < .01 vs. self-report during the same year.
p < .01 vs. school nights during the same year.
Individual differences between self-report and average actigraphy bedtimes ranged from −277 to +212 minutes. Bland-Altman plots showed wide confidence limits for the inter-method discrepancy (report - average of actigraphy) on school nights (±106 minutes. Figure 1a) and non-school nights (±133 minutes. Figure 1b) which were >1.8 times those observed for parent-reported bedtimes versus actigraphy in children (Werner et al., 2008), suggesting limited agreement between the methods in adolescents. Inter-method discrepancy was also directly correlated to the average of self-reported and actigraphy-measured school night bedtimes (r = 0.24; p = .004) and those with earliest average bedtimes showed the greatest inter-method discrepancy (Figure 1a). Interestingly, plotting inter-method discrepancy against actigraphy-measured bedtimes showed the opposite trend- a significant inverse correlation on school nights (r = −0.36; p < .001) and non-school nights (r = −0.51; p < .001), indicating greater inter-method discrepancy for adolescents with later actigraphy-measured bedtimes. Substituting earliest actigraphy-measured bedtimes for average actigraphy-measured bedtime in the Bland-Altman analyses reduced mean inter-method discrepancy and eliminated the correlation between inter-method discrepancy and average of self-report and actigraphy on school nights, but inter-method discrepancy confidence limits were similarly broad on school nights (±150 minutes) and non-school nights (±135 minutes).
Comparing Bedtime Measurements at Age 15 and 17
At age 17, on both school nights and non-school nights, students continued reporting bedtimes that were earlier than the average (both p < .001; Table 1) and closer to the earliest (both p > .05; Table 1) measured by actigraphy. The pattern of going to bed later on non-school nights versus school nights persisted according to both measures (both p < .001; Table 1) and actigraphy-measured bedtimes became more varied (p = .048; Table 1). Self-reported and average actigraphy-measured bedtimes were correlated on school nights (r = 0.36, p < .001) and non-school nights (r = 0.32, p < .001), and Bland-Altman plots showed wider confidence intervals for inter-method discrepancy on school nights and non-school nights at 17 y compared to 15 y (Figures 1c and 1d, respectively). These data suggest some consistent patterns from 15 y to 17 y including a trend toward reporting an early, “target” bedtime compared to the average measured with actigraphy, a later bedtime pattern on non-school nights compared to school nights, high night-to-night variability in actigraphy-measured bedtimes, and a correlation between the two methods.
On average, inter-method discrepancy did not differ on school nights at 15 y and 17 y but was greater at 17 y on nonschool nights (p = .02; Table 1)—findings that were consistent whether using average or earliest actigraphy-measured bedtime. Greater difficulty reporting less regular non-school night bedtime schedule could explain this discrepancy. Supporting this notion, students with greater night-to-night bedtime variability also had greater inter-method discrepancy on non-school nights at 15 y and 17 y (Figures 2b and 2d). However, mean inter-method discrepancy (using average actigraphy-measured bedtimes) was lower on non-school nights than school nights at 15 y (p = .035), and no different between school nights and non-school nights at 17 y. Further, the relationship between inter-method discrepancy and night-to-night bedtime variability on school nights was opposite that of non-school nights at 15 y and absent at 17 y (Figures 2a and 2c, respectively). Alternatively, 1–2 non-school nights of actigraphy may be insufficient to determine a true mean bedtime (Acebo et al., 1999; Wolfson et al., 2003). Additionally, inter-method discrepancies at 17 y were not correlated to those at 15 y and confidence intervals were broader at 17 y, suggesting that the discrepancy is inconsistent with age on an individual level.
According to both self-report and actigraphy, students went to bed later on school nights and non-school nights at 17 y compared to 15 y (all p < .001, Table 1), confirming that adolescents shift toward later bedtimes with age (Carskadon, 1990). Bedtimes at 17 y were correlated to those at 15 y, using either average actigraphy-measured or self-report, on school nights (r = 0.42 and r = 0.45, respectively) and non-school nights (r = 0.30 and r = 0.38, respectively; all p < .001), suggesting individuals had a similar preferred bedtime pattern at both ages.
On school nights, mean change in bedtimes from 15 y to 17 y measured using self-report and actigraphy were not different (Figure 3a). On non-school nights, however, self-reported bedtimes changed less than actigraphy-measured bedtimes (p = .02, Figure 3a, Table 1), further indicating that more varied bedtimes on non-school nights may require more nights of actigraphy or more specific questions to improve accuracy. Changes in actigraphy-measured bedtimes from 15 y to 17 y were not correlated with changes in self-reported bedtimes on either school nights or non-school nights (Figures 3b and 3c). Findings were consistent whether using two-year changes in average or earliest actigraphy-measured bedtimes. Thus, while the methods did not differ in measurement of group-level two-year change in school night bedtime, sensitivity to detect individual-level change was different according to the lack of correlation between methods. These findings suggest that bias affecting self-report is not longitudinally consistent on an individual level.
Limitations
Students reported bedtime but not sleep duration or risetime-which could have been used to calculate reported time in bed. Sleep duration or time in bed may be more relevant to adolescent health than bedtime. However, actigraphy-based measures of bedtime and sleep duration were inversely correlated amongst our participants (p < .001). We did not have a measure of self-reported sleep quality or awakenings, but such measures were previously shown to have poor agreement with actigraphy amongst adolescents (Short, Gradisar, Lack, Wright, & Carskadon, 2012). Though a self-report question with 30-minute increments seemed most practical for this age group, <30-minute increments may improve agreement with actigraphy. Similarly, although self-report was restricted from 20:00 to 04:00, none of those with actigraphy-measured bedtimes ≥04:00 reported a 04:00 bedtime and no actigraphy-measured bedtimes were <22:00. The methods may have had better agreement if students reported bedtime in the past week rather than “usual” bedtime, which may be more influenced by memories or social expectations (Werner et al., 2008). Sample size was small, although it is representative of the Icelandic population at this age (4254 born in 1999) (Statistics Iceland, 2015). Additionally, participants with and without complete follow-up data did not differ in sex distribution or bedtimes at 15 y (not shown). We found no correlation between body mass index and either measure of bedtime or inter-method discrepancy, but less than 20% of the participants were overweight or obese. Similarly, 70% of participants reported having at least one parent with a college degree—a proxy of socioeconomic status (Marco, Wolfson, Sparling, & Azuaje, 2011) but they did not differ from the other students in either measure of bedtime or inter-method discrepancy. The northern latitude and homogeneous racial and ethnic composition of Iceland may also limit the generalizability of the findings.
Conclusions
This is the first study to compare the sensitivity of self-report and actigraphy to measure longitudinal changes in adolescent bedtime. While the methods were correlated at each timepoint, the participants’ self-reported bedtimes were consistently earlier, with >85% on school days and >71% on non-school days at both ages reporting earlier bedtimes than the average bedtime measured by actigraphy, suggesting the presence of report-bias. More varied non-school night bedtimes may challenge accuracies of both self-report and actigraphy, reducing sensitivity to detect changes. On school nights, group-level inter-method difference at 15 y and 17 y did not differ, nor did mean two-year change in bedtime detected by both methods. However, according to a lack of correlation between two-year changes in bedtime by self-report and actigraphy, sensitivity to detect individual-level change was different between the methods. These results suggest bias affecting self-report is not longitudinally consistent in this adolescent sample. Caution should be used when interpreting longitudinal self-reported sleep measures in populations with similar highly varied sleep schedules. Thus, along with issues of practicality, expense, and compliance, the impact of differences in sensitivity to individual- and group-level change on study outcomes should be considered when choosing a methodology to study longitudinal sleep patterns in adolescents.
Acknowledgments
The authors would like to thank master’s students and researchers involved in data collection and participants and staff of participating schools.
Contributor Information
Robert J. Brychta, Diabetes, Endocrinology, and Obesity Branch, National Institute of Diabetes and Digestive and Kidney Diseases, Bethesda, MD, USA
Vaka Rögnvaldsdóttir, Center of Sport and Health Sciences, University of Iceland, Reykjavik, Iceland.
Sigriður L. Guðmundsdóttir, Center of Sport and Health Sciences, University of Iceland, Reykjavik, Iceland
Rúna Stefánsdóttir, Center of Sport and Health Sciences, University of Iceland, Reykjavik, Iceland.
Soffia M. Hrafnkelsdóttir, Center of Sport and Health Sciences, University of Iceland, Reykjavik, Iceland
Sunna Gestsdóttir, Center of Sport and Health Sciences, University of Iceland, Reykjavik, Iceland.
Sigurbjörn A. Arngrímsson, Center of Sport and Health Sciences, University of Iceland, Reykjavik, Iceland
Kong Y. Chen, Diabetes, Endocrinology, and Obesity Branch, National Institute of Diabetes and Digestive and Kidney Diseases, Bethesda, MD, USA
Erlingur Jóhannsson, Center of Sport and Health Sciences, University of Iceland, Reykjavik, Iceland; Department of Sport and Physical Activity, Western Norway University of Applied Sciences, Bergen, Norway.
References
- Acebo C, Sadeh A, Seifer R, Tzischinsky O, Wolfson AR, Hafer A, & Carskadon MA (1999). Estimating sleep patterns with activity monitoring in children and adolescents: How many nights are necessary for reliable measures? Sleep, 22(1), 95–103. doi: 10.1093/sleep/22.1.95 [DOI] [PubMed] [Google Scholar]
- Arora T, Broglia E, Pushpakumar D, Lodhi T, & Taheri S (2013). An investigation into the strength of the association and agreement levels between subjective and objective sleep duration in adolescents. PLoS One, 8(8), e72406. doi: 10.1371/journal.pone.0072406 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bei B, Allen NB, Nicholas CL, Dudgeon P, Murray G, & Trinder J (2014). Actigraphy-assessed sleep during school and vacation periods: A naturalistic study of restricted and extended sleep opportunities in adolescents. Journal of Sleep Research, 23(1), 107–117. doi: 10.1111/jsr.12080 [DOI] [PubMed] [Google Scholar]
- Biddle DJ, Robillard R, Hermens DF, Hickie IB, & Glozier N (2015). Accuracy of self-reported sleep parameters compared with actigraphy in young people with mental ill-health. Sleep Health, 1 (3), 214–220. doi: 10.1016/j.sleh.2015.07.006 [DOI] [PubMed] [Google Scholar]
- Bland JM, & Altman DG (1986). Statistical methods for assessing agreement between two methods of clinical measurement. Lancet, 7(8476), 307–310. doi: 10.1016/S0140-6736(86)90837-8 [DOI] [PubMed] [Google Scholar]
- Carskadon MA (1990). Patterns of sleep and sleepiness in adolescents. Pediatrician, 77(1), 5–12. [PubMed] [Google Scholar]
- Galland BC, Short MA, Terrill P, Rigney G, Haszard JJ, Coussens S, … Biggs SN (2018). Establishing normal values for pediatric nighttime sleep measured by actigraphy: A systematic review and meta-analysis. Sleep, 47(4):1–16. doi: 10.1093/sleep/zsy017 [DOI] [PubMed] [Google Scholar]
- Guedes LG, Abreu Gde A, Rodrigues DF, Teixeira LR, Luiz RR, & Bloch KV (2016). Comparison between self-reported sleep duration and actigraphy among adolescents: Gender differences. Revista Brasileira de Epidemiologia, 19(2), 339–347. doi: 10.1590/1980-5497201600020011 [DOI] [PubMed] [Google Scholar]
- Kim M, Sasai H, Kojima N, & Kim H (2015). Objectively measured night-to-night sleep variations are associated with body composition in very elderly women. Journal of Sleep Research, 24(6), 639–647. doi: 10.1111/jsr.12326 [DOI] [PubMed] [Google Scholar]
- Knutson KL, & Lauderdale DS (2007). Sleep duration and overweight in adolescents: Self-reported sleep hours versus time diaries. Pediatrics, 119(5), e1056–e1062. doi: 10.1542/peds.2006-2597 [DOI] [PubMed] [Google Scholar]
- Lewandowski AS, Toliver-Sokol M, & Palermo TM (2011). Evidence-based review of subjective pediatric sleep measures. Journal of Pediatric Psychology, 36(7), 780–793. doi: 10.1093/jpepsy/jsq119 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marco CA, Wolfson AR, Sparling M, & Azuaje A (2011). Family socioeconomic status and sleep patterns of young adolescents. Behavioral Sleep Medicine, 70(1), 70–80. doi: 10.1080/15402002.2012.636298 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rognvaldsdottir V, Gudmundsdottir SL, Brychta RJ, Hrafnkelsdottir SM, Gestsdottir S, Amgrimsson SA, … Johannsson E. (2017). Sleep deficiency on school days in Icelandic youth, as assessed by wrist accelerometry. Sleep Medicine, 33, 103–108. doi: 10.1016/j.sleep.2016.12.028 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sadeh A, Sharkey KM, & Carskadon MA (1994). Activity-based sleep-wake identification: An empirical test of methodological issues. Sleep, 77(3), 201–207. doi: 10.1093/sleep/17.3.201 [DOI] [PubMed] [Google Scholar]
- Short MA, Gradisar M, Lack LC, Wright H, & Carskadon MA (2012). The discrepancy between actigraphic and sleep diary measures of sleep in adolescents. Sleep Medicine, 73(4), 378–384. doi: 10.1016/j.sleep.2011.11.005 [DOI] [PubMed] [Google Scholar]
- Statistics Iceland. (2015). Statistical yearbook of Iceland. Reykjavik, Iceland: Statistics Iceland. [Google Scholar]
- Werner H, Molinari L, Guyer C, & Jenni OG (2008). Agreement rates between actigraphy, diary, and questionnaire for children’s sleep patterns. Archives of Pediatrics and Adolescent Medicine, 162(4), 350–358. doi: 10.1001/archpedi.162.4.350 [DOI] [PubMed] [Google Scholar]
- Wolfson AR, Carskadon MA, Acebo C, Seifer R, Fallone G, Labyak SE, & Martin JL (2003). Evidence for the validity of a sleep habits survey for adolescents. Sleep, 26(2), 213–216. doi: 10.1093/sleep/26.2.213 [DOI] [PubMed] [Google Scholar]