Abstract
Background
The use of actigraphy in pediatric sleep research has increased over the past decade, yet few guidelines exist to help investigators with scoring and interpretation. The primary aim of this study was to compare two commonly reported non-automated rules for scoring sleep-onset and sleep-offset.
Methods
Forty children (8–12 years) wore an actigraph for one week and completed a daily sleep diary. Sleep-onset and sleep-offset were scored using the “15 minute rule” (onset: 1st of ≥15 consecutive minutes of sleep after reported bedtime; offset: last minute of ≥15 consecutive minutes of sleep prior to reported wake time) and the “3/5 minute rule” (onset: 1st of ≥3 consecutive minutes of sleep after reported bedtime; offset: last minute of ≥5 consecutive minutes of sleep prior to reported wake time). A blinded “no diary” rule was also examined (using unaided judgment to identify sleep-onset and sleep-offset).
Results
Statistical differences were found between scoring rules for sleep-offset [F (2,74) = 7.68, p = .001], sleep period [F (2,74) = 5.05, p = .009], wake after sleep-onset [F (2,74) = 7.68, p = .001], sleep minutes [F (2,74) = 3.62, p = .03], and sleep efficiency [F (2,74) = 6.50, p = .003]; however, these differences were not clinically meaningful.
Conclusions
While the findings from this study suggest that data can be compared across studies that use different scoring rules, standard scoring rules are needed to ensure that reported results are valid and meaningful.
Keywords: sleep, children, actigraphy, scoring
Over the past 20 years, there has been a rapid growth in the use of actigraphy to measure sleep patterns among children (1). An actigraph is a small, wrist-watch sized activity monitor that provides an estimate of sleep-wake patterns based on data collected by an internal accelerometer. This information is translated into epochs (usually 30 seconds or 1 minute) of activity. Several different brands of actigraphs are commercially available. Each device uses different hardware, software, and scoring algorithms to identify sleep or wake for each epoch of activity. There are many benefits to using actigraphy (e.g., ability to collect multiple nights of sleep patterns information in natural environment, less expensive than polysomnography), yet there are also several challenges for researchers.
One such challenge is a lack of standard procedures for scoring actigraphy (2;3). Manuals that accompany actigraphic devices provide some guidelines about how to manually set scoring intervals, yet Acebo and LeBourgeios (2) recommend researchers new to actigraphy refer to published scoring rules reported in methods sections to determine interval start and end times. No matter how one chooses to score actigraphy, these authors state that it is extremely important to have a rule-driven scoring procedure that includes consensus from the research team for ambiguous nights. Without standard scoring rules it can be challenging for new researchers to comply with this good advice.
In the pediatric sleep literature many studies either do not utilize scoring rules (relying solely on software scoring) or do not provide these scoring rules to readers. For studies that do provide scoring rules, there is often no justification provided for how they were chosen. Further, the rules vary greatly for the determination of sleep-onset (e.g., 5 consecutive minutes of scored sleep, 15 consecutive minutes, 20 consecutive minutes, etc.). In one study of adults that compared different scoring criteria for sleep-onset (4, 5, 6, 10 and 15 minutes of immobility) significant differences were found for sleep latency, total sleep time, and wake after sleep-onset depending on which scoring criteria was applied (4).
Another challenge often faced by researchers is what to do with actigraphy data when subjects fail to complete the sleep diary on one or more nights. One study found that at least five nights of actigraphy recordings are needed to provide reliable measures of sleep in children and adolescents (5). However, the same study found that non-adherence with completing the sleep diary may reduce the number of scorable nights. Thus the authors recommend that seven nights of recordings be completed. However, participant data may be rendered entirely unusable if diary non-adherence results in less than 5 days of scorable data. This can be especially problematic for studies with small samples.
To address some of the methodological challenges of pediatric actigraphy scoring, the primary study aim of this study was to compare actigraphic variables when two commonly reported non-automated scoring rules for sleep-onset and sleep-offset are applied. In addition, to examine both the question of diary non-adherence, as well as the result when no scoring rule is applied, the secondary aim was to compare these defined scoring rules to a no-diary scoring condition.
Methods
Participants and Procedure
Community youth ages 8–12 years were recruited to participate in a larger study of sleep patterns among school-aged children. Two home visits were completed. At the first visit, children were provided with detailed instructions on the use and care of the actigraph, as well as how to complete their daily sleep diary. The child’s primary caregiver was also instructed to keep a separate daily sleep diary to be used for reliability. One week later, the actigraph and sleep diaries were collected. This study was approved by an institutional review board, and written consent/assent was obtained.
Forty children were enrolled in the study. One child did not wear the actigraph and one child did not complete the sleep diary, thus the final sample was 38 children (45% boys) ages 8–12 years (mean=9.92, SD±1.5). Participants were Caucasian (n=29), African-American (n=7), and Hispanic (n=1).
Measures
Actigraphy
Participants wore an actigraph (Sleep Watch, Ambulatory Monitoring Inc., Ardsley, NY) on their non-dominant wrist during the 7 days of the study, except during activities involving water (e.g., showering, swimming) or when the watch might be damaged or lost (e.g., contact sports). The actigraphs were downloaded and translated into 1-minute sleep-wake epochs using a validated scoring algorithm (Sadeh algorithm, Action-W software).
Sleep Diary
Participants and parents were asked to complete a daily sleep diary. Prior to going to bed, participants and parents reported on removal of the actigraph, and any periods of low activity when the child was awake. In the morning, participants and parents reported on what time the child got into bed the previous night, attempted to fall asleep, and woke in the morning.
To ensure children were reliable reporters, parent and child sleep diary data were compared. Reported bedtimes and wake times were significantly correlated (r = .95, p< .001 and r = .91, p< .001 respectively). Paired t-tests found almost identical reported bedtimes (child = 22:11, parent = 22:10, t(31) = .25, n.s.) and wake times (child = 7:30, parent = 7:33, t(37) = −.66, n.s.). Thus child-completed sleep diary data were used for actigraphy scoring.
Actigraphy Scoring
Once actigraphy files were scored for sleep and wake epochs using the validated algorithm, the “sleep period” (minutes from sleep-onset to sleep-offset) was determined by two trained research assistants (inter-rater reliability >.95) using the scoring rules outlined below. The “sleep period” for this study was defined as sleep-onset to sleep-offset. For all scoring conditions, the following measures of sleep were assessed: sleep-onset, sleep-offset, sleep period, sleep duration (number of minutes scored as sleep in the sleep period), wake after sleep-onset (WASO, number of minutes scored as wake in the sleep period), and sleep efficiency (SE, sleep duration divided by sleep period, expressed as a percent). For both scoring rules, data were scored as “unusable” for days during which there was no corresponding sleep diary data or the child did not wear the actigraph. No child reported illness during the study period.
15 Minute Rule (15MR) (6;7)
To determine sleep-onset in the 15MR condition, the starting point was 30 minutes prior to the “attempted to fall asleep” time identified on the sleep diary, with sleep-onset manually defined as the first minute of 15 consecutive minutes of sleep. To determine sleep-offset, the starting point was set 30 minutes after the diary-reported awakening time. Working backwards in time from that point, sleep-offset was marked as the last minute of 15 consecutive minutes sleep. For all coding, 1 minute of wake within the identified 15-minute period was acceptable if it was preceded and followed by 5 minutes of consecutive sleep.
3/5 Minute Rule (3/5MR) (8–10)
To determine sleep-onset in the 3/5 Minute Rule condition, the starting point was made 15 minutes prior to the “attempted to fall asleep” time identified in the sleep diary. Moving forward from that starting time, sleep-onset was defined as the first minute of 3 consecutive minutes of sleep. To determine sleep-offset, the starting point was set 15 minutes after the diary-reported awakening time. Working backwards in time from that point, sleep-offset was marked as the last minute of 5 consecutive minutes sleep.
No Diary Condition (ND)
In this condition, the sleep diary was not consulted for scoring purposes and event markers were removed from the actograms prior to scoring. To reduce bias, actigraphy data were scored by an experienced research assistant who had not been involved with the scoring of the other two conditions. Actigraphy data were interpreted for all study days, unless the watch was removed (based on the off-wrist detection feature of the actigraph).
Data Analyses
Repeated measures analysis of variance (ANOVA) with Tukey honestly significant difference (HSD) post hoc analyses were conducted to compare the three scoring rules (15MR, 3/5MR, and ND) on each of the actigraphic variables of interest (sleep-onset, sleep-offset, sleep period, sleep duration, WASO, and SE). To examine differences in night-to-night variability that may result from the different scoring rules, the standard deviations for sleep-onset and sleep-offset were also compared with repeated measures ANOVAs and Tukey HSD post hoc analyses. To examine the relation between the different scoring rules, Pearson correlation analyses were run for each scoring dyad (15MR vs. 3/5MR, 15MR vs. ND, and 3/5MR vs. ND) on each of the six actigraphy variables.
Results
Comparison of Scoring Rules
As seen in Figure 1, a significant difference was found between scoring rules for the following mean sleep variables: sleep-offset, F (2,74) = 7.68, p = .001, ηp2 = .17, sleep period, F (2,74) = 5.05, p = .009, ηp2 = .12, WASO, F (2,74) = 7.68, p = .001, ηp2 = .17, sleep minutes, F (2,74) = 3.62, p = .03, ηp2 = .09, and SE, F (2,74) = 6.50, p = .003, ηp2 = .15. No difference was found for sleep-onset, F (2,74) = 0.66, n.s., ηp2 = .08. Post-hoc analyses indicated that compared to both the 3/5MR and the ND condition, the 15MR resulted in an earlier sleep-offset time, shorter sleep period, less WASO, and a higher SE. In addition, the 3/5MR had significantly fewer sleep minutes than the ND condition.
Figure 1.
Comparison of means for 15 Minute Rule (white bars 
), 3/5 Minute Rule (grey bars 
), and No Diary condition (black bars 
). Solid brackets (
) indicate significant difference p < .01; dashed brackets (
) indicate significant difference p < .05.
Similarly, a significant difference was found for variability in sleep-offset, between the 15MR and the other two conditions, F (2,74) = 6.01, p = .004, ηp2 = .14, but not for sleep-offset variability, F (2,74) = .95, n.s., ηp2 = .03. Post-hoc analyses indicated that the 15MR had more variability in terms of sleep-offset time.
Correlations for each scoring dyad on all sleep mean variables were significant, ranging from r = .909 to r = .993 (Table 1). All correlations between scoring rule dyads for sleep-onset and sleep-offset variability were also significant, ranging from r = .866 to r = .977.
Table 1.
Correlations between each algorithm dyad
| Variable | 15MR & 3/5MR | 15MR & No Diary |
3/5MR & No Diary |
|---|---|---|---|
| Means | |||
| Sleep Onset | .993 | .956 | .961 |
| Sleep Offset | .984 | .984 | .987 |
| Sleep Period | .984 | .932 | .909 |
| Wake After Sleep Onset | .972 | .988 | .978 |
| Sleep Minutes | .992 | .953 | .947 |
| Sleep Efficiency | .975 | .991 | .983 |
| Variability (SD) | |||
| Sleep Onset | .972 | .866 | .889 |
| Sleep Offset | .958 | .964 | .977 |
Note. N = 38, all correlations p < .001
Discussion
This study is one of the first to compare commonly used actigraphy scoring rules in pediatric research. These rules were also compared to a no diary condition to examine whether researcher-identified sleep periods are valid when subjects are non-adherent with sleep diaries. Statistically significant differences were found between the rules, with the 15MR (the most conservative rule) resulting in a shorter sleep period, yet higher sleep efficiency.
Beyond statistical significance, clinical meaningfulness should also be considered, with the differences between the three groups quite small (Figure 1). For example, the average 15MR sleep-offset time was only 8 minutes earlier than the 3/5MR and 11 minutes earlier than the ND condition. Further, the 15MR sleep efficiency was less than 1% greater than the other scoring conditions. Together these results suggest that, despite statistical significance, the three scoring conditions yield similar research data. This conclusion is further supported by the near-perfect correlations between the sleep variables.
The use of a sleep diary to help identify artifacts is essential to sleep research (e.g., participant sleeping in a moving vehicle). However, these results suggest that it may be possible to include one or two nights that do not have a sleep diary, although this approach should be used sparingly to reduce potential errors. One alternative when faced with a no diary situation is to utilize the event marker feature that is part of many actigraphic devices. For example, although we did not use the event marker data for the current study, participants pressed the event marker for 77% of sleep-onset times and 71% of sleep-offset times.
This study is limited by the relatively small sample size. In addition, the children in this study were healthy and relatively good sleepers, potentially limiting variability between the scoring rules (e.g., a child who has difficulties initiating sleep may have brief motionless periods at bedtime that would be scored as sleep-onset by the 3/5MR but not the 15MR). Future research in this area should include larger samples of youth with more diverse sleep patterns. Finally, this study included only one type of actigraph (AMI Sleep Watch) and one scoring algorithm (Sadeh). Additional research is needed to examine scoring rules in pediatric research using other devices and scoring algorithms.
Established standards for the scoring and reporting of actigraphy data are needed. Until these are in place, researchers need to be conscientious when using actigraphy and clearly report scoring rules and variables. As recommended by Acebo and LeBourgeios (2), scoring rules should be established a priori, with a research team available to evaluate ambiguous nights.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Dr. Meltzer and Ms. Westin do not have any conflict of interest.
Contributor Information
Lisa J. Meltzer, Division of Pulmonary Medicine and Department of Pediatrics, Children’s Hospital of Philadelphia and University of Pennsylvania School of Medicine
Anna M. L. Westin, Department of Psychology, University of Maryland, Baltimore County
References
- 1.Sadeh A, Acebo C. The role of actigraphy in sleep medicine. Sleep Med Rev. 2002;6(2):113–124. doi: 10.1053/smrv.2001.0182. [DOI] [PubMed] [Google Scholar]
- 2.Acebo C, LeBourgeois MK. Actigraphy. Respir Care Clin N Am. 2006;12:23–30. doi: 10.1016/j.rcc.2005.11.010. [DOI] [PubMed] [Google Scholar]
- 3.Berger AM, Wielgus KK, Young-McCaughan S, Fischer P, Farr L, Lee KA. Methodological challenges when using actigraphy in research. Journal of Pain Symptom Management. 2008:1–9. doi: 10.1016/j.jpainsymman.2007.10.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Chae KY, Kripke DF, Poceta JS, Shadan F, Jamil SM, Cronin JW, et al. Evaluation of immobility time for sleep latency in actigraphy. Sleep Med. 2009 Jun;10(6):621–625. doi: 10.1016/j.sleep.2008.07.009. [DOI] [PubMed] [Google Scholar]
- 5.Acebo C, Sadeh A, Seifer R, Tzischinsky O, Wolfson AR, Hafer A, et al. Estimating sleep patterns with activity monitoring in children and adolescents: How many nights are necessary for reliable measures? Sleep. 1999 Feb 1;22(1):95–103. doi: 10.1093/sleep/22.1.95. [DOI] [PubMed] [Google Scholar]
- 6.Sadeh A, Raviv A, Gruber R. Sleep patterns and sleep disruptions in school-age children. Dev Psychol. 2000 May;36(3):291–301. doi: 10.1037//0012-1649.36.3.291. [DOI] [PubMed] [Google Scholar]
- 7.Sadeh A, Keinan G, Daon K. Effects of stress on sleep: The moderating role of coping style. Health Psychol. 2004;23(5):542–545. doi: 10.1037/0278-6133.23.5.542. [DOI] [PubMed] [Google Scholar]
- 8.Wolfson AR, Carskadon MA, Acebo C, Seifer R, Fallone G, Labyak SE, et al. Evidence for the validity of a sleep habits survey for adolescents. Sleep. 2003 Mar 15;26(2):213–216. doi: 10.1093/sleep/26.2.213. [DOI] [PubMed] [Google Scholar]
- 9.Werner H, Molinari L, Guyer C, Jenni OG. Agreement rates between actigraphy, diary, and questionnaire for children's sleep patterns. Arch Pediatr Adolesc Med. 2008 Apr;162(4):350–358. doi: 10.1001/archpedi.162.4.350. [DOI] [PubMed] [Google Scholar]
- 10.El-Sheikh M, Buckhalt JA, Keller PS, Granger DA. Children's objective and subjective sleep disruptions: links with afternoon cortisol levels. Health Psychol. 2008 Jan;27(1):26–33. doi: 10.1037/0278-6133.27.1.26. [DOI] [PubMed] [Google Scholar]