Sleep Assessments In Healthy School-Aged Children Using Actigraphy: Concordance with Polysomnography

SUMMARY

Actigraphic recordings (ACT) are widely used in school children as a less intrusive and more extended approach to evaluation of sleep problems. However, critical assessment of the validity and reliability of ACT against overnight polysomnography (NPSG) are unavailable. Thus, we explored the degree of concordance between NPSG and ACT in school-aged children to delineate potential ACT boundaries when interpreting pediatric sleep. Non-dominant wrist ACT was simultaneously recorded with NPSG in 149 healthy school-aged children (4.1 to 8.8 years old, 41.7% boys and 80.4% Caucasian) recruited from the community. Analyses were limited to the Actiware (MiniMitter-64) calculated parameters originating from 1-min epoch sampling and medium sensitivity threshold value of 40; i.e., Sleep Period Time (SPT), Total Sleep Time (TST) and Wake After Sleep Onset (WASO). SPT was not significantly different between ACT and NPSG. However, ACT significantly underestimated TST by 32.2±33.4 minutes, and overestimated WASO by 26.3±34.4 minutes. The decreased precision of ACT was also evident from moderate to small concordance correlation coefficients (0.47 for TST and 0.09 for WASO). ACT in school-aged children provides reliable assessment of sleep quantity, but is relatively inaccurate during determination of sleep quality. Thus, caution is advocated in drawing definitive conclusions from ACT during evaluation of the sleep disturbed child.

INTRODUCTION

The debate between the advocates of actigraphic (ACT) recordings and those favoring the `gold standard' of overnight polysomnographic assessments (NPSG) has been ongoing for the last 40 years (Sadeh et al., 1995, Werner et al., 2008, Tryon, 2004, Littner et al., 2003). Clear consensus exists as to ACT being a proxy to sleep measurements, i.e., it does not involve brain wave detection such as NPSG, nor does it involve subjective recognition of sleep such as sleep-logs, diaries or questionnaires. As such, ACT is primarily used because it provides objective functionality during monitoring of sleep duration and circadian patterns in adults, such that objective inferences can be made as far as interventions and their outcomes (Ancoli-Israel et al., 2003, Sadeh et al., 1995, Acebo and LeBourgeois, 2006). Considering that ACT provides an unobtrusive, cost effective, and practical tool for prolonged ambulatory monitoring and for objective measurements of both diurnal and nocturnal behaviors, one would expect that it would be widely used in pediatric populations. However, its applicability to children remains somewhat questionable as exemplified by several recent studies (Sung et al., 2009, Insana et al., 2010, Owens et al., 2009), as well as by an extensive literature search.

As a corollary to these studies, ACT has gained substantial popularity as evidenced by the increasing number of publications on ACT since 2004. The basic principle of ACT is relatively simple, and as such, may predispose it to improper use and artifacts (Sadeh et al., 1995, Gale et al., 2005, Tryon, 2004, Acebo and LeBourgeois, 2006). Inherent to its design and purpose, two interrelated aspects need to be kept in mind when using ACT: 1) presence of movement during ACT indicates wake state, and 2) a linear relationship between wake and sleep is anticipated. It becomes immediately apparent that the absence of movement does not mandatorily equal sleep, thereby leading to decreased psychometric quality of wake assessments by ACT (Acebo and LeBourgeois, 2006, Paquet et al., 2007, Sadeh et al., 1995). In addition, movements of the body and location of the ACT device may vary (Paavonen et al., 2002), and therefore, different uses of ACT may be at variance in how they detect and record `movement' (or wake), and may further use dissimilar methodologies for computing activity levels. Unfortunately, algorithms for computation of activity are not always provided in detail, possibly due to the lack of consensus (Webster et al., 1982, Morgenthaler et al., 2007, Sadeh, 1989, Cole et al., 1992, Acebo and LeBourgeois, 2006). Nevertheless, efforts to improve sleep-wake detection using machine-learning algorithms in the context of ACT in children appear promising (Tilmanne et al., 2009).

The pressing need for non-obtrusive and clinically practical tools to monitor sleep-wake patterns in the context of evaluating children with specific conditions has yielded relatively promising results (Owens et al., 2009, Goldman et al., 2009, Kothare and Kaleyias, 2008, Ohinata et al., 2008, Aronen et al., 2002, Marshall et al., 2008). However, caution should be applied when generalizing the applicability of such reports to other clinical situations, especially when considering some of the potential methodological shortcomings, such as small sample sizes, lack of established validity of the device, limited report of ACT recording and analysis details (Acebo and LeBourgeois, 2006, Morgenthaler et al., 2007) and a decreased psychometric quality of ACT in cases with disrupted or disordered sleep (Gale et al., 2005, Sadeh and Acebo, 2002, Littner et al., 2003). In addition, consideration of normative reference values for movement during sleep should also need to be incorporated into the considerations regarding ACT applicability(Aronen et al., 2001).

Given that both sleep onset and offset are gradual processes, it should be immediately obvious that ACT is incapable of capturing gradualness by design,(Acebo and LeBourgeois, 2006) and therefore, both sleep onset and sleep offset could be underestimated by ACT (Tryon, 2004, Paquet et al., 2007). The immediate consequence of such limitations of ACT would be overestimation of sleep duration and sleep efficiency, particularly in the context of errors introduced by the calculation of time in bed which is often derived from logs, rather than recorded directly on the ACT device.

The aim of the present study was to examine the robustness of monitoring sleep quantity and sleep quality using ACT in a healthy population of school-aged children when compared to concomitant NPSG recordings.

MATERIALS AND METHODS

Subjects

This study was approved by the University of Louisville Human Research Committee, and informed consent was obtained from the legal caregiver of each participant.

One hundred and forty nine healthy school-aged children (mean age = 6.5 years, SD = 1.2 years, range = 4.1 to 8.8 years old, 41.7% boys, 80.4% White Non-Hispanics, 13% African American and 6.5% other ethnicities) were recruited from the community via the Jefferson County Public School system. Children had a Mean Body Mass Index of 17.7 (SD = 5.9) kg/m², with 9.9% fulfilling the criteria for obesity (>95^th percentile).

Children were excluded if they had any chronic medical conditions, genetic, craniofacial syndromes or neurobehavioral disorders. All children were reported by the caregiver to be non-snorers. Each child wore the ACT device on their non-dominant wrist during the NPSG as part of a larger ongoing study. Inclusion criteria for the present study required a normal NPSG according to previously published normative data (Montgomery-Downs et al., 2006). Additionally, data was not included when a first night effect was suspected or when the total sleep duration was less than 4 hours (Scholle et al., 2003, Verhulst et al., 2006).

Instruments

Overnight Polysomnography

All children were accompanied by one of their caregivers who slept in the same room. Overnight polysomnography (NPSG) was performed in a quiet, darkened room with an ambient temperature of 24 °C. The following sleep parameters were measured: chest and abdominal wall movement by respiratory impedance or inductance plethysmography, heart rate by electrocardiogram (ECG), air flow was triply monitored with a sidestream end-tidal capnograph which also provided breath-by-breath assessment of end-tidal carbon dioxide levels (PETCO2; BCI SC-300, Menomonee Falls, WI), a nasal pressure cannula, and an oronasal thermistor. Arterial oxygen saturation (SpO2) was assessed by pulse oximetry (Nellcor N 100; Nellcor Inc., Hayward, CA), with simultaneous recording of the pulse waveform. The bilateral electro-oculogram (EOG), eight channels of electroencephalogram (EEG), chin and anterior tibial electromyograms (EMG), and analog output from a body position sensor (Braebon Medical Corporation, Ogsdenburg, NY) were also monitored. All measures were digitized using commercially available polysomnography systems (Rembrandt, MedCare Diagnostics, Amsterdam; Stellate Systems, Montreal, Canada). Tracheal sound was monitored with a microphone sensor (Sleepmate, Midlothian, VA), and a digital time-synchronized video recording was performed.

Sleep architecture was evaluated using standard techniques (Rechtschaffen and Kales, 1968). Sleep parameters were scored in the following manner: percentage of total sleep time (%TST) being the proportion of time spent in each sleep stage; apneic events being an obstruction in absence of airflow with continued chest wall and abdominal movement for a duration of at least two breaths, hypopneas being a decrease in oronasal flow of ≥50% with a corresponding decrease in SpO2 of ≥4% and/or arousal, apnea/hypopnea index (AHI) being the number of apneas and hypopneas per hour of TST, apnea index (AI) being the number of apneas per hour of TST.(Montgomery-Downs et al., 2006) Arousals were defined as recommended by the American Sleep Disorders Association Task Force Report (1992) and include respiratory-related (occurring immediately following an apnea, hypopnea or snore), technician-induced and spontaneous arousals and were expressed as the total number of arousals per hour of sleep time (arousal index).

Actigraphy

The Actiwatch (MiniMitter Actiwatch®-64 Co. Inc. 1998–2003, version 3.4) is 28×27×10mm and weighs 17.5g. The actiware software recording was set at 1 minute epoch. For this ACT brand, epoch registration of activity counts are determined by comparison; i.e., counts for the epoch in question and those immediately surrounding that epoch are weighted with a threshold sensitivity value (TSV or activity count) that was originally set at 40 (ACT40)(default, being medium sensitivity) [score = E₋₂*(1/25) + E₋₁*(1/5) + E₀ + E₊₁*(1/5) + E₊₂*(1/25), with E_n being activity counts for the epoch, with E₀ the scored epoch]. Two other sensitivity levels were additionally analyzed: low sensitivity being a threshold activity value of 80 (ACT80) and high sensitivity being a threshold activity level of 20 (ACT20). Subsequently, if the number of activity counts is equal or below the TSV that epoch is scored as `sleep', whereas if exceeding TSV it is scored as `wake'.

The ACT-sleep interval was manually marked for each record based on lab sleep study policy: between 20:00 and 7:30. ACT parameters of interest were thus:

• -
  Sleep period Time (SPT) or Assumed Sleep calculated by the Actiware as the difference in time between the Sleep End and the Sleep Start times;

  • ◯
    Sleep Start is determined automatically as the first 10-minute period in which no more than one epoch is scored as mobile
  • ◯
    Sleep End is identical to the previous but represents the last 10-minute period,

as a result, this is regarded similar to SPT of NPSG, and labeled respectively ACT_SPT and NPSG_SPT.

• -Total Sleep Time (TST) or Actual Sleep Time by ACT, is by definition comparable to TST of NPSG, representing the amount of time between Sleep Start and End scored as `sleep'. The algorithm sums the number of epochs that do not exceed the TSV and that value is multiplied by the epoch length in minutes; respectively, ACT_TST and NPSG_TST.
• -Wake After Sleep Onset (WASO) or Actual Wake Time by ACT, similarly in NPSG terminology the WASO, is calculated as the amount of time between Sleep Start and Sleep End scored as `wake'; i.e., ACT_WASO and NPSG_WASO

The latter two can be expressed as an index, and were calculated for ACT and NPSG: Actual Sleep Time Percentage, being the amount of assumed sleep time that is actually sleep, which is (TST/SPT)*100, denoted as Total Sleep Time Percentage (TST%), and Actual Wake Time Percentage, being the amount of assumed sleep that is actually wake or Wake After Sleep Onset Percentage (WASO %) [(WASO/SPT)*100]. Another parameter analyzed was Sleep Onset Latency (SOL), which is the period between Bedtime and Sleep Start expressed in minutes.

Statistical Analysis

All descriptive and (non)parametric group analyses (i.e., student t-test for dependent samples, Kruskal-Wallis Anova by ranks) were performed with STATISTICA (data analysis software system, version 8.0. http://www.statsoft.com/). The coefficient of variation (CV) representing the dispersion of data points around the mean can be regarded as an important measure of reliability in cases of `repeated' measurements. Upper and lower quartiles (Q) (i.e., the 25^th and 75^th percentiles) provide information on the distribution of each sleep parameter (box-plots with the bullet: Median and box: Lower and Upper Quartile). Medcalc® Version 11.0.0.0 (http//www.medcalc.be) was used for Bland-Altman plots, and concordance correlation coefficients. The Bland-Altman plot graphically represents the difference between two measurement techniques against the average of the two techniques (i.e., similar to student t-test of dependent samples output). Mean difference with limits of agreement (mean difference±1.96SD of the differences) together with the regression line (95% C.I.) are calculated. If the result is positive, ACT underestimates, and when the difference is negative ACT overestimates the measure of interest. The concordance correlation coefficient ρ_c expresses the precision and the accuracy of measures in terms of a best-fit line, which is compared to a 45° line through origin, with precision (how far each observation deviates from the best-fit line), and accuracy (how far the best-fit line deviates from the 45° line through origin). Therefore, the ρ_c coefficient has aspects innate to method comparison by expressing overlap in variance and indicating accuracy of the measures.

RESULTS

Since altering the sensitivity threshold of the actigraphy device analysis algorithm may affect TST and WASO calculations, it was anticipated and confirmed that the high sensitivity threshold (i.e., ACT20) overestimates wake or conversely underestimates sleep, while ACT 80 (high sensitivity) and ACT40 provided similar findings. Therefore to facilitate the readability of the results, most of these will focus on the default threshold (ACT40), and the results for all 3 thresholds will be shown only in the Tables. In addition, for the sake of completeness, Bland-Altman and box-plots for ACT20 and ACT80 threshold sensitivities can also be found in the Online Supplement section.

Sleep Period Time

No significant differences in SPT between NPSG and ACT40 were found, and their average difference was 4.5 minutes (Table 1 and Figure 1a). In fact, SPT was accurately and moderately precisely estimated, and therefore showed a substantial concordance correlation coefficient (i.e., 0.73 Table 2). This is also represented by the equivalence of CV and Qs, such that NPSG_SPT and ACT40_SPT were reliable (Table 1). Tentatively, based on the Bland-Altman plot regression line, one could conjecture that the extremes in the range of SPT, i.e., `short' and `long' SPT, potentially were less precisely estimated (range of −48.1 to 57.2 minutes).

Table 1.

Descriptives of sleep parameters - (mean±SD), [95% CI], the Coefficient of Variation, and quartiles- and the difference between NPSG and ACT for Total Sample (N=149)

							Quartiles
	measurement		difference	t(148)	p-value	Coefficient of Variation	lower	upper
Sleep Period Time (SPT)	NPSG	515.3 ± 32 [510.1 – 520.4]				6.2	499	535.5
	ACT40	510.8 ± 41.2 [504.1 – 517.4]	4.5 ± 27 [−0.17 – 8.9]	2.1	0.05	8.1	485	536
	ACT20 (n=105)	511.9 ± 38.9 [504.4 – 519.4]	5 ± 24.6 [0.2 – 9.7]	2.1	0.05	7.6	485	537
	ACT80 (n=105)	512.1 ± 39 [504.5 – 519.6]	4.8 ± 24.5 [0.06– 9.5]	2	0.05	7.6	485	538
Total Sleep Time (TST)	NPSG	491.7±40.6 [485.1 – 498.3]				8.3	468	521.5
	ACT40	459.5±38.0 [453.3 – 465.6]	32.2±33.4 [26.8 – 37.6]	11.8	0.0001	8.3	434	488
	ACT20 (n=105)	405.4±54.1 [394.9 – 415.9]	86±56 [75.1 – 96.8]	15.7	0.0001	13.3	364	441
	ACT80 (n=105)	459.3±43.2 [450.9 – 467.6]	32.1±44.9 [23.4 – 40.8]	7.3	0.0001	9.4	430	488
Wake After Sleep Onset (WASO)	NPSG	23.6±24.7 [19.6 – 27.6]				104.8	7	28.5
	ACT40	49.9±27.4 [45.5 – 54.4]	−26.3±34.4 [−31.9 - (−20.9)]	−9.4	0.0001	54.8	33	61
	ACT20 (n=105)	106.5±49.5 [97 – 116.1]	−81±55.9 [−91.8 - (−70.2)]	−14.9	0.0001	46.5	74	131
	ACT80 (n=105)	52.8±33.3 [46.4 – 59.3]	−27.3 ± 42.7 [−35.5 - (−19)]	−6.6	0.0001	63.1	30	70
Total Sleep Time Percentage (TST%)	NPSG	95.4±5 [94.6 – 96.2]				5.0	94.6	98.7
	ACT40	90.1±5 [89.3 – 90.9]	5.3±6.3 [4.3 – 6.3]	10.3	0.0001	5.50	87.9	93.4
	ACT20 (n=105)	79.3±9.1 [77.5 – 81]	15.8±10.3 [13.8 – 17.8]	15.7	0.0001	11.5	74.7	85.2
	ACT80 (n=105)	89.8±6.1 [88.6 – 90.9]	5.3±8.0 [3.7 – 6.84]	6.8	0.0001	6.8	87	94
Wake After Sleep Onset Percentage (WASO%)	NPSG	4.6±4.8 [3.8 – 5.4]				104.2	1.3	5.4
	ACT40	9.9±4.9 [9.1 – 10.6]	−5.3±6.3 [−6.3 - (−4.2)]	−10.2	0.0001	49.9	6.6	12.1
	ACT20 (n=105)	20.7±9.1 [19.0 – 22.5]	−15.8±10.3 [−17.8 - (−13.8)]	−15.7	0.0001	43.9	14.8	25.3
	ACT80 (n=105)	10.2±6.1 [9.1 – 11.4]	−5.3±8.0 [−6.8 - (−3.8)]	−6.8	0.0001	60	6	13.0
Sleep Onset Latency (SOL)	PSG	18.7±15.7 [16.2 – 21.3]				84.0	8	25
	ACT40	24.6±21.4 [21.2 – 28.1]	−5.9±22.1 [−9.5 - (−2.3)]	−3.3	0.001	87.0	9	32
	ACT20 (n=105)	25.1±21.8 [20.9 – 29.4]	−6.5 ± 20.5 [−10.5 - (2.5)]	−3.3	0.002	86.7	10	32
	ACT80 (n=105)	25.3±21.8 [21.1 – 29.5]	−6.6±20.5 [−10.6 - (−2.7)]	−3.3	0.001	86.1	10	32

p-value in bold: significant

Figure 1.

Bland –Altman plot with regression (95%CI) and Box Plot for distribution of sleep measure when derived from either NPSG or ACT for:

(a) Sleep Period Time (SPT);

(b) Total Sleep Time (TST);

(c) Wake After Sleep Onset (WASO);

(d) Total Sleep Time Percentage (TST %);

(e) Wake After Sleep Onset % (WASO %);

(f) Sleep Onset Latency (SOL)

In Supplement, the findings using high sensitivity threshold (ACT20) (Figure 2) and low sensitivity threshold (ACT80) (Figure 3) can be found.

Table 2.

Concordance correlation coefficient between ACT and NPSG

	concordance correlation coefficient [95% CI]			precision			accuracy
	40	20	80	40	20	80	40	20	80
Sleep Period Time	0.73 [0.65 – 0.79]	0.76 [0.67 – 0.83]	0.76 [0.67 – 0.82]	0.76	0.78	0.78	0.96	0.98	0.98
Total Sleep Time	0.47 [0.38 – 0.57]	0.13 [0.05 – 0.19]	0.33 [0.19 – 0.46]	0.64	0.33	0.43	0.75	0.37	0.77
Wake After Sleep Onset	0.09 [−0.02 – 0.19]	−0.002 [−0.05 – 0.05]	−0.02 [−0.15 – 0.11]	0.13	−0.006	−0.03	0.66	0.26	0.67
Total Sleep Time Percentage	0.09 [0.002 – 0.19]	0.0002 [−0.05 – 0.05]	−0.03 [−0.15 – 0.11]	0.16	0.009	−0.04	0.63	0.25	0.67
Wake After Sleep Onset Percentage	0.10 [0.0018 – 0.20]	0.0002 [−0.05 – 0.05]	−0.03 [−0.15 – 0.10]	0.16	0.009	−0.04	0.63	0.25	0.67
Sleep Onset Latency	0.29 [0.16 – 0.43]	0.37 [0.22 – 0.51]	0.37 [0.22 – 0.51]	0.33	0.43	0.43	0.91	0.87	0.87

Total Sleep Time

Total Sleep Time as calculated by ACT40 was on average 32.2 minutes shorter than NPSG (Table 1 and Figure 1b) and consequently ACT40 significantly underestimated actual sleep duration (p<.0001). A medium level concordance (0.47) was found, and was primarily due to the weaker precision (or decreased amount of explained variance, see Table 2). There was more variation in the NPSG_TST as can be seen in the parameter descriptives, such as CV, Qs, and the 95% CI of the means, which were not captured by ACT40. These values indicate relatively low sleep specificity. Differences between ACT40 and NPSG ranged from 97.6 minutes to −33.2 minutes, such that the longer the actual sleep duration, the more ACT40 tended to underestimate Total Sleep Time.

Wake After Sleep Onset

Wake After Sleep Onset, the putative complement of TST, was significantly overestimated by ACT40 (by an average of 26.3 minutes; Table 1 and Figure 1c). Table 2 shows the reduction in accuracy and precision of this parameter, which yielded a very small concordance correlation coefficient of 0.09. Furthermore, ACT40_WASO was unreliable, while different dispersion is seen in ACT40 versus NPSG, as inferred from the CV and the Qs. Indeed, looking at the Upper Q representing 75% of sample, WASO by ACT40 was 61 minutes as compared to 28.5 minutes by NPSG. The Bland-Altman plot further confirmed the large differences in WASO by both techniques (range:−93.7 to 41 minutes). Similar to the TST, the regression line tends towards favoring the `wake' state.

The latter two parameters, i.e., TST and WASO, can be expressed as a ratio (Table 1 and 2, Figures 1d and e), that in fact relies on SPT. Results indicate that these parameters clearly mirror one another with neither of the ACT40 indexes being a good proximate of the NSPG. Thus, ACT40_WASO % disagrees the most of all derived measures. Table 2 clearly indicates the low amount of variance explained (i.e., precision column) for WASO, TST% and WASO%.

The disagreement between TST and WASO prompted post-hoc analyses of the NPSG scores of those children whose scores were beyond the 1.96SD difference between NPSG_TST and ACT40_TST. This process indicated that only Stage 2 (expressed in min) was significantly different [H(2, N= 149) =4.9, p =.04]. Thus, children either spent significantly less or significantly more time in this stage, and ACT40 was insufficiently precise to capture these differences. Of note, significant, albeit weak correlations of several ACT40-derived parameters with age were found for:TST: −0.26p<0.05, WASO: 0.24, p<0.05, both TST % and WASO%: −0.25, p<0.01, but not for SPT.

Sleep Onset Latency

ACT40_SOL miscalculated sleep onset by about 6 minutes, and showed only weak concordance (0.33) with NPSG (Table 1), a finding that was altogether predictable, since sleep onset is a gradual process that may initially be manifest as immobility. Assessment of the Bland-Altman plot reveals that short SOL tends to be underestimated, whereas long SOL tends to be overestimated. This is similarly reflected in the discrepancy of the 95% CI and quartiles between the NPSG and ACT recordings.

Finally, when SPT, TST, WASO, TST%, WASO% and SOL analyses were assessed in relation to gender and ethnicity, we did not find evidence to indicate that the disagreement between ACT and NPSG was related to these potential confounders (Table 3).

Table 3.

Normative values (mean±SD) [95% CI] and difference between NPSG and ACT for Gender and Ethnicity samples

	Girls (N=84)				Boys (N=60)
	NPSG	ACT	NPSG-ACT	t(83)	NPSG	ACT	NPSG-ACT	t(59)
				p-value				p-value
Total Sleep Time (TST)	491.5±42.4 [482.3 – 500.7]	459.5±39.6 [450.9 – 468.1]	32±35 [24.2 – 39.6]	8.4 0.0001	492.8±38.2 [482.9 – 502.7]	459±35.7 [449.7 – 468.2]	33.9±32.3 [25.5 – 42.2]	8.1 0.0001
Sleep Period Time (SPT)	516.8±29.7 [510.4 – 523.3]	513.1±42.7 [503.8 – 522.4]	3.7±28.2 [−2.4 – 9.8]	1.2 0.23	514±35.1 [504.9 – 523.1]	508±39.3 [497.8 – 518.1]	6±26 [−0.6 – 12.7]	1.8 0.4
Wake After Sleep Onset (WASO)	25.3±28 [19.3 – 31.4]	51.5±30 [45 – 58]	−26.1±39.8 [−34.8 – (−17.5)]	−6 0.0001	21.2±19.9 [16 – 26.3]	48.8±24.3 [42.5 – 55]	−27.6±26.6 [−34.4 – (−20.9)]	−8.0 0.0001
Total Sleep Time Percentage (TST%)	95.1±5.4 [93.9 – 96.2]	89.7±5.4 [88.5 – 90.9]	5.4±7.3 [3.8 – 6.9]	6.7 0.0001	95.9±3.9 [94.9 – 96.7]	90.5±4.4 [89.3 – 91.6]	5.4±4.9 [4.2 – 6.7]	8.5 0.0001
Wake After Sleep Onset Percentage (WASO%)	4.9±5.4 [3.8 – 6.1]	10.1±5.4 [9 – 11.3]	−5.2±7.2 [−6.7 – (−3.6)]	−6.6 0.0001	4.1±3.9 [3.1 – 5.1]	9.6±4.4 [8.5 – 10.8]	−5.5±5.1 [−6.8 – (−4.2)]	−8.4 0.0001
Sleep Onset Latency (SOL)	18±14.4 [14.9 – 21.1]	23.1±17.9 [19.2 – 26.9]	−5.1±20.76 [−9.6 – (−0.58)]	−2.3 0.027	19.7±17.8 [15.1 – 24.3]	27.6±25.9 [−20.9 – 34.3]	−7.8±24.2 [−14.2 – (−1.5)]	−2.5 0.01

	White (N=111)				African American (N=18)				Other (N=9)
	NPSG	ACT	NPSG-ACT	t(110)	NPSG	ACT	NPSG-ACT	z(18)	NPSG	ACT	NPSG-ACT	z(9)
				p-value				p-value				p-value
Total Sleep Time (TST)	497.3±40 [489.8 – 504.8]	460.4±36.9 [453.4 – 467.3]	36.9±32.5 [30.8 – 43]	7.1 0.0001	482.8±41 [462.3 – 503.2]	457.7±36.2 [439.7 – 475.7]	25±28.2 [11 – 39.1]	2.94 0.01	466.6±30.6 [443.1 – 490.1]	472.3±33.4 [446.6 – 498]	−5.7±28.9 [−27.9 – 16.6]	0.41 0.68
Sleep Period Time (SPT)	519.2±29.5 [513.6 – 524.8]	514.2±39.6 [506.8 – 521.7]	5±26.5 [−0.4 – 9.9]	1.9 0.29	510.7±43.1 [489.2 – 532.1]	497.6±44.2 [475.6 – 519.5]	13.1±22 [2.2 – 24.1]	2.44 0.01	491.3±28.4 [469.5 – 513.1]	509.8±41.8 [477.7 – 541.9]	−18.4±26.6 [−38.9 – 2]	2.43 0.02
Wake After Sleep Onset (WASO)	21.9±24.9 [17.2 – 26.4]	52.6±27.2 [47.5 – 57.7]	−30.7±34.4 [−37.2 – (−24.3)]	−9.4 0.0001	27.9±26.2 [14.9 – 41]	37.1±22.4 [25.9 – 48.2]	−9.1±26.1 [−22.1 – 3.9]	1.37 0.17	24.7v23.9 [6.4 – 43.1]	37±22.9 [19.4 – 54.6]	−12.3±29.8 [−35.2 – 10.6]	1.24 0.21
Total Sleep Time Percentage (TST%)	95.7±4.8 [94.8 – 96.7]	89.6±4.9 [88.7 – 90.6]	6.1±6.4 [4.9 – 7.3]	10.2 0.0001	94.6±4.8 [92.3 – 97]	92.2±4 [90.1 – 94.2]	2.5±4.8 [0.1 – 4.9]	1.85 0.06	95±4.7 [91.4 – 98.7]	92.8±4.1 [89.7 – 95.9]	2.2±5.8 [−2.2 – 6.7]	1.24 0.21
Wake After Sleep Onset Percentage (WASO%)	4.3±4.8 [3.3 – 5.2]	10.3±4.9 [9.4 – 11.2]	−6±6.3 [−7.2 – (−4.9)]	−10.2 0.0001	5.4±4.6 [3 – 7.7]	7.8±4 [5.8 – 9.8]	−2.4±4.8 [−4.8 – 0]	1.85 0.06	5±4.7 [1.3 – 8.6]	7.2±4.1 [4 – 10.3]	−2.2±5.8 [−6.6 – 2.2]	1.24 0.21
Sleep Onset Latency (SOL)	19±15.3 [16.1 – 21.8]	25±22.4 [20.8 – 29.2]	−6.1±22.6 [−10.4 – (−1.8)]	−2.8 0.005	22.9±22.3 [11.8 – 33.9]	29.5±22.2 [18.4 – 40.5]	−6.6±24.5 [−18.7 – 5.6]	1.5 0.15	11.5±4.6 [7.9 – 15]	14.6±6.7 [9.4 – 19.7]	−3.1±8.9 [−9.9 – 3.7]	0.7 0.50

DISCUSSION

This study on the reliability of ACT for the determination of specific sleep measures in school-aged children showed that ACT is concordant on sleep quantity, but less reliable in estimating sleep quality when compared to concomitant measurements obtained during NPSG. Indeed, Sleep Period Time was accurately monitored, yet ACT underestimated Total Sleep Time by ~32.2 minutes, and overestimated Wake After Sleep Onset by ~ 26.3 minutes. This effect occurred independently of gender, and ethnicity.

Interpretations of this study are restricted to the brand of ACT used herein, and of course to the sleep data extraction methodology developed by this ACT brand. Also, we did not conduct an epoch-by-epoch comparison and did not specifically pursue the exploration of receiver operator curve characteristics of different threshold sensitivity values, and such considerations clearly merit future studies. It is possible that other devices or analytical algorithms may further improve the relatively good performance of the current one. Of note, the actigraph algorithm is a weighted algorithm based on the 1-minute epoch sampling, and we relied on the Sleep Start and Sleep End automatically calculated by the software. In addition, the children only wore the device on their non-dominant wrist (Paavonen et al., 2002, Littner et al., 2003, Morgenthaler et al., 2007). We should also point out that by design, the study was not conducted in a naturalistic environment and that the NPSG was considered as the reference. Lastly, we focused our analyses on 2 interrelated parameters that essentially do not require external input (except SOL), which is apt to potentially increase error or bias, and merely denoted actigraphic `sleep-interval' as the time window in which sleep occurred within the 24-hour period. Therefore this study excluded any sleep parameters that are known to be less valid (Paquet et al., 2007), but at the same time cannot reflect on ACT performance over a 24hr schedule.

While there is little doubt that ACT placement, algorithms and corresponding thresholds used, are all likely to affect the sensitivity and specificity of the device, our findings are in marked agreement with previous studies (Cole et al., 1992, Paquet et al., 2007, Littner et al., 2003, Paavonen et al., 2002, Insana et al., 2010). Indeed, for all previous and current studies, ACT reliably monitored and accurately delineated sleep period to within 1-hour discrepancy. However, ACT trends to enhance detection of wakefulness within the sleep period, resulting in reduced sleep specificity, and the latter could be related to the ACT-associated difficulties in classifying `Stage 2-sleep behavior' (Insana et al., 2010). Furthermore, we found weak but significant correlations between NPSG and ACT differences and age, i.e., positively with WASO and negatively with TST (Aronen et al., 2001).

The imprecise detection of nocturnal wakefulness appears to be the Achilles heel of ACT (Sadeh and Acebo, 2002). The immediate question derived from such consistent observation would be whether this error is systematic or random in nature (Tryon, 2004). Notwithstanding, the overestimation of wakefulness during the sleep period could potentially undermine ACT applicability towards monitoring sleep quality in a pre- vs. post-treatment cohort, as evidenced by our Band-Altman plot dispersion. Low sleep specificity of ACT may have a serious impact on the monitoring of `non-normal' sleep, and the 3 following guideline propositions may prevent further imprecision: (i) use of conjoint sleeplog in a naturalistic environment; (ii) a minimum of three consecutive 24-hour recording periods; and (iii) consistent positioning of the device on the non-dominant wrist (Littner et al., 2003, Morgenthaler et al., 2007). In this study, ACT was used in tandem with NPSG, and therefore, no parental or subjective reports were necessary, but such reports would be required in a home-environment. We also did not analyze any of the sleep parameters that depend on diary inputs, such that we can exclusively comment on the disagreement between 2 objective calculations obtained through ACT and through NPSG. Since our ACT calculated parameters were assessed independently from external inputs, we should be reminded that the latter could either aid in the concordance or detract from it during sleep monitoring. For example, if the discrepancy between log and ACT occurs in a naturalistic environment, which measurement should we rely upon based on the disagreement found in our study? What about inter-individual differences? This issue is particularly germane to the utility and reliability of the device, since statistically outliers will distort the mean, and additional variance will be introduced when manual scoring is implemented. Of course, in conditions that require extended ACT monitoring, we should exercise caution in the interpretation of ACT reports, particularly in children with underlying sleep alterations. Of note, the values reported for ACT_TST and ACT_WASO did not match perfectly. This minor discrepancy should be ascribed to potential measurement errors that most likely derive from ACT design. Finally, our ACT settings concur with published guidelines (Morgenthaler et al., 2007, Littner et al., 2003), and the effects of changes in TSV favor the use of the default threshold settings, and correspondingly advocate for the use of epoch-by-epoch analyses when SOL, a gradual process, is to be determined from actigraphic recordings (Acebo et al.,2006).

In summary, we have shown that when compared to actual NPSG-derived measurements in pre-school and school-aged healthy children, an ACT device provides a rather accurate estimate of sleep duration, but is prone to increased errors when attempting to assess sleep quality measures, such as awakenings during the sleep period. In this context, and particularly in a naturalistic environment, there are many questions that remain unanswered, and should be explored in the context of sleep disorders in children.