Abstract
Study Objectives:
Accurate objective measurement of sleep, an important health behavior, is needed. Individuals with type 1 diabetes mellitus (T1DM) have altered sleep architecture and reduced sleep quality in comparison with healthy controls. The aim of this investigation was to compare a commonly used actigraphy device, Actiwatch2, with polysomnography (PSG)-based measures of sleep in young adults with T1DM, and to determine which Actiwatch2 threshold setting provides the best correspondence.
Methods:
Subjects age 18–30 years with T1DM wore the Actiwatch2 while simultaneously undergoing in-laboratory PSG. Sleep parameters were derived from the Actiwatch2 using the three different sensitivity thresholds (low, medium, and high) provided by the manufacturer and compared with sleep parameters from PSG. Statistical analysis included intraclass correlation coefficients and Bland-Altman plots for comparison of sleep parameters. Cohen kappa and the prevalence-adjusted and bias-adjusted kappa (PABAK) were calculated to determine agreement between epoch-by-epoch sleep and wake data measured by the PSG versus Actiwatch2.
Results:
Twenty-seven subjects were included in the analysis. The low threshold setting provided the greatest agreement and least bias in comparison with PSG for sleep parameters (intraclass correlation coefficient range 0.38 to 0.77). Mean differences between the low setting and PSG were nonsignificant (P > .65) for all sleep parameters except sleep onset latency (P = .04). All three settings provided approximately equivalent and moderate epoch-by-epoch agreement with the PSG (PABAK coefficients ranging from 0.56 to 0.63).
Conclusions:
When measuring sleep with the Actiwatch2 in young adults with T1DM, the low threshold setting provides the most accurate estimates of sleep parameters in comparison with PSG.
Citation:
Farabi SS, Quinn L, Carley DW. Validity of actigraphy in measurement of sleep in young adults with type 1 diabetes. J Clin Sleep Med. 2017;13(5):669–674.
Keywords: actigraphy, sleep, type 1 diabetes, validation
INTRODUCTION
Sleep is increasingly recognized as an important health behavior. Accurate objective measurement of sleep is crucial to support quantitative research into healthy and disordered sleep. For both clinical and research applications, polysomnography (PSG) is recognized as the gold standard for objective measurement of sleep architecture and sleep quality. However, PSG testing can be burdensome for researchers and research participants, as it requires spending one or more nights in the laboratory. Further, polysomnography may not capture an individual's normal sleep, as a laboratory is not the natural sleeping environment. In recent years, actigraphy has been used to objectively quantify sleep in the natural environment. This technique has been validated for use in healthy subjects1–3 as well as in individuals with sleep disorders such as insomnia.4
The Actiwatch2 (Philips Respironics, Oregon, United States) is a widely used actigraph worn on the nondominant wrist that has the capability to determine sleep and wake patterns.5 It has been validated against PSG in healthy children6 and adults.3 Sleep parameters can be derived from Actiwatch2 recordings using three different threshold settings (low, medium, or high) for detection of wake-related accelerations (movements) before and during sleep. The threshold settings use different magnitudes of activity for discriminating sleep and wake states, and thus affect the sleep parameters derived from the actigraph. More specifically, the high threshold setting requires that a greater amount of activity be counted to detect wakefulness.
BRIEF SUMMARY
Current Knowledge/Study Rationale: Sleep architecture differs in individuals with type 1 diabetes in comparison with controls. Actigraphy has not been validated for use in individuals with type 1 diabetes.
Study Impact: This study demonstrates that using the low setting for the Actiwatch2 provides the most valid results in young adults with type 1 diabetes. This is important for consideration when using the Actiwatch2 in individuals with type 1 diabetes.
Investigators have used actigraphy to measure sleep in individuals with type 1 diabetes mellitus (T1DM),7,8 but to our knowledge there have been no validation studies of actigraphy in this group. Because PSG studies have identified poor sleep quality9,10 and altered sleep architecture in individuals with T1DM,11,12 it is important for ongoing research to determine if actigraphy provides a valid measure of sleep in individuals with T1DM. The aim of this investigation was to compare Actiwatch2- and PSG-based measures of sleep in young adults with T1DM, and to determine which threshold setting provides the best correspondence.
METHODS
Subjects
All procedures were approved by the Institutional Review Board at the University of Illinois at Chicago. As part of a larger study,13 subjects who were aged 18 to 30 years and had undergone treatment for T1DM for at least 5 years by insulin pump therapy were recruited from the community to participate in the current study. Exclusion criteria included self-report of pregnancy; night or rotating shift-work; use of corticosteroids; diagnosis of primary cardiovascular disease, retinopathy, nephropathy or peripheral neuropathy; uncontrolled thyroid disease; diagnosed primary sleep disorder or chronic use of oral sleep medications; use of psychiatric medications (eg, antidepressants) or illicit drugs (eg, marijuana or cocaine); or recent history (last 2 months) of severe metabolic instability (eg, hospitalization for hypoglycemia, occurrence of hypoglycemic seizures, or ketoacidosis).
Measurement of Sleep
After informed consent was obtained, subjects received an Actiwatch2 to wear on the nondominant wrist while concomitantly undergoing an overnight PSG at the Sleep Science Center of the University of Illinois at Chicago. PSG was conducted by a registered polysomnographic technologist. Lights out for each subject was between 10:00 PM and 11:00 PM and lights on was at 6:00 AM, ensuring at least 7 hours in bed.
PSG testing comprised computer-based recording (Alice5, Respironics, Pennsylvania, United States) of 2 central, 2 frontal, and 2 occipital electroencephalogram leads; bilateral referential electrooculogram; chin and anterior tibialis electromyogram; lead I electrocardiogram; respiratory movement of thorax and abdomen by piezoelectric strain gauges; airflow via oronasal thermistors and nasal pressure cannula; and arterial oxygen saturation of hemoglobin by pulse oximeter. PSG studies were scored according to the criteria of the American Academy of Sleep Medicine13 by a single American Board of Sleep Medicine Diplomate. Parameters extracted for direct comparison to Actiwatch2 measures included total sleep time (TST), sleep onset latency (SOL), wake after sleep onset (WASO), number of awakenings, and sleep efficiency (SE).
Data from the Actiwatch2 were uploaded and analyzed on 30-second epochs using Actiware 6.0 software (Philips Respironics, Oregon, United States). Actiwatch2 data were annotated to set the timing of lights out and lights on based on the PSG study. The Actiware software provided three threshold settings for identification of wake epochs: low (20 activity counts), medium (40 activity counts), and high (80 activity counts). We used the recommended setting of 10 minutes immobile (fewer than 2 activity counts in 30 seconds) for determination of sleep onset and offset. For each recording, TST, WASO, SOL, SE, and number of awakenings were determined using all three settings.
Statistical Analyses
Epoch-by-epoch analysis
Raw activity counts for each setting obtained from the Actiwatch2 were aligned by time stamps with the epoch-based scoring from the PSG. Computers used to collect PSG data and to initialize the Actiwatch2 devices were time synchronized on the same network. PSG sleep staging was converted to a binary code to align with the Actiwatch2 (1 for sleep; 0 for wake). Agreement between the Actiwatch2 and PSG was determined by calculating the Cohen kappa statistic.14 Because sleep was much more prevalent than wake during the PSG, we also calculated the prevalence-adjusted and bias-adjusted kappa (PABAK) to provide equal weights to sleep and wake, as has been previously reported.1,15 Higher values of the Cohen kappa statistic as well as the PABAK indicate better agreement between the two methods. According to the study by Landis and Koch, kappas (for both the Cohen and PABAK statistics) of 0.21–0.4 indicate fair agreement, values of 0.41– 0.6 indicate moderate agreement, 0.61–0.8 indicate substantial agreement, and values of 0.8–1.0 indicate nearly perfect agreement.16
Concordance between sleep parameters
To quantify the agreement between the methods over the entire dataset, absolute intraclass correlation coefficients (ICC) were determined using each of the three Actiwatch2 settings. We utilized a two-way random-effects model for average intraclass correlation. One-way analysis of variance with Scheffe test for multiple comparisons was used to compare sleep parameters obtained by PSG and Actiwatch2 at each of the three different threshold settings.
Bland-Altman plots were used to determine agreement between sleep parameters derived from the Actiwatch2 with those reported from the PSG.17,18 Bland-Altman plots have been used to compare a new measurement method to the gold standard in clinical practice.19 This method plots the difference versus the mean value of a variable simultaneously derived by two methods (eg, WASO measured by the Actiwatch2 versus WASO measured by PSG). As discussed in the study by Bland and Altman, it is recommended that the difference between two methods is plotted against their mean even if one is considered the gold standard.20
RESULTS
Data were collected between 2014 and 2015. Twenty-nine subjects were enrolled. However, as identified by PSG, the apneahypopnea index was greater than 5 for two of the subjects (an exclusion criteria for the study); thus, they were excluded, leaving a total of 27 subjects (16 female) for analysis. Mean (± standard deviation) age was 23.8 ± 4.1 years and mean body mass index was 26.0 ± 3.3 kg/m2. Subjects had a mean duration of diabetes of 12.1 ± 4.6 years and mean glycemic control measured by hemoglobin A1c ([HbA1c] average glycemic control for past 3 months) was 7.9% ± 1.4%. The American Diabetes Association recommends that individuals with T1DM should aim for an HbA1c of ≤ 7.0%21; thus, our subjects had, on average, higher than recommended glucose levels.
Mean summary statistics for sleep parameters are provided in Table 1 for PSG and Actiwatch2 using all three threshold settings. Using the low threshold setting, the Actiwatch2 significantly underestimated PSG-derived sleep latency (P ≤ .04), but all other parameters were statistically equivalent (P ≥ .65) to the PSG-derived estimates. In contrast, when using the high threshold setting, the Actiwatch2 overestimated TST and SE (P ≤ .01 for each) but underestimated SOL and WASO (Table 2). With the medium setting, SOL and WASO were underestimated in comparison to PSG estimates. WASO and TST differed significantly between the high and low threshold settings (P ≤ .05).
Table 1.
Table 2.
Epoch-by-Epoch Analysis
Table 2 provides the kappa and PABAK coefficients, and percent agreement between Actiwatch2 and PSG for each threshold setting. The three settings produced largely equivalent epoch by epoch agreement, with low kappas between 0.23 and 0.25 and moderate strength PABAK coefficients ranging from 0.56 to 0.63.
Concordance between Sleep Parameters
Intraclass correlations for each threshold setting are listed in Table 3. The ICC was substantial for TST and modest for all other parameters. The low threshold setting exhibited a slightly but consistently higher ICC for all parameters except SOL, which was equal for all settings.
Table 3.
Bland-Altman analysis demonstrated that when using the medium or high setting, the Actiwatch2 provided biased estimates—overestimating TST and SE and underestimating SOL and WASO (Table 4; P < .01 for each). There was no significant bias between Actiwatch2- and PSG-derived measures of any sleep parameter when using the low threshold setting. The low and medium settings tended to overestimate the number of awakenings whereas the high setting tended to underestimate the number of awakenings; but none of these differences from the PSG-derived number of awakenings were statistically significant.
Table 4.
Figure 1 presents the Bland-Altman plots for the low threshold setting. The Bland-Altman plot for SOL (Figure 1B) highlights the fact that based on actigraphy 13 subjects had an SOL of zero, as the Actiwatch2 already had identified sleep prior to PSG-determined lights out. This resulted in the straight line that is seen, as the mean of SOL from the PSG and actigraphy is the value of SOL measured by PSG. In the Bland-Altman plot for WASO (Figure 1D), there is a slight trend for underestimation by actigraphy especially with higher amounts of WASO; however, this was not significant (Table 4).
DISCUSSION
This study provides the first direct validation of wrist actigraphy in young adults with T1DM. We demonstrated that the low threshold setting for detection of wakefulness provided the greatest agreement and least bias in comparison to PSG for objective quantification of overnight summary sleep parameters (Table 1). All three settings provided approximately equivalent and moderate epoch-by-epoch agreement (indicated by PABAK) with the PSG (Table 2). Although there was no control group in the current study, some of our findings differ in comparison to what has been reported in the literature for healthy adults and children.
Differences in Sleep Parameter Agreement by Threshold Setting
The medium setting is recommended by the manufacturer and has been found to provide valid results in healthy individuals. Shin et al. reported that using the medium threshold for Actiwatch2 provided minimal bias for SOL, TST, and SE but overestimated WASO at different ambient temperatures in a small group of healthy young adults.3 Meltzer et al. reported that the medium setting in healthy children provided results most similar to PSG.6 Similarly, Ward et al. reported that in healthy children as well as those with juvenile arthritis and asthma, the medium threshold setting provided the least biased results.22 Interestingly, these authors also found that all settings provided less accurate results in children with chronic illnesses as compared to healthy children.22 We do not have a control group in the current study, but we report accuracy similar to that reported by Cellini et al. for healthy young adults during a nap.1 However, in contrast to those authors, in the current study, we found that the low threshold setting provided the most accurate results for all sleep parameters.
In the current study, we report that the Actiwatch2 overestimated TST and SE and underestimated SOL and WASO when using the medium or high settings, but these biases were not statistically significant when using the low threshold setting (Table 4). Our findings are in contrast to previous investigations in healthy study populations of children and adults. In a group of healthy adults, the Actiwatch2 underestimated TST at various ambient temperatures (7.1 to 17.6 minutes) and overestimated WASO (7.1 to 12.4 minutes) when using the medium setting.3 Using the Actiwatch64 (Philips Respironics, Oregon, United States) device, an earlier Actiwatch model no longer on the market, Cellini and colleagues reported the low and medium threshold settings underestimated TST (by 1.1 to 4.7 minutes) whereas the high threshold produced results similar to the PSG.1 These differences from the current findings may reflect that sleep is different in young adults with T1DM but also could be due to several factors, including differences in actigraphy devices and timing of sleep period (naps versus overnight). This highlights the importance of validating specific actigraphy devices and methodologies in specific populations, such as young adults with T1DM.
WASO was underestimated for all three settings; however, this underestimation was not significant for the low threshold. Similar to our findings, in a group of healthy young adults, Marino et al. found that WASO tended to be underestimated by actigraphy as compared to PSG.23 Similarly in another study, the low threshold overestimated WASO but the medium and high thresholds underestimated WASO (by 2.5–3.7 minutes).1
Estimation of SOL by the Actiwatch2 was not valid in our study; 13 subjects had a value of zero for SOL at all three settings after editing for lights out based on the PSG. Similar to previous reports,24 SOL in the current study did not differ between the sensitivity thresholds. This was because sleep onset was determined using the criterion of immobility, defined as less than 2 activity counts per 30 seconds for 10 consecutive minutes; a criterion not affected by the sensitivity thresholds for wakefulness detection. Conversely, the Actiware software allows a second choice for sleep onset detection based on the transition from detected wakefulness to detected sleep. In this case, SOL will be affected by the choice of sensitivity threshold, as recently reported by Matsuo et al. for a group of healthy adults.25 We also analyzed our data using this alternative approach to sleep onset detection, and the resulting SOL estimates did indeed depend on the threshold setting chosen. However, the net performance for estimation of SOL was also not valid, with high mean error and negative intraclass correlation versus SOL based on PSG recording.
One reason for differences between our study and those in healthy children and adults is that altered sleep architecture has been reported in people with T1DM. Children12 and young adults11 with T1DM spend more time in light (stage 2) sleep and exhibit more awakenings from sleep26 than do healthy controls. The low setting for the Actiwatch2 has a lower threshold for identification of wakefulness based on activity counts; less activity is needed for an epoch of wake to be scored. Thus, it is possible that individuals with T1DM experience more brief awakenings from sleep and the low threshold can help to detect these periods better than the medium or high thresholds.
Epoch-by-Epoch Agreement
In the current study, we found that all settings yielded similar but low kappa coefficients ranging from 0.18–0.25 for the epoch-by-epoch comparison with PSG scoring (Table 2). The PABAK coefficients, which take into account the low prevalence of wake during a sleep period were much higher but again similar, ranging from 0.56–0.63 and showing moderate agreement16 between PSG and Actiwatch2 at all three settings. Cellini et al. reported similar agreement for the Actiwatch64 versus PSG in healthy young adults during a nap period, with PABAKs ranging from 0.48 to 0.61.1
In summary, we report that using the low threshold setting for the Actiwatch2 provides the highest agreement with PSG in measures of TST, SE, WASO, and number of awakenings in young adults with T1DM. When constraining lights out time to the accurate value provided by PSG, the Actiwatch2 did not provide reliable estimation of SOL. Epoch-by-epoch agreement was substantial and not affected by threshold setting. However, caution must be used in attempting estimation of SOL when using this device without PSG in young adults with T1DM to measure sleep. We conclude that when measuring sleep with the Actiwatch2 in people with T1DM, researchers should pursue analysis using the low threshold setting to most accurately estimate sleep parameters.
DISCLOSURE STATEMENT
This study was funded in part by a grant from the American Association of Diabetes Educators Foundation/Sigma Theta Tau International and a NIH TL-1 Institutional Fellowship, 5TL1TR000049-05. Work for this study was performed at the University of Illinois Chicago, Chicago, IL. This was not an industry supported study. Dr. Carley holds stock in, receives royalties from, and owns intellectual property rights relating to RespireRx Pharmaceuticals, Inc. The other authors have indicated no financial conflicts of interest.
ABBREVIATIONS
- HbA1c
hemoglobin A1c
- ICC
intraclass correlation coefficient
- PABAK
prevalence, adjusted and bias, adjusted kappa
- PSG
polysomnography
- SE
sleep efficiency
- SOL
sleep onset latency
- T1DM
type 1 diabetes mellitus
- TST
total sleep time
- WASO
wake after sleep onset
REFERENCES
- 1.Cellini N, Buman MP, McDevitt EA, Ricker AA, Mednick SC. Direct comparison of two actigraphy devices with polysomnographically recorded naps in healthy young adults. Chronobiol Int. 2013;30(5):691–698. doi: 10.3109/07420528.2013.782312. [DOI] [PubMed] [Google Scholar]
- 2.Kanady JC, Drummond SPA, Mednick SC. Actigraphic assessment of a polysomnographic-recorded nap: a validation study. J Sleep Res. 2011;20(1 Pt 2):214–222. doi: 10.1111/j.1365-2869.2010.00858.x. [DOI] [PubMed] [Google Scholar]
- 3.Shin M, Swan P, Chow CM. The validity of Actiwatch2 and SenseWear armband compared against polysomnography at different ambient temperature conditions. Sleep Sci. 2015;8(1):9–15. doi: 10.1016/j.slsci.2015.02.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lichstein KL, Stone KC, Donaldson J, et al. Actigraphy validation with insomnia. Sleep. 2006;29(2):232–239. [PubMed] [Google Scholar]
- 5.Martin JL, Hakim AD. Wrist actigraphy. Chest. 2011;139(6):1514–1527. doi: 10.1378/chest.10-1872. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Meltzer LJ, Walsh CM, Traylor J, Westin AML. Direct comparison of two new actigraphs and polysomnography in children and adolescents. Sleep. 2012;35(1):159–166. doi: 10.5665/sleep.1608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Borel A-L, Benhamou P-Y, Baguet J-P, et al. Short sleep duration is associated with a blood pressure nondipping pattern in type 1 diabetes: the DIAPASOM study. Diabetes Care. 2009;32(9):1713–1715. doi: 10.2337/dc09-0422. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Borel A-L, Pépin J-L, Nasse L, Baguet J-P, Netter S, Benhamou P-Y. Short sleep duration measured by wrist actimetry is associated with deteriorated glycemic control in type 1 diabetes. Diabetes Care. 2013;36(10):2902–2908. doi: 10.2337/dc12-2038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Perfect MM, Elkins GR, Lyle-Lahroud T, Posey JR. Stress and quality of sleep among individuals diagnosed with diabetes. Stress Health. 2010;26(1):61–74. [Google Scholar]
- 10.van Dijk M, Donga E, van Dijk JG, et al. Disturbed subjective sleep characteristics in adult patients with long-standing type 1 diabetes mellitus. Diabetologia. 2011;54(8):1967–1976. doi: 10.1007/s00125-011-2184-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Jauch-Chara K, Schmid SM, Hallschmid M, Born J, Schultes B. Altered neuroendocrine sleep architecture in patients with type 1 diabetes. Diabetes Care. 2008;31(6):1183–1188. doi: 10.2337/dc07-1986. [DOI] [PubMed] [Google Scholar]
- 12.Perfect MM, Patel PG, Scott RE, et al. Sleep, glucose, and daytime functioning in youth with type 1 diabetes. Sleep. 2012;35(1):81–88. doi: 10.5665/sleep.1590. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Farabi SS, Carley DW, Quinn L. EEG power and glucose fluctuations are coupled during sleep in young adults with type 1 diabetes. Clin Neurophysiol Off J Int Fed Clin Neurophysiol. 2016;127(8):2739–2746. doi: 10.1016/j.clinph.2016.05.357. [DOI] [PubMed] [Google Scholar]
- 14.Feingold M. The equivalence of Cohen's Kappa and Pearson's Chi-Square statistics in the 2 x 2 table. Educ Psychol Meas. 1992;52(1):57–61. [Google Scholar]
- 15.Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–429. doi: 10.1016/0895-4356(93)90018-v. [DOI] [PubMed] [Google Scholar]
- 16.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. [PubMed] [Google Scholar]
- 17.Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. J R Stat Soc Ser Stat. Series D (The Statistician) 1983;32(3):307–317. [Google Scholar]
- 18.Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135–160. doi: 10.1177/096228029900800204. [DOI] [PubMed] [Google Scholar]
- 19.Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–310. [PubMed] [Google Scholar]
- 20.Bland JM, Altman DG. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. 1995;346(8982):1085–1087. doi: 10.1016/s0140-6736(95)91748-9. [DOI] [PubMed] [Google Scholar]
- 21.American Diabetes Association. Standards of medical care in diabetes-2015 abridged for primary care providers. Clin Diabetes Publ Am Diabetes Assoc. 2015;33(2):97–111. doi: 10.2337/diaclin.33.2.97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Ward TM, Lentz M, Kieckhefer GM, Landis CA. Polysomnography and actigraphy concordance in juvenile idiopathic arthritis, asthma and healthy children. J Sleep Res. 2012;21(1):113–121. doi: 10.1111/j.1365-2869.2011.00923.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Marino M, Li Y, Rueschman MN, et al. Measuring sleep: accuracy, sensitivity, and specificity of wrist actigraphy compared to polysomnography. Sleep. 2013;36(11):1747–1755. doi: 10.5665/sleep.3142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Chae KY, Kripke DF, Poceta JS, et al. Evaluation of immobility time for sleep latency in actigraphy. Sleep Med. 2009;10(6):621–625. doi: 10.1016/j.sleep.2008.07.009. [DOI] [PubMed] [Google Scholar]
- 25.Matsuo M, Masuda F, Sumi Y, et al. Comparisons of portable sleep monitors of different modalities: potential as naturalistic sleep recorders. Front Neurol. 2016;7:110. doi: 10.3389/fneur.2016.00110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Matyka KA, Crawford C, Wiggs L, Dunger DB, Stores G. Alterations in sleep physiology in young children with insulin-dependent diabetes mellitus: relationship to nocturnal hypoglycemia. J Pediatr. 2000;137(2):233–238. doi: 10.1067/mpd.2000.107186. [DOI] [PubMed] [Google Scholar]