Agreement between actigraphic and polysomnographic measures of sleep in adults with and without chronic conditions: A systematic review and meta-analysis

Summary

Wrist actigraphy (ACT) may overestimate sleep and underestimate wake, and the agreement may be lower in people with chronic conditions who often have poor sleep and low activity levels. The purpose of this systematic review is to compare the agreement between ACT and polysomnographic (PSG) measures of sleep in adults without chronic conditions and sleep complaints (healthy) and with chronic conditions. We conducted a systematic review and metaanalysis using PRISMA guidelines. We searched PubMed, OVIDEMBASE, OVIDMEDLINE, OVIDPsycINFO, CENTRAL, CINAHL, ClinicalTrials.gov, International Clinical Trials Registry, and Open Grey. We included 96 studies with a total of 4,134 participants, of whom 762 (18.4) were healthy adults and 724 (17.5%) were adults with chronic conditions. Among adults with chronic conditions, ACT overestimated TST, compared to PSG [M = 22.42 minutes (CI 95%: 11.92, 32.91 minutes)] and SE [M = 5.21% (CI 95%: 1.41% to 9.00%)]. ACT underestimated SOL [M = −7.70 minutes (CI 95%: −15.22, −0.18 minutes)], and WASO [M = −10.90 minutes (CI 95%: −26.01, 4.22 minutes)]. These differences were consistently larger between ACT and PSG sleep measures compared to healthy adults. Research is needed to better understand factors that influence the agreement between ACT and PSG among people with chronic conditions.

INTRODUCTION

Researchers have used actigraphs, accelerometers that estimate sleep characteristics from motor activity, since the early 1980’s using computerized algorithms calibrated against polysomnography (PSG), the “gold standard” of sleep measurement. ACT, compared with PSG, is less intrusive and expensive, more ecologically valid, and more practical for continuous data collection over days or weeks (1,2). These advantages contribute to the feasibility of ACT to estimate sleep characteristics and evaluate the effects of sleep interventions in clinical and community settings (e.g., home, nursing home, acute care hospital) and across the spectrum of health and illness (1–3). Narrative reviews indicate various levels of agreement between ACT and PSG in single studies of several populations (1). The American Academy of Sleep Medicine published guidelines for the use of wrist actigraphy for the diagnoses of sleep disorders in 2007 (4) and updated the systematic review and clinical guidelines for the use of actigraphy in people with sleep disorders in 2018 (5,6). The authors concluded that ACT is useful in estimating total sleep time (TST) and sleep onset latency (SOL) among adults with insomnia compared to PSG (5). However, this review did not compare ACT and PSG in people with medical or psychiatric disorders. Despite the use of ACT in many studies of people with chronic conditions who also frequently have poor sleep, the extent to which ACT measurements are concordant with PSG measures in these groups is not known.

Unlike PSG that relies on electro-encephalography (EEG), electro-oculography (EOG), and electromyography (EMG) to quantify sleep characteristics, ACT estimates of sleep are derived from motor activity. ACT is generally a reliable and valid measure of sleep among people without chronic conditions and with good quality sleep, yielding good agreement rates with PSG for total sleep time (3). However, because ACT measures activity (7), it may overestimate sleep and under-estimate wake with lower levels of agreement for sleep outcomes that depend upon correct identification of wake, such as SOL, SE, and WASO (3). These measures may be biased by increased or decreased motor activity, the actigraph technology (e.g., microelectromechanical system [MEMs] vs. solid-state piezoelectric), software or settings used. For example, agreement may be lower among people who lie still while awake without electrophysiologic sleep, such as those with insomnia (8,9), and those whose daily activity may be limited due to illness, treatment (10,11) or other factors that may limit motor activity (e.g., cardiac or pulmonary conditions). Although ACT is widely used to measure sleep in a variety of clinical settings and among adults with chronic conditions, there have been notably few attempts to validate its use in these patients or to synthesize existing data to assess rates of agreement with PSG. Therefore, we conducted a systematic review of the literature to (1) evaluate the agreement between wrist actigraph measured sleep characteristics (TST, SE, SOL, WASO) and PSG measured sleep characteristics and (2) to compare the degree of agreement between studies of adults without chronic conditions and sleep complaints (healthy) and adults with chronic comorbid conditions.

METHODS

Review Design

We conducted a systematic review and meta-analysis using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (12).

Inclusion and Exclusion Criteria

We included both observational studies and randomized control trials if they reported sleep characteristics for both ACT and PSG at baseline. Prior to beginning this review, we searched PROSPERO for ongoing or recently completed systematic reviews.

Inclusion and exclusion criteria are shown in Supplementary Table 1. We limited the studies to those with participants who were ≥ 18 years of age because of developmental differences in sleep characteristics and sleep disturbance between adults and children (13). We included studies with participants expected to have normal sleep, as well as those with suspected sleep disturbance for a variety of reasons, such as sleep disorders, advanced age, menopause, comorbid medical conditions (e.g., cancer, cardiovascular disease, neurological disorders, diabetes, obesity), psychiatric conditions (e.g., depression, schizophrenia, bipolar disorder, substance abuse), disability (e.g., motor disabilities, total blindness), or work conditions (e.g., rotating or shift work or jet lag associated with travel across time zones).

We included only studies that used research accelerometers and excluded studies that reported only activity-tracking devices directly available to consumers because the software, technical specifications, and validity data are often proprietary and not publicly available. In addition, the U.S. Food and Drug Administration has not approved most of these devices for clinical sleep research and treatment (14). We also excluded studies that studied physiological phenomena during zero gravity conditions.

Literature Search

We searched multiple databases, including PubMed, OVIDEMBASE, OVIDMEDLINE, OVIDPsycINFO, CENTRAL, and CINAHL. A health sciences librarian with expertise in systematic reviews created the search strategies, conducted the searches on June 21, 2017, and then searched for updated results on March 14, 2019. The final OVIDMEDLINE search strategy is shown in Supplementary Table 2. We adapted the syntax and subject headings from this search for use in other databases. We also searched the International Clinical Trials Registry Platform Search Portal and ClinicalTrials.gov to identify ongoing or recently completed trials. We searched grey literature from the Grey Literature Report, OpenDOAR, and OpenGrey. As relevant studies were identified, reviewers checked for additional relevant cited articles.

Identification and Selection of Studies

We imported the records into Covidence, an electronic systematic review management system, to screen the reports based on the inclusion and exclusion criteria. Three independent reviewers screened all abstracts and, if necessary, discussed disagreements until consensus was reached. Full-texts were screened by two reviewers and disagreements were reviewed and solved by the third reviewer (Supplementary Table 1).

Data Evaluation

We evaluated the quality of studies for risk of bias with Quality Assessment in Diagnostic Accuracy Studies (QUADAS-2), a tool for assessment of diagnostic test accuracy (15). While not all of the included studies were diagnostic test accuracy studies, the four domains used for risk and applicability assessment in QUADAS-2 are relevant to our purpose: 1) patient selection, 2) index text (actigraphy), 3) reference standard (PSG/EEG), and 4) flow and timing (e.g., the extent to which all participants received the same reference standard, inclusion of all participants in the analyses, and the simultaneous recording of ACT and PSG).

Data Management and Analysis

One reviewer extracted the data into an Excel spreadsheet, and a second reviewer checked it for accuracy. Data extracted included study authors, year of publication, sample characteristics, study setting, PSG settings, actigraph device, scoring method, ACT and PSG means and standard deviations for TST, SOL, WASO, and SE, and epoch by epoch sensitivity, specificity, and accuracy. For the meta-analysis, we used Review Manager 5.3 from Cochrane Reviews (16) to create forest plots with random effects (due to the heterogeneous nature of the studies). We also computed the I², a meta-analysis statistic that indicates the heterogeneity of the data, with values greater than 75% indicating substantial heterogeneity (17). Data were also reported in narrative form when insufficient for meta-analysis.

RESULTS

Our search resulted in 1364 records after the removal of duplicates. Screening of titles and abstracts resulted in the exclusion of 1112 records. We screened 252 full-text records and included 96 studies reported in 98 articles. See Figure 1 for a flow of the records through the screening process.

Figure 1.

PRISMA diagram: Flow of studies through inclusion and exclusion

Study Populations

A summary of each study sample description, mean or median age, and gender distribution is provided in Supplementary Table 3. There were 4,134 participants included in all the studies [age M = 43.4 (SD 16.7) years, 45.1% female]. The studies in this review included participants without chronic conditions or sleep disorders, those with sleep disorders, and others with a variety of chronic medical and mental health conditions including low back pain (18), dementia (10), bipolar disorder (19,34), schizophrenia (19), atopic dermatitis (20), depression (22,64), chronic fatigue (23), type 1 diabetes (25), heart failure (31), chronic obstructive pulmonary disease (33), Creutzfeldt-Jakob disease (39), Parkinson’s disease (43,48), Huntington’s disease gene carriers (45), depression and insomnia (47), and fibromyalgia (51) (see Table 1 for study descriptions of those with and without chronic conditions).

Table 1.

Characteristics of Studies comparing Actigraphy and Polysomnographic Measures of Sleep in Healthy Participants and with Chronic Conditions

Authors, reference number	Sample	Age M (SD)	Gender N (%)	Setting	Actigraph Characteristics (type, scoring threshold)
Alsaadi et al. 2014 (18)	N = 50 Low back pain	42.7 (15.15)	24 (48%) female	1-night research sleep lab	Actiwatch-2 (Phillips Respironics), medium threshold
Ancoli-Israel et al. 1997 (10)	N = 10 Nursing home, wheel chair bound, severe dementia	86.4 (6.0) 74–96 years	(82%) female	1-night in own bed in nursing home	Actillume recorder (Ambulatory Monitoring, Inc., Ardsley, NY)
Baandrup et al. 2015 (19)	N = 42 Schizophrenia and bipolar disorder	86.4 (9.5)	17 (40.5%) female	1-night at home unattended	Actiwatch Spectrum (Philips Respironics, Murrysville, PA, USA), medium threshold
Bender et al. 2008 (20)	N = 20 Mild to moderate atopic dermatitis	18/20 < 50 years	13 (65%) female	2-nights sleep lab	Actigraph Motionlogger (Ambulatory Monitoring Inc, Ardsley, NY)
Chakar et al. 2017 (21)	N = 38 Healthy	23.5 (15)	Not reported	1-night in sleep lab	Actiwatch-2 (Philips Respironics, Murrysville, PA, USA), default setting,
Cook et al. 2017 (22)	N = 21 Major depressive disorder	26.5 (4.6)	17 (90%) female	1-night in lab	Actiwatch-2
Creti et al. 2010 (23)	N = 49 Chronic fatigue	42.78 (1173), range 16–73	41 (83.67%) female	1-night in lab	Actitrac (IM Systems Co, Baltimore, MD, USA)
de Souza et al. 2003 (24)	N = 21 Healthy	Range: 18–33 years	14 (66.67%) female	1-night after adaption night in sleep lab	Basic 32 C Mini Motionlogger Actigraph - (Ambulatory Monitoring, Inc., Ardsley, USA), zero crossing mode
Farabi et al. 2017 (25)	N = 27 18 to 30-year old with Type 1 Diabetes	23.8 (4.1)	16 (59.26%) female	1-night in sleep lab	Actiwatch-2 (Philips Respironics), medium threshold
Fonseca et al .2017 (26)	N = 35 Healthy	52.0 (6.9)	20 (57.1%)	2-nights in hotel	Actiwatch Spectrum (Philips Respironics Inc., Murrysville, PA, USA), medium threshold
Fuller et al. 2017 (27)	N = 21 Male elite athletes	22.5 (2.7)	21 (100%) male	4-nights in home	Actical® Z series activity monitors (Philips Respironics, Oregon, USA), medium threshold
Jean-Louis et al. 2001 (28)	N = 46 Healthy postmenopausal women and healthy young adults	55.28 (6.98) Range: 19–77	42 (91.3%) female	24-hours in home	The Actillume manufacturer (Ambulatory Monitoring, Ardsley, NY)
Jean-Louis et al. 2001a (29)	N = 5 Healthy young adults	25 (6)	Not reported	5-nights in Circadian Pacemaker Laboratory at UCSD	The Mini Motionlogger (Ambulatory Monitoring, Inc. Ardsley, NY), Zero-Crossing Mode
Jean-Louis et al. 1996 (30)	N = 20 Healthy	29.95 (8.98) Range 21 to 53 years	9 (45%) female	1-night in lab	Gaehwiler Electronic (no. CH-8634; Hombrechtikon, Switzerland), analyzed using the Actigraph Data Analysis Software.
Jeon et al. 2019 (31)	N = 155 Heart failure	60.5 (16.1)	54 (34.8%) female	1-night at home	Actiwatch-64 (Philips Respironics), medium threshold
Jumabhoy et al. 2017 (32)	N=22 Healthy	29.3 (114)	Not reported	1-night in laboratory	Actiwatch Spectrum Pro
Kapella et al. 2017 (33)	N=50 Chronic Obstructive Pulmonary Disease	63.2 (8.4)	15 (30.0%)	1-night in lab	Actiwatch 2
Kaplan et al. 2012 (34)	N = 54 Bipolar and healthy controls	35.6 (11.65)	69 (77.8%) female	2-nights in lab	Actiwatch-64 (Mini Mitter, Philips Respironics Inc., Bend, OR, USA), medium threshold
Kogure et al. 2011 (35)	N = 6 Healthy participants	32.2 (3.9) Range: 30–40	4 (66.6%) female	2-nights in home	Micro-Mini Actigraph (Ambulatory Monitoring Inc.) Cole-Kripke algorithm
Kosmadopoulos et al .2014 (36)	N = 22 Young adults	23.9 (3.8)	4 (18.8%) female	2-night in sleep lab	Actiwatch-64 (Mini-Mitter Philips Respironics, Bend, OR), medium threshold
Kripke et al. 1978 (37)	N = 5 Normal subjects	Not reported	Not reported	Number of nights not reported conducted at home	Researchers made own device
Kuo et al 2017 (38)	N=81 Healthy	28.4 (5.8)	34 (42.0%) female	1-night in lab	Device developed by the authors
Landolt et al. 2006 (39)	N = 7 Creutzfeldt-Jakob disease	64.2 (9.25)	2 (28.6%) female	1-night in lab	Actiwatch (Cambridge Technology, UK)
Latshang et al. 2016 (40)	N = 51 Healthy men	Median age 24 (quartiles 20–28 years)	51 (100%) male	1-night in lab or hostel	MSR2005; Electronics GmbH, Henggart, Switzerland
Leproult et al. 2015 (41)	N = 16 Healthy non-obese	Median 25 (IOR 23, 27.8)	13 (81.3%) female	1-night at home	Actiwatch-2 (Medys, Kappellen, Belgium)
Mack et al. 2009 (42)	N = 20 Healthy adults	33.6 (12.6)	10 (50%) females	1-night in sleep lab	Not reported
Maglione et al. 2013 (43)	N = 61 Mild to moderate Parkinson’s disease	67.74 (8.88)	22 (36.7%) female	1-night in sleep lab	Actiwatch-L (Philips Respironics Andover, Ma)
Mantua et al. 2016 (44)	N = 40 Healthy young adults	22.37 (4.92) Range: 18–30	19 (47.5%) female	1-night at home	Actiwatch Spectrum (Philips Respironics, Bend, OR, USA)
Maskevich et al. 2017 (45)	N = 7 Huntington’s Disease Gene Carriers (late presymptomatic or early disease)	54.1 (6.4)	6 (85.7%) female	1-night at lab	Actiwatch Spectrum Pro
Markwald et al. 2016 (46)	N = 29 Healthy adults	24.0 (5.3)	8 (27.6%) females	1-night in sleep lab	The Actiwatch-64 actigraph (Philips Respironics, Bend, OR, USA), medium threshold
McCall & McCall 2012 (47)	N = 60 People with depression and insomnia	41.5 (12.5)	40 (66.7) female	1-night in lab	Actiwatch-64 (MiniMitter) medium threshold
Memon et al. 2017 (48)	N=25 Parkinson Disease	63.6 (8.4)	6 (24%) female	1 to 3 nights	Micro Motionlogger Sleepwatch; Ambulatory Monitoring Inc. NY USA)
Mikkelsen et al. 2018 (49)	N=15 Healthy	35.0 (14.3)	9 (60%) female	1-night in lab	Actiwatch MW8
Montgomery-Downs et al. 2012 (50)	N = 24 Healthy adults	26.1 Range: 19–41	9 (40%) female	1-night in sleep lab	Actiwatch-64 (Philips Respironics), medium threshold
Mundt et al. 2016 (51)	N = 113 Insomnia and fibromyalgia	52.68 (10.91)	110 (97.3%) female	1-night in home	Actiwatch-2 (Phillips Respironics, Bend, OR), high threshold
Nussbaumer-Ochsner et al. 2011 (52)	N = 16 Healthy mountaineers who had experienced high altitude pulmonary edema	Median 47 (quartile range 40 to 49)	2 (12.5%) female	1-night location not specified	Actigraph (MSR, Henggart, Switzerland),
Paquet et al 2007 (54)	N = 15 Healthy	39.3 (15.1) Range: 20–60	8 (53.33%) female	3-nights in lab	Actiwatch-L (Mini Mitter, Respironics Inc, Bend, OR), medium threshold
Peterson et al. 2012 (55)	N = 11 Healthy	Range: 24–55	7 (63.64%) female	1-night in lab	Actiwatch Spectrum (Philips Home Health Care Solutions, Bend, OR, USA)
Pigeon et al. 2018 (56)	N = 27 Healthy	28.4 (116)	11 (40.7%) female	1-night in lab	Actiwatch-2
Pollak et al. 2001 (57)	N = 14 Healthy young adults and older adults	Range: 21–72	7 (50%) female	7 days and nights Chronobiology Laboratory	CSA (Model 7164 Activity Monitor, Computer Science and Applications, Inc., Shalimar, Florida) and the IM (ActiTrac, IM Systems, Inc., Baltimore, Maryland), Simple threshold
Sargent et al. 2016 (59)	N = 16 Endurance cyclists	19.3 (15)	16 (100%) male	9-nights in private bedroom at training camp	Actigraph not reported, medium threshold
Shambroom et al. 2016 (60)	N = 26 Healthy adults	38 (13) Range 19 to 60	13 (50%) female	2-nights in lab	Actiwatch-64 (Mini Mitter Philips Respironics, Bend, OR, USA), Medium threshold
Slater et al. 2015 (61)	N = 108 Health adults	22.7 (0.2)	51 (47.22%) female	1-night in lab	GT3X+ activity monitors (Actigraph, FL, USA)
Tonetti et al. 2008 (62)	N = 12 Healthy	22.97 (2.62)	6 (50%) female	7-nights in lab	Actiwatchw (AW; Cambridge Neurotechnology Ltd, Cambridge, UK; Mini Mitter, Respironics Inc., Bend, Oregon, USA), Medium threshold
Wichniak et al. 2018 (63)	N = 30 Healthy	24.2 (4.9)	15 (50%) female	3-nights in laboratory	Motionwatch 8 (CamNTech)

Sleep Study Settings

See Supplementary Table 3 for a summary of the study settings. Most of the sleep measurements occurred in a sleep laboratory (n = 61 studies); 18 studies took place in participants’ homes; 2 studies were conducted in an intensive care unit; and 2 in hotels. Five reports did not specify the setting. Five studies had mixed settings (i.e., home and laboratory, laboratory and hostel, and home and hospital). One study was conducted in a hospital ward, one in a nursing home, and one in a private room at a training camp. The majority were conducted on a single night [n = 67 (69.8%)].

Actigraph Types and Settings

Researchers used several types of actigraphs. The most commonly used included Actiwatch-2 (Phillips Respironics, Inc.) (n = 12 studies), Actiwatch-64 (Phillips Respironics, Inc.) (n = 12 studies), the Actiwatch Spectrum (Phillips Respironics) (n =7 studies), Actiwatch-L (Philips Respironics, Andover, MA) (n = 6), Motionlogger (Ambulatory Monitoring Inc., Ardsley, NY) (n = 4 studies), Actiwatch (Cambridge Neurotechnology) (n = 4 studies), and Actillume (Ambulatory Monitoring Inc, Ardsley, NY) (n = 4 studies). In 4 studies the researchers developed their own actigraphs. Ten studies did not report the type of actigraph used. Forty-one studies used 45 other actigraph models. See Supplementary Table 3 for a full list of the actigraphs and scoring thresholds used in each study.

Risk of Bias of Included Studies

Overall the risk of bias was low. The largest potential risk of bias was in the criteria for participant selection and was explained by limitations in reporting methods of recruiting and identifying participants. Twenty-eight studies had a low risk of bias in patient selection; 60 had unclear risk of bias; and 10 had high risk of bias. The next most common area for potential bias was flow and timing, for which 70 studies had low risk of bias; 17 had unclear risk for bias; and six had high risk of bias (Supplementary Table 4)

Total Sample

Sixty-four studies included mean and standard deviations for one or more of the sleep variables (TST, SOL, SE, or WASO) (See Supplementary Figure 1.1 to 1.4). These studies included adults without chronic conditions and sleep complaints and adults who had a variety of sleep and chronic conditions (e.g., insomnia, sleep disordered breathing, periodic limb movements, type 1 diabetes, depression). ACT overestimated TST by M = 17.88 minutes [(95% CI 7.90, 27.85) n = 3437 participants] and overestimated SE by M = 3.77% [(95% CI 1.87, 5.59) n = 2905 participants], compared to PSG. ACT underestimated SOL by M = −6.94 minutes [(95% CI – 9.63, - 4.24) n = 2534 participants] and underestimated WASO by M = −12.87 minutes [(CI 95% − 19.30, − 6.43) n = 2537 participants.]

Forty-nine studies reported sensitivity, specificity or accuracy using an epoch by epoch method (n = 1,582 participants). Sensitivity is the percentage of epochs scored as sleep by ACT compared to the epochs scored as sleep by PSG. Specificity is the percentage of epochs scored as wake compared to the epochs scored as wake by PSG. Accuracy is the percentage of epochs scores as sleep or wake compared to the epochs scored as sleep or wake by PSG (see Supplementary Table 4). The mean sensitivity for all studies was M = .83 (SD .120). The mean specificity was M = .51 (SD = .19), and mean accuracy was M =.82 (SD .07).

Sixty-four studies reported correlations (interclass correlations, Person’s correlations, or Spearman’s correlations) between ACT and PSG variables. The range of the correlations varied considerably for TST (.115 to .98), SOL (−07 to .90), SE (−.18 to .90) and WASO (.00 to .85). See Supplementary Table 3 for the studies that reported correlations.

Healthy Adults

Twenty-nine studies (n = 762; 18.4% of the sample) included healthy adults which was defined as adults without chronic conditions or sleep complaints and excluded people with medical, mental health conditions, and sleep complaints. The mean age was 28.0 (SD 8.0) years (range 18 to 77 years; 40.0% female). Compared to PSG, there was a non-statistically significant trend for ACT to overestimate TST [M = 11.23 minutes (CI 95%: −3.98, 26.44 minutes) n = 404]; and SE [M = 1.90% higher (CI 95%: −0.65%, 4.44%) n = 328 participants]; and underestimate WASO [M = −3.11 minutes lower (CI 95%: −10.88 minutes, 4.65 minutes) n = 297 participants] . ACT significantly underestimated SOL [M = −8.09 minutes (CI 95%: −12.46, −3.71 minutes) n = 419 participants]. See Supplementary Figures 2.1 to 2.4 for the forest plots for the data obtained from this group. These studies were very heterogeneous, as indicated by the I^2.ranging from .80–87.

Eighteen studies that healthy adults reported sensitivity, specificity or accuracy. The sensitivity M = .89 (SD .216), n = 461. The specificity M = .53 (SD .20), n = 461); and the accuracy M = .88 (SD .04), n = 437. See Table 2 for sensitivity, specificity, and accuracy for healthy adults and chronic conditions.

Table 2.

Sensitivity, Specificity, and Accuracy Actigraphy and Polysomnographic Measures of Sleep in Healthy Participants and with Chronic Conditions

Authors, reference number	Sample size	Sample Description	Sensitivity	Specificity	Accuracy
Healthy
Chakar et al. 2017 (21)	38	Healthy	.96	.48
De Sousa et al. 2003 (24)	21	Healthy	.991 .972	.341 .442
Fonseca et al. 2017 (26)	49	Healthy	.455	.971	.918
Jean-Louis et al. 2001 (28)	31	Healthy	.948	.313	.921
Jean-Louis et al. 2001a (29)	5	Healthy	.987	. 277	.95
Jumabhoy et al. 2017 (32)	22	Healthy	.942–.992 among the different devices	.188–.356 among the different devices
Kogure et al. 2011 (35)	6	Healthy	.985	.342	.931
Kosmadopoulos et al. 2014 (36)	22	Young adults	.958	.377	.877
Kuo et al. 2017 (38)	59	Healthy	.952	.713	.921
Markwald et al. 2016 (46)	26	Healthy	.967	.370	.893
Montgomery-Downs et al. 2012 (50)	24	Healthy	.957	.389
O’Hare et al. 2015 (53)	20	No sleep- disordered breathing or primary insomnia	.973	.339	.855
Paquet et al. 2007(54)	15	Healthy	.953	.543	.907
Pigeon et al. 2018 (56)	20	Healthy			.887
Sargent et al. 2016(59)	16	Endurance cyclists	.875	.771	.809
Shambroom et al. 2012 (60)	26	Healthy			.876
Slater et al. 2015 (61)	108	Healthy	.897	.456	.841
Webster et al. 1982 (68)	14	Healthy			.930
Chronic Conditions
Alsaadi et al. 2014 (18)	33	Low back pain	.90	.67
Ancoli-Israel et al. 1997 (10)	10	Dementia	.87	.90
Farabi et al 2017 (25)	27	18 to 30-year old with Type 1 Diabetes			.81
Maskevich et al. 2017 (45)	7	Huntington’s Disease Gene Carriers	.97	.31	.80

Sensitivity: percentage of epochs scored as sleep by actigraph

Specificity: percentage of epochs scored as wake by actigraph

Accuracy: percentage of epochs scored as sleep or wake

1: Cole & Kripke algorithm; 2: Sadeh’s algorithm

Twenty studies of healthy adults reported correlations between sleep variables measured with ACT compared with PSG (see Table 3 for correlations for healthy and chronic conditions samples). The correlations for the TST measures ranged from .19 to .98; SOL correlations varied from −.07 to .80, while the range of correlations for SE was −.18 to .89, and the range for WASO was −.69 to .82.

Table 3.

Correlations between Actigraphy and Polysomnographic Measures of Sleep in Healthy Participants and with Chronic Conditions

Authors, reference number	Sample	TST	SOL	SE	WASO
Healthy
Chakar et al. 2017 (21)	Healthy	.88**	.01**		.77*
de Souza et al. 2003 (24)	Healthy		.69a .64b	.39a .41b
Fuller et al. 2017 (27)	Male elite athletes	83***	.24**	.35**	.33**
Jean-Louis et al. 2001 (28)	Healthy	.98
Jean-Louis et al. 2001a (29)	Healthy	.81		.55
Jean-Louis et al. 1996 (30)	Healthy	97***
Kaplan et al. 2012 (34)	Healthy	.91**	.41**	51***	.35**
Kogure et al. 2011 (35)	Healthy	95***	.69***	.89***	82***
Kosmadopoulos et al. 2014 (36)	Young adults	−. 69**			−.69**
Kripke et al. 1978 (37)	Normal subjects	.95**
Kuo et al. 2017 (38)	Healthy	.93	.53	.84	.75
Latshang et al. 2016 (40)	Healthy	.19	.16	.27*
Leproult et al. 2015 (41)	Healthy	.95***
Mantua et al. 2016 (44)	Healthy	94**		.35**
Mikkelsen et al. 2019 (49)	Healthy	.88	.79	.52	.57
Nussbaumer-Ochsner et al. 2011 (52)	Healthy	92**	.13*	.82
Sargent et al. 2016 (59)	Endurance cyclists	.251**		.306***	.258**
Shambroom et al. 2016 (60)	Healthy	.60	−.07	.36	.21
Slater et al. 2015 (61)	Healthy	.51	.32	.59	.68
Chronic Conditions
Alsaadi et al. 2014 (18)	Low back pain	.80	.33	.55	.52
Ancoli-Israel et al. 1997 (10)	Dementia	0.91***
Baandrup et al. 2015 (19)	Schizophrenia and bipolar	.78	.00	.00	.00
Bender et al. 2008 (20)	Atopic dermatitis		.606**	.442*
Creti et al. 2010 (23)	Chronic fatigue	792***	−.036	.673***	.586***
Farabi et al. 2017 (25)	Type 1 diabetes	.66	.38	.57	.48
Jeon et al. 2019 (31)	Heart failure	.62	.13	.34	.28
Kaplan et al. 2012 (34)	Bipolar	.92**	.33**	.49**	.59***
Maglione et al 2013 (43)	Parkinson’s disease	.496***		.383**	.400**
Mundt et al. 2016 (56)	Insomnia and fibromyalgia	.60**	.08*	.46**	.49**

Note. CID=chronic insomnia disorder, SDB=sleep-disordered breathing, SE=sleep efficiency, SOL=sleep onset latency, TST=total sleep time, WASO=wake after sleep onset

^a: Cole’s algorithm,

^b: Sadeh’s algorithm

^*p ≥. 05.

^**p < .05.

^***p < .001, if no * study did not report p-value

Chronic Conditions

Fourteen studies included a total of 724 (17.5%) participants [M age = 50.9 (SD 19.5), 62.6% female] who had chronic conditions, including low back pain (18), dementia (10), bipolar disorder (19,34), schizophrenia (19), atopic dermatitis (20), depression (22,64), chronic fatigue (23), type 1 diabetes (25), heart failure (31), chronic obstructive pulmonary disease (33), Creutzfeldt-Jakob disease (39), Parkinson’s disease (43,48), Huntington’s disease gene carriers (45), depression and insomnia (47), and fibromyalgia (51). Among these participants, ACT overestimated TST, compared to PSG [M = 22.42 minutes (CI 95%: 11.92, 32.91 minutes) n = 648 participants] and SE [M = 5.21% (CI 95%: 1.41% to 9.00%) n = 622 participants], while ACT underestimated SOL [M = −7.70 minutes (CI 95%: −15.22, −0.18 minutes) n = 484 participants], and these were statistically significant. There was a non-significant trend that suggested a difference in WASO [M = −10.90 minutes (CI 95%: −26.01, 4.22 minutes) n = 560 participants]. (See Supplementary Figure 3.1 to 3.4 for the forest plots for the studies that included participants with chronic conditions.) The reports of total sleep time were less heterogeneous (I² = 34%), while reports of the other sleep characteristics were highly heterogeneous (I² ranged from 83% to 90%).

Only four studies of people with chronic conditions reported sensitivity [M = .90 (SD .05], n = 50] and specificity [M = .67 (SD .15), n = 50] with epoch by epoch agreement of ACT and PSG. Only two studies reported accuracy (.80 and .81) ,(n = 34) (25,45) (Table 2). Ten studies that included participants with chronic conditions reported correlations between ACT and PSG variables for TST (.496 –.92), SOL (-.036–.38), SE (.00–67) and WASO (.00–.59) (Table 3).

DISCUSSION

This systematic review advances methodological research comparing ACT and PSG by comparing the level of agreement between ACT and PSG among people with chronic conditions. Our findings demonstrate that ACT consistently over-estimates sleep, as indicated by TST and SE, and under-estimates wake, as indicated by SOL and WASO among people with chronic conditions, compared to these measures in healthy adults. Although past studies suggest that ACT overestimates TST and SE and underestimates SOL and WASO in adults (5), these differences were much larger and statistically significant in people with chronic conditions. While the differences for participants with chronic conditions were generally within the range of clinically acceptable difference noted in the recent American Academy of Sleep Medicine (AASM) Review (5), the ranges from the AASM review were estimated for specific sleep disorders. The “clinical” importance of variations in sleep characteristics may vary depending on the sleep-related outcome of interest in chronic conditions (e.g., metabolic control, cardiovascular outcomes, sleep-related symptoms).

Chronic medical and psychiatric conditions are frequently associated with sleep disturbance (i.e., short or long sleep, fragmented sleep, altered sleep timing and perceptions of poor sleep/insomnia), although these characteristics may or may not represent primary sleep disorders. Causes of sleep disturbance among people with medical and psychiatric chronic conditions are multi-factorial and may include primary sleep disorders (e.g., insomnia, sleep-disordered breathing, or restless legs syndrome), as well as treatments (e.g., medications, devices), comorbidity, emotional distress (e.g., anxiety, depression), and the clinical environment, among others. Some medical or psychiatric conditions may also be associated with changes in motor activity that may influence activity counts detected by the actigraph and thereby contribute to agreement or discrepancy in obtained sleep measurements (31). For example, patients with cardiac or pulmonary conditions may have limited mobility, and people with depression may have psychomotor slowing. Jeon and colleagues (27) found smaller discrepancies between ACT and PSG among heart failure patients under the age of 60 than older patients, and smaller discrepancies were associated with more advanced age, higher levels of daytime motor activity and more severe insomnia and self-reported sleep disturbance in older but not younger participants. Aside from this study, few studies have considered the potential contributions of clinical and demographic characteristics or motor activity levels to ACT measures. It may be helpful to adapt electronic sleep scoring algorithms to address these factors. However, given the large number of populations and studies of adults with chronic conditions, this may be a costly and time-consuming process that could benefit from automated algorithms.

An important consideration when studying sleep in people with chronic conditions is the trajectory of the chronic conditions and related changes in treatment and environment that may influence sleep and may thereby also influence the agreement, disagreement, or accuracy of ACT compared to PSG. For example, sleep may be more fragmented during physiological exacerbations of chronic conditions than during more stable phases. Changes in motor activity secondary to changes in health may influence the consistency of the agreement between ACT and PSG over time, but these factors have seldom been evaluated. Studies that include only single nights of recording may also lead to greater bias associated with the first night effect due to the intrusive nature of PSG measurement (47). Given theseobservations and the proliferation of studies using actigraphs, the limited number of studies comparing ACT and PSG measures among people with chronic conditions is somewhat surprising. For example, ACT has been used extensively in oncology research (66), but we found no studies comparing ACT and PSG in oncology patients, and it is unclear that our findings can be generalized to groups of participants with varied conditions or treatments.

Heterogeneity of study results was very high as illustrated by the I² statistic, which indicates a lack of lack of consistency across studies with respect to over- or under-estimation of sleep. There was no discernible trend with respect to population tested, device used, or data collection method employed that could explain this lack of consistency. However, these variations should be considered in future work.

Consistent with previous reviews, studies in this review generally suggest that ACT was very sensitive to sleep but was not as specific in determining wake. This characteristic may limit the usefulness of wrist actigraphy as a measure of sleep or napping during the day, and daytime napping may be more common among people with chronic disorders. However, we did address sleep during the day, and the small number of studies that reported sensitivity and specificity, especially among people with chronic conditions, is a limitation.

We found a wide range of correlations reported in both the healthy and chronic conditions subgroup. Correlations between sleep variables that were at or near zero suggest that the relationship between ACT and PSG may suggest that there are non-linear relationships between ACT and PSG, thus looking only at correlations in a validation study may be insufficient.

Strengths and Limitations

This study has several strengths, including the use of a standardized meta-analysis and systematic review approach; a systematic search strategy designed by a trained university research librarian, and the involvement of multiple reviewers. We were able to combine the results of many, but not all studies using quantitative methods. Limitations of the review include the possibility that we did not identify all relevant literature and restriction to English-language articles. Although studies reviewed were generally of high quality, the largest risk of bias was due to incomplete reporting of recruitment of the study sample and study sample characteristics.

Although this review was among the first to evaluate the use of ACT among people with chronic conditions, there was wide variability in the nature of the chronic medical and psychiatric conditions, the ages and settings of the study participants, and the types of actigraphs and scoring algorithms used. This variability likely influenced the finding of statistical heterogeneity. Future studies are needed for specific populations with chronic medical and psychiatric conditions to more fully evaluate the levels of agreement or disagreement between ACT and PSG.

Although consumer wearable devices are increasingly used to estimate sleep, our findings are limited to research actigraphs and are not generalizable to these devices that vary widely in physical properties and scoring methods (which are proprietary and may change without the user being alerted). Mantua et al. (2016) (44) conducted a validation study of 4 different consumer wearable devices and found that TST was valid in 3 of the devices. A recent review about consumer wearable devices for the measurement of sleep found that the majority of validation studies were conducted in healthy adults, and similar to our findings, consumer wearables tend to overestimate sleep duration and that they perform more poorly in those with sleep disorders. (67) However, validation research is limited in adults with chronic conditions. As more detailed information becomes available for adults with chronic conditions, these devices should be a focus of a future systematic review.

Conclusions

Based on the results of this study, the direction of the agreements or discrepancy between ACT and PSG is similar, but the magnitude of the difference is somewhat larger in adults with chronic conditions compared to healthy adults. Future research is needed to better understand these differences in more diverse chronic conditions and to address specific factors (e.g., demographic and clinical factors) that may impact these differences.

Practice Points.

1. ACT tends to overestimate total sleep time and sleep efficiency and to underestimate sleep onset latency and wake after sleep onset in adults;

2. These differences were consistently larger in people with chronic conditions;

3. The differences identified in this review were within the specified “clinically” acceptable range for adult sleep disorders identified in the recent American Academy of Sleep Medicine review.

Research Agenda.

1. There were limited studies comparing ACT and PSG in adults with chronic conditions;

2. Causes of the agreement or discordance between ACT and PSG in people with chronic conditions remain unknown and are an area for future research;

3. Age, chronic sleep disorders, comorbidity, symptoms, decreased motor activity, disability, and other concerns related to chronic conditions may bias ACT vs. PSG measures and should be considered in future studies.

Glossary:

ACT

actigraphy

PSG

polysomnography

TST

total sleep time

SOL

sleep onset latency

SE

sleep efficiency

WASO

wake after sleep onset