A comparison of agreement between actigraphy and polysomnography for assessing sleep during posttraumatic amnesia

Abstract

Study Objectives:

Sleep disturbance often emerges in the early recovery phase following a moderate to severe traumatic brain injury, known as posttraumatic amnesia. Actigraphy is commonly employed to assess sleep, as it is assumed that patients in posttraumatic amnesia (who display confusion, restlessness, and agitation) would better tolerate this measure over gold-standard polysomnography (PSG). This study evaluated the agreement between PSG and actigraphy for determining (sleep/wake time, sleep efficiency, sleep latency, and awakenings) in patients experiencing posttraumatic amnesia. It also compared the epoch-by-epoch sensitivity, specificity, and accuracy between the Actigraph device’s 4 wake threshold settings (low, medium, high, and automatic) to PSG.

Methods:

The sample consisted of 24 inpatients recruited from a traumatic brain injury inpatient rehabilitation unit. Ambulatory PSG was recorded overnight at bedside and a Philips Actiwatch was secured to each patient’s wrist for the same period.

Results:

There were poor correlations between PSG and actigraphy for all parameters (Lin’s concordance correlation coefficient = < 0.80). The low threshold displayed the highest correlation with PSG for wake and sleep time, albeit still low. Actigraphy displayed low specificity (ranging from 17.1% to 36.6%). There appears to be a greater disparity between actigraphy and PSG for patients with increased wake time.

Conclusions:

Actigraphy, while convenient, demonstrated poorer performance in determining sleep-wake parameters in patients with significantly disturbed sleep. Ambulatory PSG can provide a clearer understanding of the extent of sleep disturbances in these patients with reduced mobility during early rehabilitation. Study findings can help design future protocols of sleep assessment during posttraumatic amnesia and optimize treatment.

Citation:

Fedele B, McKenzie D, Williams G, Giles R, Olver J. A comparison of agreement between actigraphy and polysomnography for assessing sleep during posttraumatic amnesia. J Clin Sleep Med. 2022;18(11):2605–2616.

BRIEF SUMMARY

Current Knowledge/Study Rationale: There is limited research that has evaluated sleep disturbance during posttraumatic amnesia and these studies have mainly employed actigraphy as an indirect sleep assessment method. This is the first study to evaluate the accuracy of actigraphy against the gold-standard polysomnography for determining sleep and wake parameters in this patient population.

Study Impact: There is low agreement between both assessment methods, with actigraphy poorly detecting wake time. Study findings can be used to design future assessment and treatment protocols of sleep disturbance, which currently do not form part of routine hospital practice.

INTRODUCTION

Following a moderate to severe traumatic brain injury (TBI), sleep disturbance commonly emerges during the early posttraumatic amnesia (PTA) recovery stage. PTA is a temporary recovery phase that follows coma (loss of consciousness). During PTA, individuals present with significant sleep disturbances^1–3 (likely onset by the head trauma disrupting brain areas that are associated with sleep regulation),⁴ alongside other common symptoms such as no continuous (anterograde) memory, retrograde memory disturbance, confusion, disorientation, and/or behavioral disturbance (agitation/restlessness).⁵ The few studies that have characterized early posttraumatic sleep disturbances during PTA have reported disturbances to sleep macro-architecture (reduced slow-wave sleep),¹ sleep quality (increased sleep latency, frequent awakenings, increased wake time, poor sleep efficiency),^1,2 rest-activity cycle consolidation,⁶ and endogenous melatonin production (sleep-regulating hormone).¹

The clinical presentation of patients experiencing PTA has significantly influenced the sleep assessment methods employed in this group. Given the continuous daily memory impairments during PTA,⁵ patient self-report sleep measures (eg, questionnaires, sleep diaries, or interviews)⁴ are not feasible. Prior limited research³ involving PTA samples has rather mainly employed actigraphy (an accelerometer-based wrist Actigraph device that indirectly estimates sleep and wake parameters) to assess rest-activity cycles and sleep in place of polysomnography (PSG) (a multiparametric objective sleep study). PSG, however, remains the gold-standard sleep measure⁷ and involves directly monitoring various physiologic signals (eg, brain wave activity, eye and muscle movement) to determine sleep-wake parameters and sleep architecture. A recent pilot study successfully conducted PSG in patients experiencing PTA and reported similar PSG feasibility to actigraphy.¹ Prior to this however, it has been largely assumed that patients in PTA who are typically confused, restless, and agitated would have better tolerance of actigraphy and poorly adhere with the elaborate monitoring requirements of PSG.^6,8 While actigraphy has gained popularity for its convenience, simplicity,⁹ low-cost,⁶ noninvasiveness,⁶ and ability to monitor activity for prolonged periods, it also has some limitations. Actigraphy is unable to directly quantify sleep architecture (which is specifically disturbed during PTA)¹ unlike PSG. Actigraphy relies on physical motion (activity) to estimate sleep and wake parameters, whereby immobility or absence of wrist movement generally signifies sleep,⁸ and movement signifies wakefulness. Its performance may be hindered during early TBI inpatient rehabilitation as patients typically have reduced activity levels.¹⁰ Actigraphy has demonstrated reasonable agreement in healthy adult populations.¹¹ However, actigraphy tends to overestimate sleep indices in certain groups who display greater sleep disruption,^12–14 which is common post TBI⁹ and during PTA.¹

It is currently unknown how closely actigraphy agrees with PSG, specifically for patients within the PTA recovery phase. Within the broader TBI literature, there are inconsistent findings regarding the concordance between both measures. One study found that there was a greater disparity between actigraphy and PSG in hospitalized patients post mild to severe TBI with poorer sleep (eg, increased wake time, lower sleep efficiency, and lower sleep time).⁹ Patients in this study⁹ were a subset of a larger study in which the majority of patients no longer displayed PTA at the time of PSG.¹⁵ Another study reported high sensitivity (sleep detection) and low specificity (wake detection) across a small hospitalized sample that comprised 17 patients with severe orthopedic and/or spinal cord injuries, of which 6 patients also had a moderate to severe TBI.⁷ However, between-group comparisons demonstrated that actigraphy had significantly lower sensitivity in the TBI group compared with the rest of the sample.⁷ Other studies, such as a TBI Veteran study¹⁶ have reported moderate to strong correlations for sleep indices between both measures.

A comparison of actigraphy against PSG is needed in the PTA population for various reasons. Overall, there remains limited research on sleep disturbance during PTA, representing a significant literature gap.³ The degree of sleep disturbance also appears to differ by TBI recovery stage.³ A systematic review reported that patients experiencing PTA displayed an increased incidence of sleep-wake cycle disturbance (79%) compared to patients who had recovered from PTA (36%) (the period after anterograde memory and orientation returns), as well as less consolidated rest-activity cycles (an indirect measure of sleep-wake cycles) and significantly worse sleep efficiency.³ While actigraphy has been mainly employed as a first-line measurement of sleep during PTA, without a comparison to PSG, its limitations for this specific patient group remain unknown. Although PTA is significantly associated with sleep-wake disturbances,³ hospital management of PTA remains generally focused on the evaluation of memory and behavior and does not routinely include the assessment and treatment of sleep disturbance.^7,17 It is therefore necessary to determine whether actigraphy is sufficiently accurate to inform future treatment assessment protocols and influence therapeutic approach and treatment. Early management is crucial, as sleep disturbance can detrimentally affect patient functioning and recovery and even disrupt rehabilitation intervention.⁷ Sleep disturbance may disrupt essential sleep functions (eg, learning, memory consolidation, neurogenesis, neuroplasticity),¹⁸ with research reporting an association between the return of continuous memory and normalization of sleep patterns for patients in PTA.² This may then prolong patients’ emergence from PTA.⁶ Early sleep-wake cycle disturbances have also been associated with poorer function on hospital admission,¹⁹ increased hospital length of stay,^19,20 longer PTA duration,²⁰ and greater disability on discharge.⁶ The various consequences of early sleep disturbances highlight the need for more routine assessment of sleep. While the increased employment of actigraphy is compatible with the clinical features of PTA, it is still unclear whether its use as an alternative, indirect, and proxy assessment of sleep disturbance is comparable to PSG in this group.

Study aims and hypotheses

The primary aim of this study was to evaluate the agreement between PSG and actigraphy for determining sleep and wake parameters (ie, total sleep time, total wake time, sleep efficiency, sleep onset latency, and number of awakenings) in patients experiencing PTA. It also aimed to compare the epoch-by-epoch sensitivity, specificity, and accuracy between the 4 wake threshold settings (low, medium, high, and automatic) specified by the actigraphy device used with PSG. It was hypothesized that there would be low agreement between actigraphy and PSG for determining the sleep-wake parameters. It was also hypothesized that actigraphy would underestimate total wake time and overestimate total sleep time relative to PSG.

METHODS

This study emerged from a broader observational study that reported the feasibility of conducting PSG during PTA. This broader study was originally approved as a pilot study¹ by the Epworth HealthCare Human Research Ethics Committee (Study Number: 55212) and was extended to include a larger sample using the same methodology (approved by the Monash Health HealthCare Human Research Ethics Committee; Study Number: RES-19-0000-156E). Data collected from both these studies were combined for the planned purpose of the current study.

Participants

Participants were recruited from Epworth HealthCare’s specialist inpatient TBI Rehabilitation Unit. The general criterion for patient admission to rehabilitation from acute care was medical stability. The inclusion criteria were (1) patients aged ≥ 18 years, (2) patients who sustained a moderate to severe TBI, and (3) patients experiencing PTA as measured by the Westmead PTA scale (WPTAS).²¹ Participants were excluded if they were extremely agitated or behaviorally disturbed and therefore unable to tolerate the study measures (as advised by the treating rehabilitation physician). Similar to other TBI studies,^6,22 patients with a premorbid documented sleep disorder (eg, insomnia, hypersomnia, sleep apnea, narcolepsy) and/or sleep complaints (eg, difficulties with sleep initiation or sleep maintenance) in their electronic health record were also excluded.

PSG was conducted on 45 inpatients at rehabilitation bedside. There were 6 patients who emerged from PTA during the study assessment period (as determined by the WPTAS) and were, therefore, excluded. Of the remaining 39 patients in PTA, 10 patients had < 6 hours of PSG recording time. This was due to lost lead signals where the channels were not clear enough to score sleep (n = 7) or where the patient removed the device (n = 3). This equates to a PSG adherence rate of 74.4% (29 of 39 patients). An additional 5 patients were also excluded who removed the actigraphy wristwatch device while undergoing PSG. The analytic sample therefore consisted of 24 patients in which PSG (≥ 6 hours) and actigraphy were simultaneously recorded.

Measures

PSG

Ambulatory PSG was conducted overnight at the rehabilitation bedside using the Compumedics Somté (V1 system; Compumedics Ltd, Abbotsford, Australia) device. The lead placement employed in this study has been published elsewhere¹ and was in accordance with guidelines²³ for Type 2 studies (unattended, ambulatory PSG). It is recommended that PSG is conducted for a minimum of 6 hours.²⁴ Senior Sleep Scientists at Epworth HealthCare scored PSG data according to the American Academy of Sleep Medicine Scoring Manual.²⁵ Scorers evaluate the recorded physiological signals and assign a sleep stage every 30 seconds (referred to as epochs). Summary statistics of epoch data then determined sleep-wake parameters (ie, sleep onset latency, sleep time, wake time, sleep efficiency, number of awakenings). Arousals per hour were also calculated.

Actigraphy

Participants wore an accelerometer-based wrist Actigraph device from Philips Respironics (Bend, OR). The actigraphy device calculated activity counts at user-specified time intervals (epochs) using a weighted algorithm that has been validated against PSG.²⁶ Epochs were scored as either “wake” or “sleep” using Philips Actiware software (Version 6.0) by comparing each epoch’s activity count to the software’s preprogrammed wake threshold values: low (value: 20), medium (40), high (80), or automatic (calculated based on the wearer’s mean activity). An epoch with an activity count greater than the threshold value was scored as wake, and, if less than or equal to the wake threshold value, it was scored as sleep. Lower threshold values (which require less movement to detect sleep) generally have higher specificity.¹³ The Actiware software calculated summary statistics of epoch data that determined the same sleep-wake parameters as PSG. The Actigraph models utilized (ie, Actiwatch 2, Actiwatch Spectrum, and Actiwatch Spectrum Plus) have a sampling frequency of 32 Hz and have demonstrated equivalent performance in terms of activity recordings and derived sleep statistics.^27,28

WPTAS

The 12-item WPTAS²¹ is a standardized bedside assessment for monitoring PTA status. The items measured patients’ orientation and anterograde memory, including date of birth, age, time, place, assessor’s face/name, and recall of picture cards. This scale is routinely administered daily until a patient reaches perfect totaled scores (12/12) for 3 consecutive days. The patient has emerged from PTA on the first of these days and PTA duration (days) was calculated from the date of injury. The duration of PTA, which may range from days to months, can be used to classify TBI severity.²⁹

Demographics and injury characteristics

Demographics and injury characteristics were collected from patients’ electronic health records as per Table 1. Trauma severity was graded utilizing the Injury Severity Score³⁰ by a senior physiotherapist. Six body regions (head and neck, face, chest, abdomen, extremities, and external) are scored between 1 (minor) to 6 (maximal/untreatable). The top 3 most severe region scores were squared and totaled to produce the Injury Severity Score. Major trauma is defined as an Injury Severity Score greater than 15.³¹

Table 1.

Sample demographics and clinical characteristics.

	Analytic Sample (n = 24)		Excluded Patients† (n = 15)
	Mean/Percentage	SD	Mean/Percentage	SD
Male	58.3%	–	73.3%	–
Female	41.7%	–	26.7%	–
Age (years)	48.3	22.8	44.7	21.7
GCS (at scene of injury)	6.9	3.9	9.1	4.3
Acute LOS (days)	24.2	12.2	24.1	9.0
Rehabilitation LOS (days)	97.7	70.7	112.9	83.7
PTA duration (days)	59.6	35.7	73.8	48.2
Severe TBI	20.8%	–	6.7%	–
Very Severe TBI	79.2%	–	93.3%	–
Time post injury (days)	38.2	16.8	48.7	36.4
ISS	34.8	13.1	29.0	12.2
Major Trauma*	95.8%	–	100.0%	–
Cause of Injury	MVA: 54.2% PED vs MVA: 25.0% Fall: 8.3% Bicycle vs MVA: 8.3% Workplace: 4.2%	–	MVA: 66.7% PED vs MVA: 13.3% Fall: 13.3% Bicycle vs MVA: 6.7%	–

*Major trauma (ISS > 15).³¹ †Patients not included in the analytic sample (excluding the n = 6 patients not in PTA). GCS = Glasgow Coma Scale, ISS = Injury Severity Score, LOS = length of stay, MVA = motor vehicle accident, PED = pedestrian, PTA = posttraumatic amnesia, SD = standard deviation, TBI = traumatic brain injury.

Procedure

Consecutive admissions to the inpatient TBI Rehabilitation Unit were prospectively screened for inclusion. During PTA, patients do not have the capacity to make reliable decisions due to memory disturbance. Informed written consent was instead obtained by the designated person responsible. The TBI Unit’s Associate Nurse Unit Managers were trained to conduct overnight PSG. The portable Somté PSG device was attached to the patient’s torso with a Velcro strap. A detailed description of the procedures for preparing skin sites for electrode placement has been published previously.¹ The Actigraph devices were preprogrammed through the software to record for the entire duration of the overnight nursing shift. At the commencement of “lights out” on the ward (2100), Nurse Unit Managers connected the Somté PSG device and secured the Actigraph device for the same recording period as PSG. Nurses removed both devices at the “lights on” (0600) period on the ward. The Actiware software automatically generated rest and sleep periods based on its recorded activity data. However, for the purpose of comparison to PSG and to synchronize the timing of both measures, we manually added the rest period (which was the time between the start and end time of PSG recording). The Actiware software then automatically calculated the sleep and wake parameters based on activity levels within this set rest period. There was some variability in the start and end time of PSG recording relative to “lights out” due to the practicalities of a working ward. However, this is not anticipated to affect the study results as comparisons were made between assessment methods (which both had the same start and end times) and not across patients.

Prior research has indicated that Actigraph wrist side placement yielded no difference in measured sleep indices.³² Therefore, the Actigraph device was fastened to the most mobile upper limb (eg, limb with no fractures, paresis, contracture, or spasticity) consistent with current TBI guidelines.³³ The nursing staff monitored the study measures overnight at regular ward rounding intervals.

Statistical analysis

Correlation between PSG and actigraphy for determining sleep and wake parameters

Lin’s concordance correlation coefficient (CCC) was employed^34,35 to compare agreement between the Actigraph device’s 4 wake threshold values (low, medium, high, and automatic) against a gold-standard test (PSG) for determining each of the 5 sleep-wake parameters. Tooth and Ottenbacher³⁶ defines agreement as interchangeability or the ability of 2 measures to generate the same score. Lin’s CCC can range from −1.0 to 1.0, with higher positive values indicating stronger agreement. A CCC < 0.9 is considered poor when comparing a new method to a gold-standard method.³⁷ Bland-Altman plots³⁸ provided a visual interpretation of the agreement between PSG and actigraphy for sleep and wake time. To examine whether agreement changed (proportional bias) or remained constant (fixed bias) across the range of observed values for wake time and sleep time, reduced major axis regression (RMAR)^39,40 was employed. An RMAR slope that is not equal to 1 suggests proportional bias. In the absence of such bias, an RMAR intercept different from 0 suggests fixed bias. Gottlieb et al⁴¹ recently applied both Lin’s CCC and RMAR to sleep data.

Epoch-by-epoch comparison analysis

At regular time intervals (epochs), actigraphy and PSG scorers recorded whether that epoch is classified as “wake” or “sleep.” For PSG only, each sleep epoch was also assigned a sleep stage (eg, N1 [non-rapid eye moment stage 1], N2 [non-rapid eye movement stage 2], N3 [non-rapid eye movement stage 3], or R [rapid-eye moment sleep]). Sleep scientists analyzed the PSG recorded data for every 30-second epoch, and for actigraphy, the user specified the epoch length. The Actigraph devices were initially programmed to store data in 1-minute epochs (for the first 5 patients) and were then prospectively changed to 30-second epochs to correspond with PSG. For these first 5 patients, their PSG recordings were converted from 30-second epochs to 1-minute epochs to qualify for the epoch-to-epoch comparison analysis. Following prior TBI research,⁷ each set of 2 consecutive 30-second epochs both needed to be scored as sleep by PSG to then be recategorized as sleep. If 1 or both 30-second epochs were scored as wake, the 1-minute combined epoch was classified as wake.

Epoch data were used to calculate the sensitivity, specificity, and overall accuracy of actigraphy against PSG. Sleep sensitivity was calculated as total epochs correctly scored as sleep by actigraphy (true positive)/total number of epochs scored as sleep by PSG. Wake specificity was calculated as total epochs correctly scored as wake by actigraphy (true negative)/total number of epochs scored as wake by PSG. Overall accuracy was calculated as the sum of true positive and true negative epochs/total number of epochs. A binomial generalized linear regression model^42,43 was employed to take possible clustering within epochs and within patients into account and compare the above measures between low and medium thresholds. This single comparison was chosen to reduce type 1 errors and because lower threshold settings appear to be more sensitive to wake, which is commonly increased during PTA.¹ All statistical analyses were conducted using Stata 17 (Stata Corporation, College Station, TX).

RESULTS

Sample characteristics

The demographic and clinical characteristics of the analytic sample (n = 24) and excluded patients who were in PTA were relatively similar (Table 1). Independent samples t-tests found no statistically significant differences between both groups in terms of age, Glasgow Coma Scale scores, length of stay, PTA duration, and time post injury. The study measures were conducted at an average of 38.2 days (standard deviation [SD] = 16.8) post injury. Patients experienced PTA for an average of 59.6 days (SD = 35.7), indicating that the severity of TBI was in the very severe (79.2%) to severe TBI (20.8%) range.²⁹

PSG outcomes

PSG was recorded overnight for an average of 477.5 minutes (approximately 8.0 hours). The mean start time (PM) and end time (AM) of PSG recording was: 21:34:29 (SD: 00:55:11) and 5:33:48 (SD: 00:56:21), respectively. These mean times corresponded with the ward’s lights out period. Mean sleep onset latency was 38.6 minutes (SD = 32.8), suggesting that patients had difficulties with initiating sleep (sleep latency of ≤ 15 minutes generally indicates good sleep quality).⁴⁴ Overall, the mean duration of sleep periods was shorter (340.2 minutes, SD = 76.5) than normative values.⁴⁵ Sleep periods were disrupted by frequent awakenings (n = 26.2, SD = 14.0), and patients subsequently displayed longer wake time (84.8 minutes, SD = 60.7) than normative values.⁴⁵ Mean sleep efficiency was therefore reduced (71.6%, SD = 14.5) and below normative values.⁴⁵ Sleep efficiency for the majority of patients (n = 20, 83.3%) was in the poor quality range (≤ 85%).⁴⁴ Mean arousals per hour were 10.4 (SD = 4.3), which were similar to normative values.⁴⁵ Figure 1 displays a patient example of a hypnogram (graphical representation of the PSG recording) and the corresponding actogram (graphical representation of actigraphy recording).

Figure 1. Patient example of hypnogram (PSG) and actogram (actigraphy) for sleep period.

Example of a patient’s hypnogram (a graphical representation of sleep period from PSG) and actogram (a graphical representation of sleep period from actigraphy) for the same time period. (A) Actogram: The black lines indicate activity as measured by actigraphy and the dark blue shaded area indicates sleep time. This patient displayed increased movement (black activity lines) throughout their sleep period. (B) Hypnogram: The colored blocks represent each stage of sleep as indicated on the left, and the white gaps indicate wake time. For this patient, the majority of sleep was spent in N1 and N2 with no N3 (deep sleep). N1 = nonrapid eye movement stage 1, N2 = nonrapid eye movement stage 2, N3 = nonrapid eye movement stage 3, PSG = polysomnography, R = rapid eye movement, W = wake time.

Correlation between PSG and actigraphy for determining sleep-wake parameters

The mean differences and level of agreement (Lin’s CCC) between PSG-derived and actigraphy-derived sleep and wake parameters are displayed in Table 2. The mean activity counts calculated by the automatic threshold varied within the sample (132.6, SD = 44.6). The low threshold displayed the highest correlation with PSG for determining wake time (Lin’s CCC: 0.7, 95% confidence interval [CI]: 0.6–0.9), sleep time (Lin’s CCC: 0.7, 95% CI: 0.5–0.9), and sleep efficiency (Lin’s CCC: 0.6, 95% CI: 0.3–0.8). Actigraphy appeared poorly to estimate sleep latency and number of awakenings across all 4 thresholds. There were large standard deviations for the mean differences between actigraphy and PSG, suggesting that there was variability in the estimates produced by actigraphy when compared with PSG. For almost half of the sample, the mean difference between actigraphy (employing the low threshold) and PSG was outside these specified ranges (> 30 minutes)⁴⁶ for both wake (n = 11 patients/45.8%) and sleep time (n = 13 patients/54.2%).

Table 2.

Correlation between PSG and actigraphy for sleep and wake parameters.

Statistic	Wake Time (minutes)	Sleep Time (minutes)	Sleep Efficiency (%)	Sleep Latency (minutes)	Number of Awakenings
PSG
Mean (SD)	84.8 (60.7)	340.2 (76.5)	71.6 (14.5)	38.6 (32.8)	26.2 (14.0)
Actigraphy Threshold
Low
Mean (SD)	79.6 (40.4)	366.4 (73.7)	76.8 (12.2)	26.0 (33.9)	32.9 (13.5)
Mean difference from PSG (SD)†	5.2 (38.3)	−26.3 (56.6)	−5.1 (11.6)	12.6 (39.6)	−6.7 (18.1)
Lin’s CCC (95% CI)	0.7 (0.6–0.9)	0.7 (0.5–0.9)	0.6 (0.3–0.8)	0.3 (−0.1–0.6)	0.1 (−0.2–0.5)
Medium
Mean (SD)	57.4 (34.0)	390.2 (70.2)	81.5 (10.8)	26.0 (34.0)	30.2 (12.9)
Mean difference from PSG (SD)	27.5 (41.6)	−50.0 (57.2)	−9.9 (11.5)	12.6 (39.3)	−4.0 (18.2)
Lin’s CCC (95% CI)	0.6 (0.4–0.8)	0.6 (0.3–0.8)	0.5 (0.2–0.7)	0.3 (−0.1–0.6)	0.1 (−0.3–0.5)
High
Mean (SD)	38.3 (27.2)	413.1 (67.9)	86.3 (8.9)	22.6 (31.5)	24.0 (10.9)
Mean difference from PSG (SD)	46.6 (48.5)	−72.9 (57.9)	−14.6 (11.4)	16.1 (34.8)	2.2 (17.0)
Lin’s CCC (95% CI)	0.3 (0.1–0.5)	0.4 (0.2–0.7)	0.3 (0.1–0.5)	0.4 (0.0–0.7)	0.1 (−0.3–0.5)
Automatic
Mean (SD)	28.3 (22.2)	427.2 (72.6)	89.1 (9.0)	18.9 (30.7)	22.9 (20.2)
Mean difference from PSG (SD)	56.5 (47.0)	−87.0 (59.9)	−17.5 (11.5)	19.8 (33.1)	3.4 (25.2)
Lin’s CCC (95% CI)	0.3 (0.1–0.4)	0.4 (0.2–0.6)	0.3 (0.1–0.4)	0.4 (0.1–0.7)	−0.1 (−0.4–0.3)
Satisfactory a priori differences‡	≤ 30 minutes	≤ 30 minutes	≤ 5%	–	–

†Mean difference between PSG and actigraphy. Positive values (actigraphy underestimates) and negative values (actigraphy overestimates) the variable. ‡Prior research defined differences (a priori) between actigraphy and PSG that are considered satisfactory.⁴⁶ CI = confidence interval, Lin’s CCC = Lin’s Concordance Correlation Coefficient, PSG = polysomnography, SD = standard deviation.

Bland-Altman plots comparing PSG and actigraphy (low threshold) agreement

The Bland-Altman plots in Figure 2 display the agreement between PSG and the Actigraph device’s low threshold for measuring wake time and sleep time through plotting the difference in score between PSG and actigraphy. The plots suggest that actigraphy demonstrated some agreement with PSG for both variables, with the majority of patients residing within the limits of agreement (upper and lower dotted lines). However, there appears to be considerable variability in the performance of actigraphy, with each patient (dots) spread both above and below the mean bias line. This indicates that actigraphy both overestimates (dots below the mean bias line) and underestimates (dots above the mean bias line) the parameter in question. It also appears that as patients’ mean wake time increases, the disparity between actigraphy (low threshold) and PSG also increases. This pattern suggests that actigraphy particularly tends to underestimate wake time for patients displaying increased sleep disturbance (ie, patients are awake for more time than actigraphy reports). The Bland-Altman plot for sleep time displayed no specific patterns. These results were further investigated through RMAR, which correspondingly suggested proportional bias for wake time (slope = 1.5; intercept = 35.0) (bias or discrepancy increases for higher values of wake time) and fixed bias for sleep time (slope = 1.0; intercept = −40.1) (bias remained constant). For the medium, high, and automatic thresholds, the Bland-Altman Plots displayed larger differences between PSG and actigraphy for wake time with each increase in threshold setting (wake threshold value) (Figure S1 ^{(190.1KB, pdf)} in the supplemental material). This can be seen with more observations outside the limits of agreement. Higher thresholds generally required more movement to be scored as wake, and so it is not unexpected that higher threshold settings were less sensitive to wake in our hospitalized patient population. Similar to the low threshold, the other thresholds displayed no specific patterns for sleep time (Figure S2 ^{(190.1KB, pdf)} ).

Figure 2. Bland-Altman plots of PSG and actigraphy (low threshold) measurement for (A) wake time and (B) sleep time.

Each dot represents a single patient within the analytic sample. The horizontal x-axis displays the mean of the 2 measures (eg, mean of PSG wake time and actigraphy wake time) and the vertical y-axis displays the difference between the 2 measures (eg, PSG wake time minus actigraphy wake time). The solid horizontal line represents the mean of the differences (mean bias) between PSG and actigraphy. Mean differences that are positive values (dots above the mean bias) signify that actigraphy underestimates the parameter and the reverse (overestimates) for negative mean difference values (dots below the mean bias). Perfect agreement between both measures for the given variable would display patients (dots) residing exactly on the mean bias line. The dotted lines reflect the 95% limits of agreement for the mean bias. PSG = polysomnography.

Sensitivity, specificity, and accuracy of actigraphy—epoch-by-epoch agreement with PSG

Actigraphy and PSG both provide a binary score for each individual epoch (sleep or wake). There was a mean of 817.4 (SD = 180.8) epochs per patient. The sensitivity or positive agreement, specificity or negative agreement, and overall agreement or accuracy of the 4 actigraphy thresholds are shown in Table 3. Sensitivity (correctly scored epochs classified by PSG as sleep) ranged from 85.1% to 95.6%. The automatic threshold was individually tailored to each patient’s mean activity value and displayed the highest sensitivity (sleep detection). However, actigraphy demonstrated low specificity (correctly scored epochs classified by PSG as awake) across its wake threshold settings (ranging from 17.1% to 36.6%). While the overall accuracy of actigraphy (all correctly identified sleep and wake epochs) was reasonable and relatively similar across the thresholds (ranging from 71.1% to 72.8%), this should be interpreted with caution in this sample, as it appears to be driven by high sensitivity or positive agreement. As expected, the low threshold setting, which required less movement to score as sleep, yielded the highest specificity (36.6%), albeit still low. Sensitivity, specificity, and overall agreement comparisons were made between the Actigraph device’s low and medium threshold using binomial regression, as described previously. The medium threshold had a slightly higher sensitivity or ability to detect sleep periods compared with the low threshold (relative risk [RR] = 1.05, 95% CI = 1.04–1.1, P < .001). However, the medium threshold had lower specificity or ability to detect wake periods than the low threshold (RR = 0.8, 95% CI = 0.7–0.8, P < .001). Overall, the medium threshold demonstrated a slightly increased agreement compared with the low threshold (RR = 1.01, 95% CI = 1.001–1.03, P = .025).

Table 3.

Sensitivity, specificity, and accuracy between actigraphy compared to PSG – epoch-by-epoch comparison agreement.

Statistic	Sleep Sensitivity* (True positive)	Wake Specificity† (True negative)	Overall Accuracy‡
Actigraphy Threshold
Low
% (95% CI)	85.1 (81.3–89.0)	36.6 (26.6–46.5)	71.1 (65.4–76.7)
Epochs (number) correctly identified by actigraphy	11,850	2,077	13,927
Total number of epochs	13,919	5,681	19,600
Medium
% (95% CI)	89.7 (86.5–92.9)	29.1 (20.1–38.0)	72.1 (66.3–77.9)
Epochs (number) correctly identified by actigraphy	12,486	1,657	14,143
Total number of epochs	13,921	5,697	19,618
High
% (95% CI)	93.7 (91.4–95.9)	20.9 (13.0–28.7)	72.5 (66.6–78.4)
Epochs (number) correctly identified by actigraphy	13,038	1,189	14,227
Total number of epochs	13,921	5,697	19,618
Automatic
% (95% CI)	95.6 (93.8–97.3)	17.1 (10.6–23.6)	72.8 (67.0–78.6)
Epochs (number) correctly identified by actigraphy	13,306	973	14,279
Total number of epochs	13,921	5,697	19,618

*Sleep Sensitivity calculated as total epochs correctly scored as sleep by actigraphy (true positive)/total number of epochs scored as sleep by PSG. †Wake Specificity calculated as total epochs correctly scored as wake by actigraphy (true negative)/total number of epochs scored as wake by PSG. ‡Overall accuracy calculated as: (sum of true positive and true negative epochs)/total number of epochs. CI = confidence interval, PSG = polysomnography.

Actigraphy (low threshold) sensitivity by sleep stage

Table 4 displays the influence of sleep stages on the sensitivity of actigraphy (utilizing its low threshold setting). While the Philips Actigraph devices do not provide an estimate of sleep architecture (staging) for each epoch of data scored as sleep, PSG will further identify the specific stage of sleep (ie, N1, N2, N3, and R). These data were utilized to determine the sensitivity of actigraphy (total epochs correctly scored as sleep by actigraphy), categorized by the sleep stages as identified by PSG. Overall, the sensitivity of actigraphy appeared to be highest (90.5%) for N3 (deep, slow sleep) and lowest (79.1%) for N1 sleep. This suggests that as patients transitioned to deeper stages of sleep, actigraphy was better able to detect epochs as sleep. However, for the earlier, transitioning period from wake to sleep (N1), actigraphy was less likely to detect these stages as sleep. Binomial regression indicated that there was a statistically significant overall difference in sensitivity between the 4 sleep stages (likelihood ratio chi-square χ² (3) = 12.8, P = .005). Using N1 as the reference category, actigraphy was better able to detect sleep when patients are within N2 or N3 sleep compared with N1. As shown in Table 4, N3 has a significantly higher sensitivity (RR = 1.1) than N1 (95% CI = 1.05–1.2, P = .002), and N2 similarly has a significantly higher sensitivity (RR = 1.1) than N1 (95% CI = 1.02–1.1, P = .01).

Table 4.

Actigraphy (low threshold) sensitivity by sleep stage.

Sleep Stage	Actigraphy Low Threshold Sleep Sensitivity†	Relative Risk‡	P	95% CI
N1 (Reference Category)
% (95% CI)	79.1 (73.5–84.7)	–	–	–
Epochs (number) correctly identified by actigraphy	1,259
Total number of epochs	1,592
N2
% (95% CI)	85.2 (81.4–88.9)	1.1	.010*	1.02–1.1
Epochs (number) correctly identified by actigraphy	7,343
Total number of epochs	8,623
N3
% (95% CI)	90.5 (83.7–97.4)	1.1	.002*	1.05–1.2
Epochs (number) correctly identified by actigraphy	1,549
Total number of epochs	1,711
R
% (95% CI)	85.2 (78.5–92.0)	1.1	.094	.99–1.2
Epochs (number) correctly identified by actigraphy	1,699
Total number of epochs	1,993

†Sleep Sensitivity calculated as total epochs correctly scored as sleep by actigraphy (low threshold)/total number of epochs scored as sleep by PSG. ‡ N1 = reference category for binomial regression. *Significantly different at P < .05. CI = confidence interval, N1 = nonrapid eye movement stage 1, N2 = nonrapid eye movement stage 2, N3 = nonrapid eye movement stage 3, R = rapid eye movement.

DISCUSSION

This study demonstrated that there was poor concordance between actigraphy and gold-standard PSG for determining sleep and wake parameters in this patient group. The Philips wake threshold settings, which each apply a different wake threshold value, displayed poor agreement (CCC < 0.8) with PSG across all 5 parameters, consistent with our hypothesis. There is also considerable variability in the estimates produced by actigraphy, with it both overestimating and underestimating parameters. The difference between actigraphy and PSG appears to be more pronounced in patients with poorer, more disturbed sleep, indicated by increased wake time during sleep periods. While actigraphy displayed reasonable sensitivity (its ability to detect sleep), this was offset by reduced specificity (its ability to detect wake), which ranged from 17.1% to 36.6% across the wake threshold settings. These findings support our hypothesis that actigraphy underestimates sleep disturbance (ie, total wake time). In terms of sleep architecture, the low specificity may be driven partly by actigraphy’s inability to detect the early, transition stage between wake and sleep (N1). The low threshold setting generally performed the best of the settings, albeit still had generally poor agreement with PSG.

Importance of accurate detection of sleep disturbance during PTA

Within the last decade or so, there is increasing evidence within limited studies of the exacerbated sleep disturbances experienced during PTA.^1–3,6 This PSG study reported similar disturbances to sleep as per prior research, which were also more disrupted compared with normative values.⁴⁵ Patients commonly exhibited difficulties with initiating and maintaining sleep continuity, which resulted in shortened and disturbed sleep periods. Patients also displayed frequent awakenings, increased wake time during sleep periods, and decreased sleep efficiency. The arousal index was relatively similar to normative data. However, this index alone does not appear to provide an accurate indication of sleep disturbance in our patients given the abnormalities seen to the other sleep and wake parameters. Rather, our study suggests that the arousals may commonly transition into awakenings as indicated by the increased awakenings relative to normative values and therefore wake time. This may partly explain the changes seen to sleep architecture during PTA (eg, specifically reduced deep, slow wave sleep),¹ whereby frequent awakenings are preventing the transition to deep sleep. This study, and others, continue to demonstrate the importance of implementing routine screening of sleep disturbances, which have an early onset during the PTA phase. Actigraphy is certainly convenient, noninvasive, and has been perceived as more tolerable given the clinical presentation of patients in PTA. However, this study suggests that as a stand-alone measure, it may not provide an accurate portrayal of the extent of sleep disturbances experienced during PTA. Study findings instead suggest that actigraphy had poor agreement and low specificity compared with PSG for patients in PTA and should not entirely replace the gold-standard measure. PSG, while challenging, was a feasible and tolerable sleep assessment method during early recovery and PTA. This study found a PSG adherence rate (74.4%) higher than previously reported in a smaller pilot study¹ and is also similar to actigraphy compliance rates in patients following TBI.⁴⁷ The challenges this study encountered regarding PSG compliance were more commonly related to lost or unclear signals (from unattended monitoring) than from patient’s removing the device. PSG still had distinct advantages compared with actigraphy, including directly monitoring sleep architecture using electroencephalogram channels and diagnosis of sleep disorders, which are prevalent in the TBI population,⁴⁸ although additional respiratory channels would be required.

Sleep disturbances have been broadly associated with cognitive impairments, affective disorders, fatigue, pain, and restrictions in function and participation.⁴⁹ In the longer term, research has also reported sleep disturbances for years post injury and the development of sleep disorders in the TBI population.⁴⁸ It is therefore essential to employ measures that accurately identify sleep disturbances during PTA for implementation of appropriate management strategies,¹⁶ to promote earlier patient recovery,⁷ and to potentially reduce the risk of chronic sleep disorders.⁵⁰ This may include a combination of pharmacological treatment (eg, melatonin supplementation) and environmental strategies (eg, sleep hygiene practices, light therapy).

Problems with actigraphy for sleep measurement during PTA

Findings within the current study, and prior research,^9,51 suggest that the performance of actigraphy compared to PSG was influenced by an individual’s degree of sleep integrity. Actigraphy has demonstrated moderate validity and reliability in normal healthy populations,¹¹ who are generally asleep for the majority of the night.¹³ However, actigraphy’s accuracy appears to reduce in clinical situations where there is greater sleep disruption (ie, reduced sleep duration, poorer quality sleep, and increased wakefulness) such as for patients in PTA. Prior studies have reported that actigraphy frequently overestimates total sleep time and/or sleep efficiency across various populations who commonly display sleep disturbance, such as patients with sleep disorders,¹³ insomnia,¹² major depression,¹⁴ and also in patients post TBI.⁹ For example, the TBI study reported that actigraphy reasonably estimated sleep parameters in patients with “good sleep,” described as higher sleep efficiency (> 90%), increased sleep duration (> 7 hours), and decreased wake time.⁹ However, the disparities between both measures increased in those with poorer sleep.⁹ This pattern was also seen in our study, with larger differences between both measures found in patients with increased wake time. The RMAR also suggested proportional bias for wake time, whereby the actigraphy device’s low threshold estimated differing wake times relative to PSG. Actigraphy’s reduced accuracy in those with sleep disturbances may be the result of different signals to indicate sleep-onset (ie, immobility for actigraphy and brainwave activity for PSG).⁵² Sleep onset may begin after an increased period of immobility.⁵² Immobility may also not directly translate to sleep, particularly in those experiencing PTA, with frequent awakenings and who are motionless but still remain awake.⁵³

There are other potential factors that may influence the accuracy of a “proxy” indirect sleep measure (actigraphy), which is dependent on movement/activity. These factors are specific to the characteristics and therefore management of patients in PTA. A key hospital management strategy for patients experiencing PTA is to avoid overstimulation by limiting environmental stimuli (eg, noise, hospital visitors) as this may increase fatigue or agitation.⁵ Accordingly, patients’ mobility is constrained by secured ward sections, and certain agitated, high injury risk patients are managed with Craig beds (a bed with high padded surrounding walls). In this study, two-thirds of patients also sustained orthopedic injuries to upper/lower extremities. Studies have reported inactive or sedentary activity levels in patients in inpatient TBI rehabilitation settings,¹⁰ and patients in PTA also display reduced daytime activity ratios.⁶ Such mobility impairments can lead to actigraphy missing arousals⁵⁴ and regarding periods of quiet wakefulness as sleep.⁵⁵ Actigraphy’s algorithms have also not been validated in specific patient populations,⁵⁶ such as PTA. As can be seen in Figure 1, patients in PTA often display heightened overnight activity, with research suggesting this may affect the standard actigraphy algorithm.¹⁶ Muscle contractions and activity levels that naturally occur during sleep and specific stages also appeared to be an influencing factor. Actigraphy was better able to detect sleep when patients were in N3 or R sleep (where there is reduced muscle activity) and less likely to detect N1 sleep (where there is often muscle contractions and hypnic jerks). In this study, there was also very low agreement between actigraphy and PSG for number of awakenings. This could derive from PSG utilizing cortical arousals and actigraphy relying on body movement to indicate awakenings.

Future recommendation in the assessment of sleep during PTA

Prior TBI studies have commonly employed actigraphy in place of gold-standard PSG. However, this study suggests that using actigraphy alone may largely underestimate the actual extent of sleep disturbances experienced by patients and may therefore affect the treatment strategies offered. This study offers clinical evidence to facilitate developing future screening and assessment protocols during PTA, which are currently not available. Gold-standard PSG is feasible and, where available in clinical settings, is encouraged for more routine use in patients experiencing PTA. The benefit of employing PSG at patient bedside is that it minimizes the “first-night effect” by keeping patients within their typical hospital environment, as opposed to a sleep laboratory setting. In our study, we did not administer PSG on the first night of admission to ensure patients were habituated to the ward environment. While actigraphy cannot entirely replace PSG, it has certain benefits given its simplicity and ability to monitor sleep-wake cycles over prolonged periods, including daytime activity (ie, frequency and duration of daytime napping) that may affect overnight sleep disturbance. However, there are certainly potential research avenues to determine how actigraphy can best be incorporated in this patient group, for instance, as a routine screening tool prior to PSG. Research has suggested that manual scoring of actograms, the graphical representation of actigraphy’s recorded activity data, is superior to other scoring methods.⁸ Manual scoring involves manually determining the commencement and end of a rest period based on activity and light.⁸ However, as can be seen in Figure 1, manually scoring the commencement of bedtime (ie, when activity levels reduce) may be difficult in patients who display restlessness. An alternative may instead be to develop customized, validated algorithms and/or threshold settings for specific clinical groups such as those in PTA. This also removes variability in subjective input that comes with manual coding. It has been suggested that given actigraphy’s tendency to overestimate sleep indices, separate algorithms could be developed for daytime and nighttime periods.¹¹ There was some variability in the actigraphy to PSG concordance (ie, overestimates of sleep time relative to increasing wake time). While this is common in populations with increasing sleep disturbance such as PTA, future studies in larger samples could evaluate the potential impact of other factors on the degree of concordance between both variables (eg, medication and/or environment). The low setting, albeit still with low performance, appears to have a closer resemblance to PSG. Caution should also be exercised in terms of interpreting overall accuracy, which in this study appears to be largely inflated by high sensitivity.

Study limitations

There are some limitations in this study. While it included a small sample, this sample size exceeded other TBI studies which administered PSG during acute recovery.^1,22 There were some challenges with recruiting and administering study measures in patients who were often confused and agitated.¹ Time was also a critical influencing factor, as patients can quickly emerge from PTA as seen in this study. Given the small sample size, we did not investigate other factors (eg, demographic or injury characteristics) that may potentially influence the performance of actigraphy. This is a potential area for future research; however, other studies have demonstrated no relationship between such variables and the precision or recall of actigraphy.⁹ PSG monitoring is also typically for 1 night and is less appropriate for longer term assessment due to more complex monitoring requirements.⁸ This study only used Philips Actiwatch devices, and so, actigraphy devices from other manufacturers may perform differently compared with PSG.

CONCLUSIONS

This study demonstrated that there is generally poor agreement between actigraphy and gold-standard PSG for determining sleep and wake parameters in patients experiencing PTA. In this group of patients with significantly disturbed sleep, actigraphy displayed low specificity, even when employing different wake threshold values. Furthermore, actigraphy exhibited less agreement with PSG for patients with increased wake time. Future studies should be cautious when employing actigraphy alone, which may overestimate the quality of sleep. In PTA, the accurate detection of sleep disturbances may facilitate appropriate management strategies and earlier recovery.

ABBREVIATIONS

CCC

concordance correlation coefficient

CI

confidence interval

N1

non-rapid eye movement stage 1

N2

non-rapid eye movement stage 2

N3

non-rapid eye movement stage 3

PSG

polysomnography

PTA

posttraumatic amnesia

R

rapid eye movement

RMAR

reduced major axis regression

RR

relative risk

SD

standard deviation

TBI

traumatic brain injury

WPTAS

Westmead Post-Traumatic Amnesia Scale

DISCLOSURE STATEMENT

All authors have seen and approved this manuscript. Work for this study was performed at Department of Rehabilitation, Epworth HealthCare, 89 Bridge Road, Richmond, Victoria, 3121, Australia. This study was funded by the Professor Jack Cade AM Intensivist Major Development Grant from the Epworth Research Institute (ERI) and the Epworth Medical Foundation Grant Family Bequest. These grants provided the study equipment and staffing support outside the study team. The authors report no conflicts of interest.