Actigraphy in the Assessment of Insomnia: A Quantitative Approach

Abstract

Study Objective:

The lack of quantitative criteria for identifying insomnia using actigraphy represents an unresolved limit for the use of actigraphy in a clinical setting. The current study was conducted to evaluate the most efficient actigraphic parameter in the assessment of insomnia and to suggest preliminary quantitative actigraphic criteria (QAC).

Participants and Measurements:

Performing a retrospective study we recovered 408 actigraphic records from 3 sleep measure databases: 2 regarding insomnia patients (n = 126) and one normal sleepers (n = 282). We compared the 2 samples analyzing the following actigraphic sleep parameters: time in bed (TIB), sleep onset latency (SOL), total sleep time (TST), wake after sleep onset (WASO), sleep efficiency percentage (SE%), number of awakenings longer than 5 minutes (NA > 5) and mean motor activity (MA). Moreover, a linear discriminant function (LDF) was developed to identify and combine the most useful actigraphic sleep parameters to separate insomnia patients from normal sleepers. Using Youden index we calculated the preliminary QAC for each actigraphic sleep parameter and for LDF. Receiver operator characteristic (ROC) curves for classifying the accuracy of QAC were performed.

Results:

All sleep parameters recorded by actigraphy significantly differentiated the 2 groups, except TIB. An LDF analysis showed that the most useful combination of actigraphic sleep parameters to assess insomnia was TST, SOL, and NA > 5, which obtained the best ROC and the best balance between positive and negative predictive values compared to any single actigraphic parameter.

Conclusion:

Actigraphy provided a satisfactory objective measurement of sleep quality in insomnia patients. The combination of TST, SOL, and NA > 5 proved the best way to assess insomnia using actigraphy. Acknowledging that the lack of a technological standard and some methodological limitations prevent us generalizing our results, we recommend additional studies on larger populations using different actigraph models.

Citation:

Natale V; Plazzi G; Martoni M. Actigraphy In The Assessment Of Insomnia: A Quantitative Approach. SLEEP 2009;32(6):767–771.

EPIDEMIOLOGICAL DATA INDICATE THAT INSOMNIA IS THE MOST COMMON SLEEP COMPLAINT IN THE INDUSTRIAL WORLD.^1,2 THE FIRST STEP TOWARD EFFECTIVE treatment of insomnia is an initial clinical and instrumental assessment to establish a correct differential diagnosis. Data obtained by history-taking and sleep diaries are often challenging and even confounding as many people with insomnia seriously underestimate how much they really do sleep. A person can sleep well objectively but subjectively feel a poor sleeper (negative misperception) and vice versa (positive misperception).³ Consequently sleep alterations characterizing insomnia are on a continuum, making the diagnostic process very difficult. Thus, an objective assessment of sleep in insomnia is crucially important.⁴

Although polysomnography (PSG) is the accepted gold standard for sleep assessment, there is considerable variability in the sleep of insomniacs, and first night effect may produce results that are not representative of an individual's usual sleep pattern. Therefore, PSG is not indicated for routine evaluation of chronic insomnia unless the signs or symptoms of another sleep disorders are present.^5, For these reasons an alternative means of sleep assessment is desirable. Validation studies against PSG have confirmed that when sleep stage identification is not required actigraphy can provide objective and naturalistic measurements of sleep pattern. Actigraphy allows us to study patients in their own home environment for multiple nights and has a relatively low cost compared to PSG.^6,7 Even if no technological solution has as achieved the status of standard as yet, we know that different commercially available actigraphs have very similar performance.^8–10

In 1995 the American Academy of Sleep Medicine (at that time American Sleep Disorders Association) founded a working group,⁷ known today as Standards of Practice Committee (SPC), to review, among other sleep medicine issues, the current knowledge of actigraphy use and to update guidelines. Since then several other updates have been published.^11,12 The latest one, published in 2007,¹² emphasized that the explosion in the number of studies using actigraphy, together with increased clinical experience in actigraphy use, has led to the inclusion of actigraphy as a possible adjunctive tool for the measurement of sleep duration and sleep patterns for certain sleep disorders in the second edition of the International Classification of Sleep Disorders (ICSD-2),¹³ including: paradoxical insomnia, idiopathic hypersomnia, and circadian rhythm disorders.

One of the unresolved limits of actigraphy is the lack of quantitative criteria for the assessment of sleep quality. Vallières and Morin¹⁴ showed that actigraphy is a reliable method for monitoring treatment response among insomnia patients, i.e., for comparing pre-and post-treatment sleep quality. However, lacking standard quantitative criteria, it is difficult to use actigraphy to aid the diagnosis of sleep disorders, including insomnia.

This limit is partially shared with PSG.¹⁵ Despite the high prevalence and ubiquity of insomnia,^1,2 an attempt to propose specific cut-off values was made only recently,^15,16 and will serve to define the so-called research diagnostic criteria (RDC) for assessing insomnia.¹⁷ This gap is partially due to the wide inter- and intra-individual variability characterizing sleep and subjective sleep perception.

The most defensible quantitative criteria for insomnia are sleep onset latency (SOL) or wake time after sleep onset (WASO) > 30 minutes, occurring ≥ 2 nights per week for > 6 months.^15,18 These criteria, obtained by analyzing sleep diaries, cannot be adopted for actigraphy because actigraphic data are very different from subjective sleep estimation. While actigraphy is a single-channel measurement system that bases its analysis on motor activity, the sleep diary provides a measure of subjective sleep evaluations, which do not always agree with objective evaluations. For example actigraphy systematically underestimates SOL in comparison to sleep logs.^14,19

The best way to obtain quantitative actigraphic criteria (QAC) would be to have the actigraphic data of a large sample of normal sleepers. By comparing actigraphic data of insomnia patients with those of normal sleepers we would be able to suggest actigraphic cut-off values for the assessment of sleep quality. To this end, Ambulatory Monitoring, Inc. (Ardsley, NY) marketed the Activity Data Bank (ADB) software, subsequently included as a utility in Action W-2 software. This software initially allowed several actigraphic data of normal sleepers to be collected in a database. Later, the software automatically accounted for statistical values of the disagreement between a patient's score and the mean of the group of normal sleeper on a 3-level scale: absent, mild or serious. To our knowledge such a procedure has not been sufficiently developed.

The present retrospective study aimed to compare the actigraphic data of insomnia patients with normal sleepers to establish the most efficient actigraphic parameter in assessing insomnia and to put forward preliminary quantitative actigraphic criteria (QAC) for the assessment of insomnia. In addition, we aimed to clarify the usefulness of a specific actigraphic parameter: motor activity (MA). This was in line with the latest recommendation for future research put forward by the Standards of Practice Committee of the American Academy of Sleep Medicine,¹² suggesting an analysis of specific actigraphic parameters which do not have PSG correlation and thus are not usually considered during sleep evaluation. To establish whether a panel of actigraphic parameters rather than a single actigraphic sleep parameter could better differentiate insomnia patients from the control group we performed a multivariate discriminant analysis (MDA).

METHODS

Subjects

To perform this retrospective study we recovered actigraphic records from three anonymous databases. Data collection and database formation were originally performed with informed consent.

Actigraphic recordings of insomnia patients came from the Center of Sleep Medicine of the Department of Neurological Sciences (database A) and the Service for Behavioral Treatment of Sleep Disorders of the Department of Psychology (database B), University of Bologna. We considered only patients with a diagnosis of primary insomnia according to the qualitative criteria of the ICSD-2¹³, and the Research Diagnostic Criteria for Primary Insomnia.¹⁷ Patients who referred nocturnal sleep difficulties with associated daytime impairment ≥ 3 nights a week and for ≥ 6 months were considered. The study complied with recommendations contained in the recently published Recommendations for a Standard Research Assessment of Insomnia.¹⁸ Diagnosis was performed under the supervision of accredited sleep specialists and included semi-structured interviews. Interviews also included questions designed to rule out subjects with narcolepsy, sleep apnea, restless legs and periodic limb movement, psychiatric disorders, sleep-disruptive medical conditions, and use of psychoactive medications or other drugs affecting sleep. To exclude anxious or depressed patients, we also checked recorded Beck Depression Inventory (BDI)²⁰ and State-Trait Anxiety Inventory (STAI)²¹ scores; patients with BDI ≥ 20 or STAI ≥ 40 were excluded. All patients completed a 3-week sleep evaluation protocol; actigraphy was performed during the second week of the evaluation protocol. The final study sample consisted of 126 patients (68 males, 58 females) aged 40.39 ± 14.28 years (range 16–71).

Database C (control group) was compiled at the Laboratory of Applied Chronopsychology of the Department of Psychology, University of Bologna, using a series of different studies^8,22,23 involving healthy subjects. It consisted of 282 subjects (117 male, 165 female) aged 38.51 ± 14.06 years (range 7–65). None worked flexi-time or night shifts and none had complaints of sleep disturbance or daytime symptoms due to unsatisfactory sleep. Exclusion criteria¹⁸ included sleep disorders, mental disorders, serious or acute illness, use of psychoactive medication, and disabilities interfering with or restricting mobility. Subjects had to fill out questionnaires to assess whether they met the inclusion criteria. Specifically they filled out the General Health Questionnaire (GHQ-12),²⁴ the Sleep Disorders Questionnaire (SDQ),²⁵ and the Profile of Mood States (POMS).²⁶ Subjects without complaints of sleep disorders on SDQ and scores of ≤ 4 on GHQ-12 and ≤ 250 on POMS were included.

Actigraph

Basic Mini-Motionlogger, (Ambulatory Monitoring, Inc., Ardsley, NY) was used. The hardware consists of a piezoelectric accelerometer with a sensitivity ≥ 0.01g. The sampling frequency is 10 Hz, and filters are set to 2-3 Hz. Actigraphs were initialized by ACT Millennium software (Ambulatory Monitoring, Inc., Ardsley, NY) for zero crossing mode to collect data in 1-min epochs. Actigraph data files were analyzed by Action W-2 version 3.23 software (Ambulatory Monitoring, Inc., Ardsley, NY). This software identified each epoch as sleep or wake using the mathematical model validated by Cole and coauthors.^27,28 This algorithm computed a weighted sum of the activity in the current epoch, the preceding 4 epochs, and the following 2 epochs as follows: S = 0.0033 (1.06an4 + 0.54an3 + 0.58an2 + 0.76an1 + 2.3a0 + 0.74a1 + 0.67a2); where an4 to an1 were the activity counts from the previous 4 minutes, and a1 and a2 were those related to the following 2 minutes. The current minute was scored as sleep when S < 1. Action W-2 underwent 5 additional re-scoring rules developed by Webster et al.²⁹ The rescoring rules were developed to minimize the tendency of actigraphy to overestimate total sleep time.

All subjects wore the actigraph on the nondominant wrist ≥ 7 consecutive nights³⁰ (mean number of nights = 8.4 ± 1.3 for controls and 7.8 ± 1.1 for insomniacs). Subjects were instructed to push the event-marker button on the device to mark occurrences such as time in and out of bed. During the recorded period, subjects were asked to fill in the sleep log daily within 30 minutes of the last morning awakening.

Using both event-marker points and information present in the diary, automatic scoring was checked by an experienced scorer to set the time spent in bed. If subjects provided only one source of information for a night (e.g., forgot to push the event-marker button), the scorer referred only to source of information provided. If both kinds of information were lacking, the night was not counted.

Sleep Measures

We considered the following sleep measures: the time the subject went to bed and switched off the light (Light Off) and the time the subject last woke up in the morning (Sleep End); time in bed (TIB) (time in minutes, between Light Off and Sleep End); sleep onset latency (SOL) (interval in minutes, between Light Off and sleep start); total sleep time (TST) (sum, in minutes, of all sleep epochs between sleep onset and Sleep End); wake after sleep onset (WASO) (sum, in minutes, of all wake epochs between sleep onset and Sleep End); sleep efficiency percentage (SE%, the ratio of the total sleep time to time in bed multiplied by 100); and the number of wake episodes > 5 minutes (NA > 5). We also considered a specific actigraphic index, i.e., mean motor activity (MA) (mean number of movements within one minute) calculated for TIB.

Statistical Analyses

For every sleep measure, independent sample ttests were performed to compare the insomnia group to the control group. Multivariate analysis based on stepwise multiple linear discriminant analysis (MDA) with the Wilks lambda criterion was used to select the best combination of actigraphic parameters for differentiating the insomnia group from the control group. All of the above sleep parameters were considered in MDA. Most of the variables (SE%, SOL, WASO, NA > 5, and MA) deviate significantly from a normal distribution (tested with the Shapiro-Wilks method), with the deviation being corrected by natural logarithm transformation of the data. The final linear discriminant function (LDF) was applied to the data, and a receiver operating characteristic (ROC)³¹ curve was generated.

Six actigraphic parameters (TST, SOL, WASO, NA > 5, SE%, and MA) were also analyzed singly for their ability to distinguish the insomnia group from the control group. For each actigraphic parameter we generated a ROC curve. Values for the area under the ROC curve (AUC) which graphically depicts the relation between sensitivity and specificity were used as figures of merit. A shared guide for classifying the accuracy of a diagnostic test is: 0.90–1 = excellent; 0.80–0.90 = good; 0.70–0.80 = fair; 0.60–0.70 = poor; < 0.60 = fail. Various cut-off values for each parameter were tested with regard to sensitivity and specificity. Sensitivity was the proportion of accurately classified individuals who reported insomnia. Specificity was the proportion of accurately classified individuals who reported having no sleep problem. The Youden index (i.e., the higher value obtained calculating sensitivity+specificity-1) was used to determine optimal cut-off values (QAC).³²

Sensitivity and specificity are intrinsic features of a test, and they correctly detect the probability of someone being an insomnia patient or a normal sleeper within a known sample. However, sensitivity and specificity do not detect the probability that a positive (or negative) result is really positive (or negative) when analyzing a single subject. For this reason, for each QAC, the positive (the probability that someone who resulted positive with actigraph really had insomnia) and negative (the probability that someone who resulted negative with actigraph was really a normal sleeper) predictive values were also computed.

All statistical analyses were performed by SPSS 14.0 (SPSS, Inc. Chicago, Ill).

RESULTS

As shown in Table 1, all sleep parameters differentiated the 2 groups significantly, except TIB. Insomnia patients went to bed and woke up at the same time as the control subjects; i.e., the 2 samples had the same sleep phase. Insomnia and control groups stayed in bed the same amount of time, but the insomnia group slept significantly less than the control group. The insomnia group showed significantly higher values for SOL, WASO, MA, and NA > 5, and a significantly lower sleep efficiency than the control group.

Table 1.

Sleep Measures (Means and SD) for Insomnia and Control Group

	Control Group	Insomnia Group	t value	p
Light Off	24:03 ± 1:59	23:42 ± 2:25	1.54	n.s.
Sleep End	08:13 ± 1:19	08:02 ± 1:46	1.22	n.s.
TIB	483.28 ± 61.12	476.34 ± 62.49	1.08	n.s.
SOL	9.34 ± 5.44	16.08 ± 16.20	6.25	<0.00001
TST	455.64 ± 57.85	413.42 ± 71.38	6.32	<0.00001
MA	10.94 ± 4.53	16.27 ± 9.51	7.66	<0.00001
NA>5	1.07 ± 1.05	3.33 ± 2.56	12.64	<0.00001
WASO	18.35 ± 13.98	45.20 ± 38.16	10.37	<0.00001
SE%	94.18 ± 3.14	86.35 ± 9.25	12.68	<0.00001

Light Off refers to the moment in which subjects go to bed and switch off the light;

Sleep End refers to the moment in which subjects wake up for the last time in the morning; TIB refers to time in bed (min.);

SOL refers to sleep onset latency (min.);

TST, total sleep time (min.);

MA, mean motor activity (number of movement in one minute);

NA>5, number of awakenings longer than 5 minutes;

WASO, wake after sleep onset (min.);

SE%, sleep efficiency.

The best linear combination of actigraphic sleep parameters was selected by MDA using the minimum Wilks lambda test. The MDA selected the following equation: Discriminant Score (DS) = TST*0.00528 + SOL*−0.04117 + NA > 5*−0.4867 + −1.00665. The classification performance varied across DS and actigraphic sleep parameters; the area under the ROC curve (AUC) ranged from 0.68 to 0.87, showing better performances for DS (0.87) (Figure 1), NA > 5 (0.84) and SE% (0.83) (Table 2). According to the Youden index, the following cut-off values should be optimal and are therefore recommended: SOL = 12 minutes, TST = 440 minutes, MA = 16, NA > 5 = 1.8, WASO = 25 minutes, SE% = 92%, and DS = −0.78. Considering these preliminary QAC, NA > 5 and SE% performed better in negative predictive values (i.e., better efficiency in detecting normal sleepers), whereas DS performed better in positive predictive values (i.e., better efficiency in detecting insomnia patients) (Table 2).

Figure 1.

ROC curves for the combination (Discriminant Score, DS) of TST (total sleep time), SOL (sleep onset latency) and NA>5 (number of awakenings longer than 5 minutes) and for the single actigraphic sleep parameter with the largest ROC curve area (NA>5).

Table 2.

For Each Sleep Measure is Indicated the Value of the Area Under the ROC Curve, and the Cut-Off Value with Relative Sensitivity, Specificity and Positive and Negative Predictive Values

	Area under ROC curve	Cut-off value	Sensitivity	Specificity	Positive predictive	Negative predictive
SOL	0.68	12	0.55	0.81	0.56	0.80
TST	0.68	440	0.66	0.61	0.43	0.80
MA	0.68	16	0.45	0.87	0.61	0.78
NA>5	0.84	1.8	0.71	0.79	0.60	0.86
WASO	0.75	25	0.66	0.75	0.54	0.83
SE%	0.83	92%	0.71	0.78	0.58	0.86
DS	0.87	–0.78	0.64	0.92	0.81	0.83

DISCUSSION

In agreement with previous findings,^33–35 we confirmed the validity of actigraphy in the assessment of sleep quality: all actigraphic sleep parameters significantly differentiated insomnia patients from normal sleepers, except TIB.

By performing ROC analyses we were able to evaluate the efficacy of actigraphic parameters in distinguishing the insomnia group from the control group. The combination of TST (sleep duration is a core concept in sleep medicine), SOL (index of difficulty in the transition form wake to sleep) and number of wake episodes > 5 minutes (NA > 5) (index of sleep interruption) obtained the best performance. Each of these parameters expresses a specific manifestation of insomnia: sleep onset insomnia, maintenance insomnia, and terminal awakening insomnia.¹³ As some patients decided to see a sleep specialist only after several years they often reported all of these symptoms. This probably explains the good performance of the combination TST, SOL, and NA > 5.

Considering the single actigraphic sleep parameters separately, SE and NA > 5 obtained the best ROC results. Sleep efficiency is a parameter that uses wake and sleep actigraphic estimations simultaneously, thus reducing most likely actigraphic limits in evaluating wake and/or sleep, regardless of the type of insomnia.¹⁹ NA > 5 reflects a sleep interruption, i.e., a problem of sleep maintenance, which is probably a constant feature of primary insomnia. It is interesting to note that NA > 5 performed well both alone and in combination with SOL and TST.

With reference to the performance of the preliminary QAC calculated in the present study, we obtained overall higher values for negative predictive values versus those for positive predictive values for each of the single sleep parameters considered. In conclusion, current preliminary QAC for each single actigraphic sleep parameter seem more reliable in recognizing good sleepers than insomniac patients. By contrast, the combination of TST, SOL, and NA > 5 (DS) produced a good balance between positive (0.81) and negative (0.83) predictive values, suggesting this combination would prove reliable in a clinical setting.

This work also focused on a specific actigraphic parameter lacking a corresponding PSG measure: mean motor activity (MA). MA showed a high specificity (0.87) but a low sensitivity (0.45) in detecting insomnia. This result is not surprising, since insomnia patients often complain of staying awake but being immobile in bed. Thus, MA proved a non-primary parameter for diagnosing primary insomnia, but could probably be an important actigraphic parameter to consider in other sleep disorders, in which impaired motor activity is an intrinsic feature (such as periodic limb movement disorder).

By selecting participants based on sleep parameters, we would probably have obtained higher specificity and sensitivity values, but we consider our sample of insomnia patients very similar to those encountered daily in a sleep medicine center.

The present results should be considered with caution, given the methodological limitations in regard to the sample selection. The study is retrospective, meaning we retrieved actigraphic data from databases which adopted different questionnaires in the insomnia cases as compared to those used for normal sleepers. Moreover, patient data come from 2 different database sources. Finally, the results cannot be generalized to apply to different actigraphic models. MA is computed in different ways, depending on the actigraphic model used.

Further work is needed to compare results from different types of insomnia using larger clinical samples like those performed in multicenter studies. We also suggest using a larger cohort to evaluate the age effect on QAC and identify which QAC are stable over a lifetime.³⁶ Additional studies should also compare QAC among different actigraphic models.

DISCLOSURE STATEMENT

This was not an industry supported study. The authors have indicated no financial conflicts of interest.