Criterion validity of the Pittsburgh Sleep Quality Index: Investigation in a non-clinical sample

Abstract

The objective of this study was to investigate the reliability and validity of the Pittsburgh Sleep Quality Index (PSQI) in a non-clinical sample consisting of younger and older adults. There has been little research validating the PSQI with respect to multinight recording as with actigraphy, and more validation is needed in samples not specifically selected for clinical disturbance. Also, the degree to which the PSQI scores may reflect depressive symptoms versus actual sleep disturbance remains unclear. One-hundred and twelve volunteers (53 younger and 59 older) were screened for their ability to perform treadmill exercises; inclusion was not based on sleep disturbance or depression. Internal homogeneity was evaluated by correlating PSQI component scores with the global score. Global and component scores were correlated with a sleep diary, actigraphy, and centers for epidemiological studies – depression scale scores to investigate criterion validity. Results showed high internal homogeneity. PSQI global score correlated appreciably with sleep diary variables and the depression scale, but not with any actigraphic sleep variables. These results suggest that the PSQI has good internal homogeneity, but may be less reflective of actual sleep parameters than a negative cognitive viewpoint or pessimistic thinking. The sleep complaints measured may often be more indicative of general dissatisfaction than of any specifically sleep-related disturbance.

INTRODUCTION

Subjective sleep quality is important to researchers and clinicians in that sleep dissatisfaction accompanies a number of physical and mental disorders and is associated with an impaired quality of life.^1,2 Most surveys of sleep disturbance do not adequately assess sleep quality, but of the few standardized measures that exist, the most widely used is the Pittsburgh Sleep Quality Index (PSQI).¹

The PSQI was designed to differentiate between “good” and “poor” sleepers, and to distinguish between subgroups of poor sleepers. Additionally, the PSQI was designed to be an assessment that is brief and easy to interpret, yet describes a wide array of sleep problems.¹

The PSQI is a 24-item scale that measures sleep disturbances along 7 dimensions: subjective sleep quality, sleep latency, sleep duration, habitual sleep efficiency, sleep disturbances, use of sleep medication, and daytime dysfunction. Scores from these seven areas are added together into a global score. Responses are based on the majority of days (and nights) of the previous month.¹

Psychometric evaluation of the PSQI has been quite limited despite its proliferation in research and clinical practice.^3–6 Test construction and item development, as well as reliability and validity analyses, were reported with the original publication of the measure.¹ Initial analyses, conducted among a group of controls, depressed patients and sleep-disordered patients (who complained of disorders of initiating or maintaining sleep or disorders of excessive somnolence) showed that, overall, the measure had high internal homogeneity, internal consistency, and test–retest reliability. The primary assessment of discriminative validity was an analysis of covariance (ANCOVA) with age and sex as covariates. Controls differed from depressives, DIMS and DOES patients on global PSQI scoring; DIMS and depressives also had significantly higher scores than DOES. Using a cut-off score of 5, the PSQI was able to correctly identify 88.5% of all patients and controls, representing a 89.6% sensitivity and a 86.5% specificity.¹

Further, concurrent validity was explored by comparing PSQI scores to polysomnographic (PSG) measures of sleep.¹ Groups differentiated by PSQI scores were also differentiated by PSG measures of sleep latency, sleep efficiency, sleep duration and number of arousals. However, PSQI scores did not correlate well with their PSG counterparts, with the strongest relationship between PSQI and PSG sleep latency being only r = 0.3.

Further psychometric evaluations were conducted with medical samples.² Findings suggested a high internal consistency, and a moderately high internal homogeneity. Additionally, high correlations with sleep-related items and subscales on other measures established criterion validity. Convergent and discriminant validity were assessed by demonstrating strong correlations with items related to sleep disturbance (e.g. depression and energy) and low correlations with measures of unrelated variables (e.g. nausea).

Gentili et al.⁷ investigated test–retest reliability in nursing home residents, showing high correlations between global and component scores on two administrations. Additionally, high test–retest reliability was demonstrated in patients with primary insomnia.⁸

Discriminative validity has been established by numerous studies that have used PSQI scores to differentiate groups in predicted ways. All of these groups are thought to represent poor sleepers and have been discriminated as such by the PSQI global score. Among the groups that have been studied using the PSQI are chronic pain patients,^5,6 sleep disorders patients,^4,9,10 cancer patients,¹¹ irritable bowel syndrome patients,¹² depressed patients,^13–15 family caregivers,¹⁶ fibromyalgia patients,¹⁷ blind patients,¹⁸ HIV patients,^19–21 Holocaust survivors,²² elderly people^7,23 and patients with traumatic brain injury.³ It is uncertain, however, what the common element of “poor sleep quality” might be among these diverse groups, although all may tend to be depressed.

The present study is a further investigation into the criterion validity of the PSQI. Global and component scores were correlated with objective sleep measures (wrist actigraphy), other subjective sleep measures (sleep diary), and a measure of depression (centers of epidemiological studies – depression scale [CESD]).²⁴ Actigraphy is recommended and commonly used for the estimation of objective sleep when multinight recording is needed,^25,26 and although there has been a great deal of research utilizing the PSQI, multinight validation using actigraphy has not been explored. Also, validation in a sample not specifically selected for sleep disturbance has not been explored, despite the prevalence of the PSQI as a screening measure in sleep-disturbed and non-sleep-disturbed individuals. Finally, the degree to which the sleep complaints reported on the PSQI reflect a negativistic cognitive viewpoint associated with depression versus actual sleep disturbance is still unclear.

METHODS

Participants

Subjects were recruited as part of a larger laboratory study contrasting circadian phase response curves in younger and older adults. Included in the present analysis were the first consecutive 112 subjects, consisting of a group aged 18–32 years (n = 59), with a mean age of 23 (SD = 3.9) and an older group aged 59–75 years (n = 53), with a mean age of 66 (SD = 4.8). The younger group was 69.5% female and consisted of 11.9% Asian, 5.1% African-American, 22.0% Hispanic, and 61% white participants. The older group was 64.2% female and consisted of 5.7% Asian, 1.9% African–American, 1.9% Hispanic, and 90.6% white participants. These subjects were screened for ability to perform treadmill exercises, but selection was not focused on those with sleep disturbance or depression. Thus, a wide range of scores was expected.

Procedure

Subjects completed the PSQI as part of an initial screening. At a later date (median = 66 days; percentiles = 37.25 (25th), 108.00 (75th); range 4–228) a daily sleep diary was maintained for 1 week and a wrist actigraph was worn to objectively record sleep. At the end of this week, the CESD was administered.

Materials

Actigraphy

Objective measures of sleep were recorded for a week using an Actillume (Ambulatory Monitoring, Inc., Ardsley, NY). The Actillume is a wrist monitor that contains a photometer to measure light exposure and an accelerometer to measure movement activity. Epochs of 1 minute of light exposure and movement were transferred to a computer and analyzed using ACTION3 software (Ambulatory Monitoring, Inc., Ardsley, NY). Actigraphic sleep estimation has been repeatedly validated and has demonstrated reliability in a variety of situations, including those of the present study.²⁶

From sleep log data and the illumination and activity recordings, bedtime was precisely determined. Sleep in bed was scored by validated algorithms,^27,28 supplemented by hand-editing of each record. Although compliance issues were minimal, not all subjects wore the Actillume for a full 7 days. The weekly averages were obtained from a mean of 6.27 days (SD = 1.0).

Sleep diary

During the actigraphic recordings, subjective measures of sleep variables were obtained using a daily sleep diary. These sleep diaries were to be filled out every morning and evening for 7 days, reporting time in bed, time asleep, awakenings, naps, lights, use of sleeping aids, etc. For the present analyses, only diary measures of sleep efficiency, total sleep time, sleep latency and wake-after-sleep onset were investigated. These values were averaged across all 7 nights.

CESD

The CESD score²⁴ was completed at the end of the actigraphic recording week.

Data analyses

Power analyses demonstrated that the present study had 80% power to detect a correlation of 0.33 with a one-tailed Spearman's rho. Age group differences were determined using the Mann–Whitney U-test, due to the ordinal nature of many of the scores, as well as the skewed distributions. In order to demonstrate the validity of correlating the PSQI with the data gathered at times varying from 4 to 228 days later, the sample was divided into two groups: those with a time span of 31 days or less (21 subjects, mean 18 days) and those with a time span of greater than 31 days (91 subjects, mean 88 days). The PSQI was correlated with the actigraph, sleep diary and CESD measures separately in these groups to see if correlations deteriorated over time.

To examine internal homogeneity, the PSQI component scores were correlated with the global score. Additionally, to investigate criterion validity, PSQI component and global scores were correlated with weekly averages of actigraphic measures of sleep latency, total sleep time, wake-time-after-sleep onset and sleep efficiency. Weekly averages of sleep diary measures were also taken: sleep latency, total sleep time, wake-time-after-sleep onset, sleep efficiency, and CESD total score. Additionally, sleep diary measures were correlated with actigraphic measures. All correlations were computed using Spearman's rho, due to the non-normal distribution of the data.

RESULTS

Group differences

Age-group differences are reported in Table 1. Although actigraphic measures, the sleep diary and CESD scores distinguished between the old and young groups with P = 0.01, neither PSQI global score nor component scores significantly distinguished the age groups.

Table 1.

Means and interquartile ranges for PSQI, actigraphic sleep, sleep diary variables, CESD

		Younger		Older
Variable	Mann–Whitney U-test	Mean	Interquartile range	Mean	Interquartile range
PSQI
Global Score	1344.5	4.07	3.00	3.92	5.00
Component 1 – Subjective sleep quality	1354.0	0.69	1.00	0.59	1.00
Component 2 – Sleep latency	1343.0	0.85	1.00	0.69	1.00
Component 3 – Sleep duration	1462.5	0.42	1.00	0.53	1.00
Component 4 – Habitual sleep efficiency	1392.5	0.35	1.00	0.55	1.00
Component 5 – Sleep disturbances	1537.5	1.05	<0.01	1.02	<0.01
Component 6 – Use of sleeping medication	1520.0	<0.01	<0.01	0.14	<0.01
Component 7 – Daytime dysfunction	1363.5	0.60	1.00	0.47	1.00
Actigraphy
Sleep efficiency (%)	243.0**	88.70	5.97	76.88	10.17
Total sleep time (min)	677.0**	418.48	69.07	367.81	59.57
Wake-after-sleep onset (min)	337.5**	44.54	27.40	97.16	44.10
Sleep latency (min)	817.5**	7.42	4.26	10.89	5.51
Sleep diary
Sleep efficiency (%)	1102.0*	94.32	0.05	88.49	0.17
Total sleep time (min)	1133.5*	433.29	89.43	402.21	94.29
Wake-after-sleep onset (min)	1014.0**	10.22	14.00	34.57	55.00
Sleep latency (min)	1507.5	16.20	12.43	19.16	15.50
CESD
Total score	950.5**	8.75	9.00	3.84	5.00

^*Age group differences significant at the P = 0.01 level;

^**age group differences significant at the P = 0.001 level.

Overall scores

PSQI global and component scores are reported in Table 1 for both age groups. The mean PSQI global score for the younger group was 4.07 and was 3.92 for the older group; 34.5% (35% of the younger group and 34% of the older group) met the proposed cut-off of 5 for poor sleepers.¹ Mean values for actigraphic data, sleep diary variables and CESD are reported in Table 1 for both age groups. Regarding the cut-off of 16 for the CESD, only one of the older group scored 16, but 12% of the subjects in the younger group (n = 7) had CESD scores between 16 and 29.

Correlations between homologous sleep diary and actigraphic variables showed varying degrees of agreement between diary and actigraphic measures. When the two groups were combined, correlations were moderate to high for sleep efficiency (rho = 0.336, P < 0.0005), total sleep time (rho = 0.600, P < 0.0005) and wake after sleep onset (rho = 0.371, P < 0.0005). Actigraphic and diary estimates of sleep latency were not significantly related. In the older group, this pattern was generally maintained, with significant relationships found for sleep efficiency (rho = 0.292, P < 0.05), total sleep time (rho = 0.288, P < 0.05) and sleep latency (rho = 0.292, P < 0.05) and a trend for wake-after-sleep onset (rho = 0.231, P = 0.051). In the younger group, actigraphy and the diary strongly agreed on total sleep time (rho = 0.802, P < 0.0005) but no significant correlations were found for other variables.

Internal homogeneity

The PSQI global score was correlated with all PSQI component scores at the 0.0005 level, except for component six (use of sleeping medication), which was not significantly correlated with global score at all. The correlations ranged from rho = 0.372 (sleep disturbances in the older group) to rho = 0.804 (sleep latency in the older group), except for use of sleeping medication, which produced a rho of 0.234 in the younger group and 0.178 in the older group. There was a very low rate of sleeping medication use in this sample. Since 102 of the 113 subjects did not report any use of sleeping medications (and all the rest but one only reported using them less than once per week) in the past month, there was little variance.

Correlations with PSQI global score

Considering objective actigraphic measures of sleep, the PSQI global score did not correlate significantly with actigraphic measures of sleep latency, total sleep time, wake-time-after-sleep onset or sleep efficiency in either the younger or older groups (see Table 2). The PSQI global score was, however, correlated highly with CESD total score in the older group and with sleep diary scores. In the older group, the PSQI global score significantly correlated with sleep diary averages of sleep efficiency, sleep latency, total sleep time and wake-time-after-sleep onset. In the younger group, only the correlation with sleep latency was significant.

Table 2.

Spearman's rho correlations of PSQI global and component scores with Actillume measures, sleep diary and CESD

	Actigraphy				Sleep diary				CESD
Group	Sleep efficiency	Total sleep time	Wake-after-sleep onset	Sleep latency	Sleep efficiency	Total sleep time	Wake-after-sleep onset	Sleep latency	Total score
Global score
Combined (n = 112)	0.046	0.130	0.011	0.022	−0.562**	−0.307**	0.262**	0.480**	0.305**
Younger (n = 59)	0.018	0.115	−0.006	0.090	−0.220	−0.084	0.034	0.349**	0.170
Older (n = 53)	−0.122	0.104	0.198	0.074	−0.764**	−0.548**	0.561**	0.557**	0.364**
Component 1 – subjective sleep quality
Combined (n = 112)	0.034	0.160	0.002	0.007	−0.432**	−0.240*	0.210*	0.354**	0.153
Younger (n = 59)	−0.117	0.035	0.094	0.111	−0.091	−0.006	−0.006	0.185	−0.018
Older (n = 53)	−0.038	0.232	0.135	0.019	−0.707**	−0.581**	0.542**	0.511**	0.281*
Component 2 – sleep latency
Combined (n = 112)	0.086	0.275**	−0.005	0.038	−0.378**	−0.101	0.241*	0.488**	0.193*
Younger (n = 59)	0.034	0.303*	0.063	0.007	−0.064	0.093	0.070	0.432**	−0.047
Older (n = 53)	−0.161	0.152	0.252	0.196	−0.644**	−0.417**	0.555**	0.560**	0.396**
Component 3 – sleep duration
Combined (n = 112)	−0.126	0.204*	0.094	0.039	−0.473**	−0.454**	0.235*	0.275**	0.205*
Younger (n = 59)	0.009	−0.275*	−0.101	−0.018	−0.136	−0.331*	0.049	0.043	0.224
Older (n = 53)	−0.144	−0.087	0.137	0.034	−0.631**	−0.581**	0.530**	0.464**	0.331*
Component 4 – habitual sleep efficiency
Combined (n = 112)	−0.103	−0.053	0.106	0.085	−0.563**	−0.411**	0.260**	0.242*	−0.005
Younger (n = 59)	0.250	0.032	−0.251	−0.021	−0.009	−0.131	−0.114	−0.081	0.045
Older (n = 53)	−0.226	0.024	0.233	0.115	−0.789**	−0.642**	0.611**	0.495**	0.101
Component 5 – sleep disturbances
Combined (n = 112)	−0.032	−0.051	0.054	0.032	−0.034	−0.070	0.040	0.156	0.120
Younger (n = 59)	−0.071	−0.080	0.038	0.079	−0.155	−0.160	0.071	0.367**	0.141
Older (n = 53)	−0.017	0.045	0.166	0.020	0.027	0.078	0.012	−0.093	0.148
Component 6 – use of sleeping medication
Combined (n = 112)	0.126	0.090	−0.119	−0.070	−0.199*	−0.005	0.114	0.146	0.026
Younger (n = 59)	0.143	0.119	−0.079	−0.158	−0.081	−0.032	0.105	0.224	0.057
Older (n = 53)	0.247	0.067	−0.252	−0.106	−0.212	0.040	0.085	0.105	0.032
Component 7 – daytime dysfunction
Combined (n = 112)	0.033	0.044	−0.008	−0.029	−0.307**	−0.168	0.105	0.206*	0.317**
Younger (n = 59)	−0.200	−0.155	0.144	0.065	−0.322*	−0.097	0.063	0.103	0.187
Older (n = 53)	0.011	0.190	0.045	0.037	−0.430**	−0.329*	0.268	0.312*	0.421**

^*P < 0.05;

^**P < 0.01.

Separate correlations were computed for subjects for whom the time span between the PSQI and sleep log and actigraphic variables was less than and greater than 31 days. Correlations were not consistently higher when the time span between the two was less than 31 days.

Correlations with PSQI component scores

Correlations between PSQI component scores and actigraphic sleep, sleep diary and CESD variables are also reported in Table 2. Many of the PSQI components correlated with homologous sleep diary scores, and sometimes, the CESD correlated as well or better. Surprisingly, there were no significant correlations between PSQI components and actigraphic data, except a significant correlation of the total sleep time with the PSQI Sleep Duration component scale and a positive correlation of the total sleep time with the PSQI Sleep Latency component scale.

DISCUSSION

The present study examined criterion validity of the PSQI by examining correlations between global and component PSQI scores with objective estimates of sleep (gathered from the Actillume), subjective estimates of sleep (gathered from sleep diaries) and self-reported ratings of depression (using CESD). Two groups were examined: a younger group (age 18–32 years) and an older group (age 59–75 years), both selected for good physical health and aerobic fitness. Both groups had a strong minority meeting PSQI criteria for poor sleep quality.

Reliability of the PSQI

Reliability was examined by correlating PSQI component scores with the global score. The PSQI demonstrated reasonably good internal homogeneity, although the use of sleeping medication scale had poor sensitivity in this sample. Unfortunately, the high homogeneity may suggest a lack of specificity among the PSQI subscales.

PSQI and sleep diary

Overall, the significant correlations between the PSQI global score and sleep diary measures suggested appreciable criterion validity, but the change in this relationship that is seen when age groups were examined separately revealed that the relationships were seen to a lesser extent in younger adults. This is consistent with recent findings⁸ that indicate that in a middle-aged sample, global PSQI score correlated highly with sleep diary measures of total sleep time and sleep latency. In the younger group, the only PSQI components that correlated with any sleep diary variables were sleep duration (which only correlated with the sleep diary counterpart total sleep time), sleep latency (which also only correlated with the sleep diary counterpart sleep latency) and sleep disturbances (which only correlated with the sleep diary counterpart sleep latency). Moreover, PSQI components had similar correlations to each of the sleep diary measures, suggesting a lack of specificity in PSQI component scales, sleep diary components, or both. Correlations between the PSQI and sleep diary variables describing similar information have not previously been investigated. Although the sleep diary and PSQI are both subjective self-report measures of sleep, the present investigation shows that these instruments may not be redundant in the measurement of the same constructs.

PSQI and Actillume

The present study examined correlations between the PSQI and objective estimates of sleep which were averaged over a week to reduce effects of daily variation. The present data were consistent with previous findings of poor correlations of PSQI with objective sleep measures,^1,8 in that the global score did not significantly correlate with any actigraphic sleep measurements in either age group. At the level of component scores, the PSQI sleep latency component correlated significantly (but anomalously) with actigraphic total sleep time in both age groups, and the PSQI sleep duration component correlated significantly with actigraphic total sleep time in only the younger group. No other significant relationships were found. It is interesting that PSQI correlated better with a depression scale and subjective sleep complaints than with objectively observed sleep disturbances.

PSQI and CESD

It has been previously reported that PSQI global scores correlate significantly with measures of depression, including the CESD.² Additionally, depressed subjects have been shown to significantly differ from controls on global PSQI score, but have not been shown to differ from non-depressed subjects with disorders of initiating or maintaining sleep. This was regarded as a strength of the PSQI, due to the presumed common sleep quality disturbance present in depression.¹

Although the presence of physiological sleep disturbance is characteristic of depression, there may be an alternate explanation that is supported by these previous findings, as well as the present data. Beck's classic model of depression²⁹ proposes that depressed individuals are biased in their perceptions of themselves, which may extend to negatively biased perceptions of their physiological state (e.g. somatization). Perhaps subjective sleep quality measures such as the PSQI better detect this negative cognitive viewpoint than the types of sleep disturbances observed with actigraphy.

The present data are consistent with previous findings, in that the CESD total score correlated significantly with PSQI global, as well as component scores of subjective sleep quality, sleep latency, sleep duration, habitual sleep efficiency and daytime dysfunction in the older group. These results suggest that the PSQI may partially reflect the negative cognitive viewpoint and pessimistic thinking characteristic of depression, without great specific relevance to objectively observed aspects of sleep. In this respect, the PSQI may describe typical sleep complaints, which may often be more indicative of a negative viewpoint and general dissatisfaction than of any objectively definable sleep disturbance. It is notable that the PSQI was not able to distinguish the well-known disturbances of sleep accompanying aging, which were well-documented by the actigraphic recordings and sleep logs. This would tend to support the view that the PSQI is more a measure of negative attitude and dissatisfaction than physiological sleep quality, as measured with recording devices such as actigraphy.

Limitations and future directions

There are several limitations to this study. First, the younger and older samples were not specifically selected for validation of a self-report measure of sleep disturbance, nor were they selected to represent any clinical group. Thus, scores that indicate sleep problems or problems with depression were fairly low and do not represent a full range of possible scores. We believe that psychometric assessment in this sample was useful however, as the PSQI is routinely used as a broad screening measure in a variety of populations that, like this one, may or may not complain of sleep disturbance. In addition, the fact that a third of unusually healthy people met PSQI criterion of 5 for poor sleep quality throws the specificity of that criterion into doubt.

Second, the older group may not be a sample that is easily generalizable to the older population. These subjects were specifically selected to be physically fit and active; these traits are not very common in the general population over the age of 60 years.

Third, the time period measured by the PSQI and that measured by the sleep diary, CESD and actigraphy were quite different. The PSQI was based on the month prior to its completion. The sleep diary, actigraphic recording, and the CESD were focused on a week that began a median of 66 days later. While this may be the most substantial limitation to this study, we believe the data retain their usefulness. We could not demonstrate consistently better correlations for those subjects studied over shorter time spans versus the longer time spans. Additionally, the time span was not too great to demonstrate correlations of PSQI with sleep diary measures and CESD, measured at the same time as actigraphy.

Fourth, there may be some overlap between the CESD and the PSQI item content, since the CESD contains one item that assesses sleep. However, correlations of the PSQI and CESD (minus that item) yielded similar results. It is important to note that the CESD was validated as a depression measure and not an accurate measure of sleep, and it has not been validated without the inclusion of these items.

Another limitation may be that actigraphy insufficiently measures arousals and sleep architecture which may be related to the PSQI. Actigraphy does monitor arousals and midsleep awakenings, though not as well as it monitors total sleep time. It is true that actigraphy does not detect some brief sleep arousals without movement, but such brief arousals are only scored with considerable difficulty in polysomnography.³⁰ It is likewise true that actigraphy does not distinguish the sleep stages, but it is widely recognized that insomnia and other disturbances of sleep have more to do with total sleep time, time awake in bed, and arousals than they have to do with quantities of each sleep stage.³¹ One should not suppose, for example, that the PSQI measures slow wave sleep.

Further psychometric evaluations should investigate gender differences in reporting of sleep complaints. Also, the implications of cultural differences have not been explored.

While there has been much utilization of the PSQI since its publication over 10 years ago, the question remains whether the PSQI reflects disturbances specific to sleep and distinct from depression.